August 8th, 2012, 08:09 AM
Only allow google to index my pages
I have a problem with my website. I find that too many bots on the internet are requesting my pages.
I want to make a .htaccess file. That only allows the google robots to index my pages. And dissalow all the other robots.
Is there also a way to limit the amount of requests a bot can make to my website?
August 9th, 2012, 11:15 AM
You can tell other bots that you don't want them to access your pages, but you cannot force them to not access your pages. You would do this using a robots.txt file.
August 9th, 2012, 06:11 PM
So just Google, huh? Don't care about Bing or Yahoo or Yandex or MSN or Exabot or anybody else? Tsk tsk.