
March 13th, 2002, 02:36 PM
|
|
Contributing User
|
|
Join Date: Oct 2000
Location: Back in the real world.
|
|
|
why do you want to stop him? seems like a crawler from a search engine and if it only requests robots.txt, this is fine, it complies to the standards.
give it a "robots.txt" with:
User-agent: *
Disallow: /
and it won´t bother you...
there is 10000s like this on the web, so you need to block quite many IPs. (i know of altavista having at least 100 different ones that showed up in my server logs already)
|