What's better way to stop Spider/Robot?

I am wondering what's best way to stop (trap) them? Using the mod_rewrite, perl script or what? My opinion, put something in Apache is the best method to trap them.

Thanks,
Mezz

Just put whatever like BrowserMatchNoCase ^Scooter bad_robot within <IfModule mod_setenvif.c> or define them globally.
Then set Deny from env=bad_robot anywhere you wish, preferably within <Directory>.
Multiple robots can assign the same env so you just need to Deny bad_robot once.

Thanks freebsd!

I am going to put "Deny from env=bad_robot" in <Directory /></Directory>, so is it good idea? I will have to test this afternoon to see if it will stop the spider/robot.


<IfModule mod_setenvif.c>
BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4\.0" force-response-1.0
BrowserMatch "Java/1\.0" force-response-1.0
BrowserMatch "JDK/1\.0" force-response-1.0
BrowserMatchNoCase ^Scooter bad_robot
</IfModule>

Umm, I already tested it recently. I have decided to do it right now and it doesn't works, because the spider tool that I have can fool the user_agent. When I put the it as "Mozilla/4" then it can spider/robot on server as well, so is there better way or just impossible to stop it?

Thanks,
Mezz

>> so is there better way or just impossible to stop it?

Impossible.