October 25th, 2013, 01:19 PM
Prevent crawlers from submitting forms using htaccess
Crawlers like Google submit forms to try to get more pages to index. I heard there is way to prevent crawlers from submitting form using htaccess and robots.txt.
Do you know how to do it using htaccess?
October 25th, 2013, 01:25 PM
October 25th, 2013, 01:28 PM
but there is an actual rule to achieve this but can't remember it at all
October 25th, 2013, 01:31 PM
Did you read that link? There are a couple things they specifically mention.
October 25th, 2013, 01:38 PM
They mention that it's an experiment but the article if from 2008 so i think it's already active now from long time. If it's not active, how some forums have search queries indexed?