|
|
|||||||||
|
|||||||||
| |||||||||
|
|
|
| |||||||||
![]() |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
#1
|
|||
|
|||
|
Google not honoring robots.txt ?
I have a simple robots.txt in the root of my website with the following:
User-Agent: * Disallow: /my_folder_name_not_to_index/ where "my_folder_name_not_to_index" is the name of the folder I do NOT want google (or any other bot) to index. However I can still search google and find items listed. Help ? Once items are in Google's cache, can they ever be removed ? |
|
#2
|
||||
|
||||
|
Hi,
I'd suggest using a no index meta tag as well as a robots.txt file to prevent caching. You can get rid of pages cached by Google, how quickly depends on how keen you are. Take a look at this Google Help section that deals with both.
__________________
ATB ![]() UK Political Debates online shopping Tec-books: Photoshop instruction manuals |
|
#3
|
|||
|
|||
|
ok, that worked.
one more thing, i also have the following in my robots.txt Disallow: /*.pdf$ Disallow: /*.PDF$ However, I still see several links to pdf's that show up with a google search. I dont have the directories listed though. |
![]() |
| Viewing: Dev Shed Forums > Web Design > Search Engine Optimization > Google not honoring robots.txt ? |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
|