|
|
|||||||||
|
|||||||||
| |||||||||
|
|
|
| |||||||||
![]() |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
#1
|
|||
|
|||
|
What directories to list in robots - but not revel?
Is it relevant to list directories where php includes are, where css is etc.. or where images are stored - in a robots file?
In the future all of these (except obvious) will be above public level. I don't like listing the directory my includes are in if I don't need to. So how important is it -- The only directories where anything will "print" to a robot is CSS, php page which print the HTML output, and images themselves.... PS: I use a template system with php building the html -- should I store the templates above public access to prevent people loading them and seeing the parsed variables etc..? |
|
#2
|
||||
|
||||
|
A bot won't and can't search above the web root, so specifying anything to that affect is pointless. And IIRC, any path in robots.txt is relative to the web root, so I'm not even sure how you'd specify above the web root. Remember that a bot is just a browser in terms of how it views your site, so if you can't view a page in the browser, the bot can't view it either.
I personally store anything that does not need to be accessed directly above the web root. Why make something available if it doesn't need to be?
__________________
# Jeremy Explain your problem instead of asking how to do what you decided was the solution. |
![]() |
| Viewing: Dev Shed Forums > Web Design > Search Engine Optimization > What directories to list in robots - but not revel? |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
|