SunQuest
           Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsWeb DesignSearch Engine Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old October 20th, 2005, 10:26 PM
MasterMinds MasterMinds is offline
Contributing User
Dev Shed Novice (500 - 999 posts)
 
Join Date: Oct 2004
Posts: 556 MasterMinds Negative: is most likely a SPAMMER and a traitor to the cause. 
Time spent in forums: 2 Days 12 h 9 sec
Reputation Power: 0
What directories to list in robots - but not revel?

Is it relevant to list directories where php includes are, where css is etc.. or where images are stored - in a robots file?

In the future all of these (except obvious) will be above public level.

I don't like listing the directory my includes are in if I don't need to.

So how important is it -- The only directories where anything will "print" to a robot is CSS, php page which print the HTML output, and images themselves....


PS: I use a template system with php building the html -- should I store the templates above public access to prevent people loading them and seeing the parsed variables etc..?

Reply With Quote
  #2  
Old October 21st, 2005, 12:48 AM
jharnois's Avatar
jharnois jharnois is offline
mod_dev_shed
Dev Shed God 19th Plane (14000 - 14499 posts)
 
Join Date: Sep 2002
Location: Atlanta, GA
Posts: 14,299 jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level)jharnois User rank is Major General (70000 - 90000 Reputation Level) 
Time spent in forums: 1 Month 1 Week 2 Days 1 h 54 m 17 sec
Reputation Power: 760
A bot won't and can't search above the web root, so specifying anything to that affect is pointless. And IIRC, any path in robots.txt is relative to the web root, so I'm not even sure how you'd specify above the web root. Remember that a bot is just a browser in terms of how it views your site, so if you can't view a page in the browser, the bot can't view it either.

I personally store anything that does not need to be accessed directly above the web root. Why make something available if it doesn't need to be?
Comments on this post
Gnome101 agrees!
__________________
# Jeremy

Explain your problem instead of asking how to do what you decided was the solution.

Reply With Quote
Reply

Viewing: Dev Shed ForumsWeb DesignSearch Engine Optimization > What directories to list in robots - but not revel?


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 5 hosted by Hostway