Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsWeb DesignSearch Engine Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old March 24th, 2008, 04:43 PM
adammc's Avatar
adammc adammc is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Sep 2004
Location: Cairns, Australia
Posts: 471 adammc User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 2 Days 10 h 34 m 29 sec
Reputation Power: 0
Question Robots file - disallow dirs

Hi guys,

I tried to ban the search engines from accessing my shopping cart pages but I have just discovered that google has indexed many of my cart pages?

example:
www.domainname.com.au/store/cart/phpcart.php?aaaaaaaa


my robots file looks like this and is placed in my domains root:

User-agent: *
Disallow: /cgi-bin/
Disallow: /CMS/
Disallow: /store/cart/
Disallow: /captcha/

Any ideas why it didnt work and what I can do?
__________________
Aussie Webmaster Forums | | Webmaster Tools

Reply With Quote
  #2  
Old March 24th, 2008, 05:15 PM
tj_nt tj_nt is offline
Contributing User
Dev Shed Intermediate (1500 - 1999 posts)
 
Join Date: Feb 2004
Posts: 1,544 tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level)tj_nt User rank is Brigadier General (60000 - 70000 Reputation Level) 
Time spent in forums: 1 Month 6 Days 23 h 10 m 49 sec
Reputation Power: 619
try this:

Disallow: /store/cart

Some crawlers see the following "/" telling it to ignore the "index".php file etc... if you want to deny the entire dir. try omitting the following /
__________________
I'm so old...When I was born the Dead Sea was just sick...

Reply With Quote
Reply

Viewing: Dev Shed ForumsWeb DesignSearch Engine Optimization > Robots file - disallow dirs


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

 Free IT White Papers!
 
Accelerating Trading Partner Performance
One in five. That's how many partner transactions have at least one error. That is an amazing statistic, particularly given the extraordinary leaps in innovation across the global supply chain during the past two decades. Download this white paper to learn more.

 
Competing on Analytics
This Tech Analysis is designed to help identify characteristics shared by analytics competitors, and includes information about 32 organizations that have made a commitment to quantitative, fact-based analysis.

 
Cost Effective Scaling with Virtualization and Coyote Point Systems
An overview of the industry trend toward virtualization, how server consolidation has increased the importance of application uptime and the steps being taken to integrate load balancing technology with virtualized servers.

 
Five Checkpoints to Implementing IP Telephony
Implementation planning for IP PBX software and IP telephony has become vital as businesses replace discontinued legacy PBX phone systems. This informative whitepaper outlines five "checkpoints" for any implementation plan that will help make IP communications a successful proposition.

 
Hosted Email Security: Staying Ahead of New Threats
In the last two years, email has become a fierce battleground between the nefarious forces of spam and malware, and the heroes of messaging protection. The spam volumes increased alarmingly every month, bringing clever new forms of phishing and virus propagation attacks.

 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 3 hosted by Hostway