Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed Forums > Web Design > Search Engine Optimization
Reload this Page

bot control

Discuss bot control in the Search Engine Optimization forum on Dev Shed.
bot control Search Engine Optimization forum discussing related topics such as Google, PageRank, Link Exchange, and Link Popularity. Improve your ranking in the Search Engines by employing good optimization techniques.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old March 23rd, 2005, 09:12 AM
G00SE's Avatar
G00SE G00SE is offline
.... escape ....
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Jun 2004
Posts: 448 G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 19 h 42 m 8 sec
Reputation Power: 11
bot control
The bots are out of control!!!

Is there a way to limit what bots can do on your site?

At the moment they go through everything, I want to stop them indexing small sections of the sites like profiles?

What would I need to put in the meta tags?

The whole site is based on index.php? so I can't use a txt file to say no and yes.
Reply With Quote
  #2  
Old March 23rd, 2005, 11:08 AM
dejaone dejaone is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Mar 2004
Posts: 300 dejaone User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 8 h 7 m 22 sec
Reputation Power: 5
use robots.txt file. serach web for details.
Reply With Quote
  #3  
Old March 26th, 2005, 05:01 PM
G00SE's Avatar
G00SE G00SE is offline
.... escape ....
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Jun 2004
Posts: 448 G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 19 h 42 m 8 sec
Reputation Power: 11
Robots.txt won't work in this case.

The whole site is index.php?something&other
__________________
Reply With Quote
  #4  
Old March 28th, 2005, 10:22 AM
timothytrice timothytrice is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Dec 2004
Location: Houston, TX
Posts: 144 timothytrice User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 2 Days 2 m 32 sec
Reputation Power: 4
Send a message via MSN to timothytrice Send a message via Yahoo to timothytrice
Quote:
Originally Posted by G00SE
Robots.txt won't work in this case.

The whole site is index.php?something&other




<META NAME="robots" CONTENT="noindex">

Also, read http://www.robotstxt.org/wc/exclusion.html

Tim
Reply With Quote
  #5  
Old March 28th, 2005, 02:55 PM
G00SE's Avatar
G00SE G00SE is offline
.... escape ....
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Jun 2004
Posts: 448 G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level)G00SE User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 19 h 42 m 8 sec
Reputation Power: 11
Thanks for that, I guessed it must be in the meta somewhere.
Reply With Quote
Reply

Viewing: Dev Shed Forums > Web Design > Search Engine Optimization > bot control

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 5 hosted by Hostway.