Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed Forums > Web Design > Search Engine Optimization
Reload this Page

Prevent to crawl some pages

Discuss Prevent to crawl some pages in the Search Engine Optimization forum on Dev Shed.
Prevent to crawl some pages Search Engine Optimization forum discussing related topics such as Google, PageRank, Link Exchange, and Link Popularity. Improve your ranking in the Search Engines by employing good optimization techniques.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old February 5th, 2007, 05:41 AM
samser_m's Avatar
samser_m samser_m is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Jul 2005
Location: Kolkata, India
Posts: 296 samser_m User rank is Lance Corporal (50 - 100 Reputation Level)samser_m User rank is Lance Corporal (50 - 100 Reputation Level)samser_m User rank is Lance Corporal (50 - 100 Reputation Level) 
Time spent in forums: 1 Week 15 h 24 m 10 sec
Reputation Power: 4
Send a message via MSN to samser_m Send a message via Yahoo to samser_m Send a message via Google Talk to samser_m Send a message via Skype to samser_m
Prevent to crawl some pages
Hi!
I want to hide some pages mainly from Google, Yahoo etc crawlers. How can I do this?

Thanks in advance!
Samser
Reply With Quote
  #2  
Old February 5th, 2007, 08:51 AM
stymiee's Avatar
stymiee stymiee is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Nov 2003
Posts: 219 stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 20 h 15 m 53 sec
Reputation Power: 48
Use a robots.txt file to list the pages you don't want crawled.
Reply With Quote
  #3  
Old February 6th, 2007, 03:45 AM
bryan_m's Avatar
bryan_m bryan_m is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Jan 2007
Posts: 91 bryan_m User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 14 h 52 m 21 sec
Reputation Power: 2
Also check this for more help; http://www.google.com/intl/en/help/faq_codesearch.html#robots
Reply With Quote
Reply

Viewing: Dev Shed Forums > Web Design > Search Engine Optimization > Prevent to crawl some pages

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 1 hosted by Hostway.