Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed Forums > Web Design > Search Engine Optimization
Reload this Page

BLOCK ALL bots, crawler ect.?

Discuss BLOCK ALL bots, crawler ect.? in the Search Engine Optimization forum on Dev Shed.
BLOCK ALL bots, crawler ect.? Search Engine Optimization forum discussing related topics such as Google, PageRank, Link Exchange, and Link Popularity. Improve your ranking in the Search Engines by employing good optimization techniques.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old January 9th, 2008, 03:54 PM
mfleming mfleming is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Oct 2004
Posts: 137 mfleming User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 Day 2 h 26 m 52 sec
Reputation Power: 5
Question BLOCK ALL bots, crawler ect.?
Hello.

How do I block all search engines from NOT adding my webpage to the search?

Basically I want to have the robots.txt file say not to allow until the site is finished builing and then when I go to enable it I can delete that file and start submitting my page to the search engines.
Reply With Quote
  #2  
Old January 9th, 2008, 06:24 PM
stymiee's Avatar
stymiee stymiee is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Nov 2003
Posts: 225 stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 20 h 48 m 23 sec
Reputation Power: 49
User-agent: *
Disallow: /
Reply With Quote
Reply

Viewing: Dev Shed Forums > Web Design > Search Engine Optimization > BLOCK ALL bots, crawler ect.?

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

 Free IT White Papers!
 
How to Present Effectively Online
This white paper offers practical and actionable advice on the key steps that any presenter should consider as they plan and execute a Webinar or online meeting.
 
Open Source Security Myths
Open Source Software (OSS) is computer software whose source code is available to the general public with relaxed or non-existent intellectual property restrictions (or arrangement such as the public domain), and is usually developed with the input of many contributors.
 
Power and Cooling Capacity Management for Data Centers
This paper describes the principles for achieving power and cooling capacity management.
 
Scalable, Fault-Tolerant NAS for Oracle - The Next Generation
For several years NAS has been evolving as a storage alternative for Oracle databases, and for good reason: NAS is quite often the simplest, most cost-effective storage approach for Oracle. Learn about the benefits that HP's approach to scalable NAS brings to Oracle environments in this comprehensive white paper.
 
Understanding Web Application Security Challenges
This white paper discusses many common threats and preventive measures for Web application security, and explains what you can do to help protect your organization.
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 1 hosted by Hostway. 
Stay green...Green IT