Apache Development
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me

The Shed is going Social! Join us on FaceBook and Twitter and chime in on the conversation.

Go Back   Dev Shed Forums > System Administration > Apache Development
Reload this Page

Only allow google to index my pages


Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old August 8th, 2012, 07:09 AM
dabomb_gent dabomb_gent is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Apr 2004
Posts: 297 dabomb_gent User rank is Sergeant (500 - 2000 Reputation Level)dabomb_gent User rank is Sergeant (500 - 2000 Reputation Level)dabomb_gent User rank is Sergeant (500 - 2000 Reputation Level)dabomb_gent User rank is Sergeant (500 - 2000 Reputation Level)dabomb_gent User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 13 h 32 m 3 sec
Reputation Power: 16
Only allow google to index my pages
Hi,

I have a problem with my website. I find that too many bots on the internet are requesting my pages.

I want to make a .htaccess file. That only allows the google robots to index my pages. And dissalow all the other robots.

Is there also a way to limit the amount of requests a bot can make to my website?

Greets,
Thomas
Reply With Quote
  #2  
Old August 9th, 2012, 10:15 AM
E-Oreo's Avatar
E-Oreo E-Oreo is offline
Lost in code
Click here for more information.
  
Join Date: Dec 2004
Posts: 7,931 E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)E-Oreo User rank is General 91st Grade (Above 100000 Reputation Level)  Folding Points: 945 Folding Title: Novice Folder
Time spent in forums: 2 Months 7 h 48 m 54 sec
Reputation Power: 7053
You can tell other bots that you don't want them to access your pages, but you cannot force them to not access your pages. You would do this using a robots.txt file.
__________________
PHP FAQ
How to program a basic, secure login system using PHP

Quote:
Originally Posted by Spad
Ah USB, the only rectangular connector where you have to make 3 attempts before you get it the right way around
Reply With Quote
  #3  
Old August 9th, 2012, 05:11 PM
requinix's Avatar
requinix requinix is offline
Still alive
Click here for more information.
  
Join Date: Mar 2007
Location: Washington, USA
Posts: 12,698 requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)  Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1
Time spent in forums: 5 Months 1 Week 4 Days 4 h 54 m 57 sec
Reputation Power: 8969
Send a message via AIM to requinix Send a message via MSN to requinix Send a message via Yahoo to requinix Send a message via Google Talk to requinix
So just Google, huh? Don't care about Bing or Yahoo or Yandex or MSN or Exabot or anybody else? Tsk tsk.
Reply With Quote
Reply

Viewing: Dev Shed Forums > System Administration > Apache Development > Only allow google to index my pages

« Previous Thread | Next Thread »

Developer Shed Advertisers and Affiliates



Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 


Powered by: vBulletin Version 3.0.5
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.

© 2003-2013 by Developer Shed. All rights reserved. DS Cluster - Follow our Sitemap