Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsWeb DesignSearch Engine Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old September 20th, 2007, 05:37 PM
mv5869 mv5869 is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2004
Posts: 9 mv5869 User rank is Sergeant (500 - 2000 Reputation Level)mv5869 User rank is Sergeant (500 - 2000 Reputation Level)mv5869 User rank is Sergeant (500 - 2000 Reputation Level)mv5869 User rank is Sergeant (500 - 2000 Reputation Level)mv5869 User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 45 m 38 sec
Reputation Power: 0
Robots.txt to block autolinks page

I have a small casino website (1st-casinos.co.uk) and use gotlinks.com for links. Because it is an automated site and generates lots of links I've exluded the links page from robots.txt (obviously my real links page isnt excluded). Will Google still penalise me for using Gotlinks? i.e will it see the links page even though the robots file tells it not to index it?

Reply With Quote
  #2  
Old September 22nd, 2007, 09:31 AM
jazajay jazajay is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2007
Posts: 336 jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 6 h 16 sec
Reputation Power: 9
As long as you have the robots.txt set up properly Google will read it as gospel.

Reply With Quote
  #3  
Old October 1st, 2007, 08:04 AM
dominoman's Avatar
dominoman dominoman is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2004
Location: london
Posts: 332 dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 1 Day 6 h 1 m 54 sec
Reputation Power: 37
Send a message via MSN to dominoman
In that case I guess there is no harm in using free linkfarms that ask for a reciprocal link, if you can just exclude it from robots.txt?
__________________
Interested in online gaming? Access full reviews of all UK Online casinos here.

Roulette Strategy |William Hill | Viurgin | Jackpot joy

Reply With Quote
  #4  
Old October 1st, 2007, 10:38 AM
jazajay jazajay is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2007
Posts: 336 jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 6 h 16 sec
Reputation Power: 9
well very slightly link farms will automatically be devalued if not kicked out of the index. If some one checks the link back sees the robots.txt they'll remove it from their site.

You could spend weeks getting loads of ricips. from spammy sites or you could spend days getting a few good links from decent sites and rank the same if not better.

Reply With Quote
  #5  
Old October 1st, 2007, 11:10 AM
stymiee's Avatar
stymiee stymiee is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Nov 2003
Posts: 225 stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level)stymiee User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 20 h 48 m 23 sec
Reputation Power: 49
Quote:
Originally Posted by dominoman
In that case I guess there is no harm in using free linkfarms that ask for a reciprocal link, if you can just exclude it from robots.txt?

That's a heck of a gamble. If they consider any link to a link farm to be black hat, whether or not it is blocked using robots.txt or nofollow, you run the risk of being banned for assisting the link farm. Not worth the risk IMHO especially considering that those links are just about worthless if not completely worthless. And all it would take is a competitor to report your site for them to find out.
Comments on this post
MarbleHost.com agrees: Solid advice.

Reply With Quote
  #6  
Old October 1st, 2007, 11:42 AM
jazajay jazajay is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2007
Posts: 336 jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 6 h 16 sec
Reputation Power: 9
Yeah totally agree with you stymiee when you say -
"...a competitor to report your site for them to find out."
sorry really tired. Nice catch.

I've used techniques like this and they work. Why should I try to increase my rank when I can just knock other sites straight out of the run for the top 3 who are using blatant black hat. It makes my job a lot easier.

If you get banned/penalized by a personal evaluation it'll be longer than if the algo does, from what other users have noticed. I've never been banned so I cant say for sure, but it does makes sense. If you manage to cheat the algo and it is only detectable via a personal eval. then you are more of a "threat" to relevance so longer consequences as a result.

I do disagree that they would work it out as robots.txt is gospel. They wouldn't check the pages out it would mean extra bandwidth, electricity and crawl time thats why SE love robots.txt it makes their job easier and saves them time and money.

Getting reported by a competitor is a higher possibility especially in your competitive market. Again stymiee says it perfectly -
"those links are just about worthless if not completely worthless"
Get good back links it'll make your job a lot easier and less time consuming.
Comments on this post
dominoman agrees: Good balanced advice. Thanks!

Reply With Quote
  #7  
Old October 2nd, 2007, 04:38 PM
vision1 vision1 is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2007
Location: 01010111010111110011
Posts: 41 vision1 User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 8 h 8 m 3 sec
Reputation Power: 2
actually gotlinks is one of the best, you have control who links to you, and which category etc, I've noticed also that sometimes I get a customer coming from gotlinks, not much, but one or two every some month.

But if you use the robots.txt although it is not a black hat method it is still unethical, because you have agreed to show this links.

Reply With Quote
  #8  
Old October 9th, 2007, 06:29 AM
MarbleHost.com MarbleHost.com is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Sep 2007
Posts: 36 MarbleHost.com User rank is Sergeant (500 - 2000 Reputation Level)MarbleHost.com User rank is Sergeant (500 - 2000 Reputation Level)MarbleHost.com User rank is Sergeant (500 - 2000 Reputation Level)MarbleHost.com User rank is Sergeant (500 - 2000 Reputation Level)MarbleHost.com User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 7 h 2 m 6 sec
Reputation Power: 15
I dunno, it's pretty villainous in my book.

Part of the deal going on is that these other sites are expecting to rise a little in rank from NOT being on your robots.txt just as you expect to not be in theirs.

The only reason someone could possibly have for getting loads of impressions of people that don't even meet their target audience would be to farm ads, or rather, if their target audience were "anyone who qualifies the site to receive ad revenue and nothing more".

Honestly, it isn't hard to build a user base, you just have to get creative and work hard at it, or even just pay money for traditional CPC or CPM at a focused target audience for your services.

Reply With Quote
  #9  
Old October 10th, 2007, 10:38 AM
jazajay jazajay is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2007
Posts: 336 jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 6 h 16 sec
Reputation Power: 9
Yeah totally agree.

Also you haven't taken into account the "Muppet Factor"

Whats the Muppet factor?
Well the Muppet factor is when some one like me comes along, knowing what they are doing, and ruins your day.

For example if I followed a link back and saw it was being blocked by robots.txt which one of these options do you think I would take -

1. Remove the link from my site and move on.

2. Remove the link from my site and then email everyone else who are getting screwed and let them know. Then sit back and watch every one desert the sinking ship while taking their equity with them.

Hay they are my competitor at the end of the day so why should I fight up hill when I can level the battle field.

Personally I would recommend people especially my competitors do block links via robots.txt it wastes their time and gives me a lot easier job.

Last edited by jazajay : October 10th, 2007 at 10:40 AM. Reason: spelling

Reply With Quote
  #10  
Old October 10th, 2007, 10:43 AM
jazajay jazajay is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2007
Posts: 336 jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level)jazajay User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 6 h 16 sec
Reputation Power: 9
Quote:
Originally Posted by vision1
actually gotlinks is one of the best, you have control who links to you, and which category etc, I've noticed also that sometimes I get a customer coming from gotlinks, not much, but one or two every some month......


Let me guess you own gotlinks. Nice ad

Reply With Quote
Reply

Viewing: Dev Shed ForumsWeb DesignSearch Engine Optimization > Robots.txt to block autolinks page


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 4 hosted by Hostway
Stay green...Green IT