Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsWeb DesignSearch Engine Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old August 12th, 2006, 04:29 AM
Andrew_Smart Andrew_Smart is offline
Contributing User
Dev Shed Intermediate (1500 - 1999 posts)
 
Join Date: Nov 2003
Posts: 1,755 Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 5 Days 23 h 55 m 8 sec
Reputation Power: 59
Pages just wont spider...

Hi, this webpage i have had up for a while now keeps spidering on google but only the homepage. It simply wont spider the internal pages but i cant see any reason why not. I'm using the same methods i always use. Can anyone shed any light on this.

the address is http://www.war-medals.com/

Thanks

Reply With Quote
  #2  
Old August 12th, 2006, 05:13 AM
Hombre's Avatar
Hombre Hombre is offline
Pixel Cruncher
Dev Shed Novice (500 - 999 posts)
 
Join Date: Jan 2005
Location: UK
Posts: 647 Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)  Folding Points: 3232 Folding Title: Novice Folder
Time spent in forums: 2 Weeks 1 Day 9 h 1 m 56 sec
Reputation Power: 102
Try removing the preceeding slash from the hrefs (<a href="/services.htm") or use the absolute path.

Reply With Quote
  #3  
Old August 12th, 2006, 05:58 AM
teamgrowth teamgrowth is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2006
Posts: 12 teamgrowth User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 h 37 m 26 sec
Reputation Power: 0
According to me, use sitemap in your site. Sitemap is useful. It gives information about your site. Sitemap helps in improved spidering.

Reply With Quote
  #4  
Old August 14th, 2006, 05:24 PM
Andrew_Smart Andrew_Smart is offline
Contributing User
Dev Shed Intermediate (1500 - 1999 posts)
 
Join Date: Nov 2003
Posts: 1,755 Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 5 Days 23 h 55 m 8 sec
Reputation Power: 59
Quote:
Originally Posted by Hombre
Try removing the preceeding slash from the hrefs (<a href="/services.htm") or use the absolute path.


i will give this a whirl and report back

Reply With Quote
  #5  
Old August 15th, 2006, 12:18 AM
Noodles23's Avatar
Noodles23 Noodles23 is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jun 2004
Location: Auckland, NZ
Posts: 426 Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)Noodles23 User rank is Major (30000 - 40000 Reputation Level)  Folding Points: 856301 Folding Title: Super Ultimate Folder - Level 2Folding Points: 856301 Folding Title: Super Ultimate Folder - Level 2Folding Points: 856301 Folding Title: Super Ultimate Folder - Level 2Folding Points: 856301 Folding Title: Super Ultimate Folder - Level 2Folding Points: 856301 Folding Title: Super Ultimate Folder - Level 2Folding Points: 856301 Folding Title: Super Ultimate Folder - Level 2Folding Points: 856301 Folding Title: Super Ultimate Folder - Level 2
Time spent in forums: 2 Days 23 h 23 m 29 sec
Reputation Power: 387
Using google sitemaps will also tell you which pages google is having problems with spidering (if any).

Also google is reporting back no inbound links to your site, this may have something to do with why you're not getting spidered

http://www.google.com/search?hl=en&lr=&q=link%3Awww.war-medals.com&btnG=Search
__________________
Cooking Forums

Reply With Quote
  #6  
Old August 18th, 2006, 06:25 AM
bermuda bermuda is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2006
Posts: 3 bermuda User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 12 m 45 sec
Reputation Power: 0
I haven't seen any webmaster complaining about the Google sitemap, so nowadays it's really a must for websites.

Reply With Quote
  #7  
Old September 3rd, 2006, 03:31 AM
Andrew_Smart Andrew_Smart is offline
Contributing User
Dev Shed Intermediate (1500 - 1999 posts)
 
Join Date: Nov 2003
Posts: 1,755 Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level)Andrew_Smart User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 5 Days 23 h 55 m 8 sec
Reputation Power: 59
Quote:
Originally Posted by Hombre
Try removing the preceeding slash from the hrefs (<a href="/services.htm") or use the absolute path.


I used absolute paths and they spidered. To be honest i thought google was more clever than that

Reply With Quote
  #8  
Old September 3rd, 2006, 10:51 AM
Hombre's Avatar
Hombre Hombre is offline
Pixel Cruncher
Dev Shed Novice (500 - 999 posts)
 
Join Date: Jan 2005
Location: UK
Posts: 647 Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)  Folding Points: 3232 Folding Title: Novice Folder
Time spent in forums: 2 Weeks 1 Day 9 h 1 m 56 sec
Reputation Power: 102
Quote:
Originally Posted by Andrew_Smart
I used absolute paths and they spidered. To be honest i thought google was more clever than that

It is kinda strange. Some sites I monitor are crawled like crazy with relative URLs and others seem to require an absolute path.

I have been wondering (in my case at least) if its anything to do with hosting several domains on the same hosting account server but I can't see why it should cause issues providing each domain points to it's own www (home) directory.

Reply With Quote
  #9  
Old September 7th, 2006, 07:48 PM
dominoman's Avatar
dominoman dominoman is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2004
Location: london
Posts: 292 dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level)dominoman User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 1 Day 1 h 41 m 45 sec
Reputation Power: 26
Send a message via MSN to dominoman
Very strange indeed. Surely relative links makes no difference. Its the volume of links that matters.
__________________
Interested in online gaming? Access full reviews of all UK Online casinos here.

Make Money from home |Roulette Strategy |William Hill | Viurgin | Jackpot joy | Spread Betting UK

Reply With Quote
  #10  
Old September 8th, 2006, 08:32 AM
Hombre's Avatar
Hombre Hombre is offline
Pixel Cruncher
Dev Shed Novice (500 - 999 posts)
 
Join Date: Jan 2005
Location: UK
Posts: 647 Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)Hombre User rank is Second Lieutenant (5000 - 10000 Reputation Level)  Folding Points: 3232 Folding Title: Novice Folder
Time spent in forums: 2 Weeks 1 Day 9 h 1 m 56 sec
Reputation Power: 102
Quote:
Originally Posted by dominoman
Very strange indeed. Surely relative links makes no difference. Its the volume of links that matters.

That's the rub - It 'should' make no difference, a link is a link and providing the target page exists there should be no difference whether it's referenced by absolute or relative paths.

A crawler should come along, read the robots.txt file, obey the rules it finds () and crawl the ahref's it finds.

The quality and amount of external links is important but ineffective from an SEO point of view if they aren't being spidered.

Last edited by Hombre : September 8th, 2006 at 08:36 AM. Reason: typo

Reply With Quote
  #11  
Old September 14th, 2006, 10:30 PM
loon loon is offline
Permanently Banned
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Sep 2006
Posts: 6 loon Negative: is most likely a SPAMMER and a traitor to the cause. 
Time spent in forums: 1 h 15 m 13 sec
Warnings Level: 10
Number of bans: 1
Reputation Power: 0
Try add this into your website

<meta name="ROBOTS" content="INDEX, FOLLOW" />

hope this help

loon
http://www.webhostforasp.met
It is better to be hated for who you are than to be loved for what you are not

Reply With Quote
  #12  
Old November 8th, 2006, 06:50 PM
scolli scolli is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2006
Location: UK
Posts: 86 scolli User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 12 h 25 m 32 sec
Reputation Power: 2
I've set up a google sitemap and it's all looking fine in the webmaster tools.
darn googlebot still won't get past my homepage though!

Reply With Quote
  #13  
Old December 27th, 2006, 07:08 AM
jbjacob's Avatar
jbjacob jbjacob is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2006
Location: United Kingdom
Posts: 169 jbjacob User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 2 Days 15 h 12 m 31 sec
Reputation Power: 2
Send a message via ICQ to jbjacob Send a message via MSN to jbjacob
Quote:
Originally Posted by bermuda
I haven't seen any webmaster complaining about the Google sitemap, so nowadays it's really a must for websites.


Yes I agree with you on this.Those whoa are complaining about Google sitemap simply do not know how to use it for getting maximum benefits.If you follow the guidelines set by Google properly and if your site is up to date then I don't think of any possible reason for not getting benefit from it.
__________________

VPS Hosting
| Dedicated Server Hosting

Email & MSN :: ryanw @ eUKhost.com || AIM :: RyanUKweb
Complete Web Hosting Solutions at eUKhost.com

Reply With Quote
  #14  
Old January 6th, 2007, 07:37 PM
Dave A Dave A is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2007
Posts: 5 Dave A User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 59 sec
Reputation Power: 0
Quote:
Originally Posted by Andrew_Smart
Hi, this webpage i have had up for a while now keeps spidering on google but only the homepage. It simply wont spider the internal pages but i cant see any reason why not. I'm using the same methods i always use. Can anyone shed any light on this.

the address is (URL address blocked: See forum rules)

Thanks


Hi Andrew your website is fine and is indexing quite well so it's not a problem with your code at all, I own a new Zealand based search engine and I am just indexing it for you right now and it is indexing fine:
So far we have found the following files that have been indexed:
-
-contact.htm
-medals-1.htm
-medals-2.htm
-medals-3.htm
-medals-4.htm
-medals-5.htm
-medals.htm
-services.htm

Reply With Quote
Reply

Viewing: Dev Shed ForumsWeb DesignSearch Engine Optimization > Pages just wont spider...


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode