Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsWeb DesignSearch Engine Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old January 12th, 2007, 01:55 PM
zwebdeveloper zwebdeveloper is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2006
Posts: 2 zwebdeveloper User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 22 m 32 sec
Reputation Power: 0
Question SEO & SSI Question?

What do search engine bots/crawlers/spiders crawl when they're looking for pages to index on your site?

My understanding was that these crawlers indexed your web pages on your server as if your page was viewed in a text editor and not what is displayed to browsers (after server processing). Is this correct? Do the crawlers see your pages with server side includes as one file (like the browser does) or does the crawler index your page making special note of your linked .css/.js files and SSIs and crawl them shortly afterwards but not as part of the index of the main page that called them?

Thanks much!!!

Reply With Quote
  #2  
Old January 14th, 2007, 12:30 AM
jabba_29's Avatar
jabba_29 jabba_29 is offline
Back in HEL
Click here for more information.
 
Join Date: Feb 2002
Location: Finland
Posts: 8,912 jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)jabba_29 User rank is General 11st Grade (Above 100000 Reputation Level)  Folding Points: 57777 Folding Title: Beginner FolderFolding Points: 57777 Folding Title: Beginner FolderFolding Points: 57777 Folding Title: Beginner Folder
Time spent in forums: 3 Months 4 Weeks 1 Day 13 h 4 m 29 sec
Reputation Power: 1693
Send a message via ICQ to jabba_29 Send a message via AIM to jabba_29 Send a message via MSN to jabba_29 Send a message via Yahoo to jabba_29 Send a message via Google Talk to jabba_29 Send a message via Skype to jabba_29
Facebook
Yep.

SSI are executed on the server, so the SE will see the finalized page -
not the SSI itself.

You might want to block the directory that you store your SSI files in from robots though...as you don't want these files indexed individually.
__________________
Cheers,

Jamie

# mdb4u | mobile movie database] | Please help to test and promote
# skiFFie | Home of the 'accessibility module' for Drupal
# Jamie Burns [me] Accessibility Module [drupal]
# guidelines | search | wap resources | not getting help | fold to cure

__________________

Let the might of your compassion arise to bring a quick end
to the flowing stream of the blood and tears .....
Please hear my anguished words of truth.



__________________

Reply With Quote
Reply

Viewing: Dev Shed ForumsWeb DesignSearch Engine Optimization > SEO & SSI Question?


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 1 hosted by Hostway
Stay green...Green IT