#1
  1. Oheim
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2001
    Location
    Austria
    Posts
    69
    Rep Power
    14

    Search engines and Dynamic websites...


    Hy-
    I was wondering....

    how do spiders treat dynamic websites? What I mean is this:
    A URI like this one:
    http://forums.devshed.com/newthread.php?s=&action=newthread&forumid=26

    usualy isn't indexed by spiders, at least I was told so!, can anybody tell me if that is true?
    And if so, why?

    All they do is to open a http connection and then parse links, right? So I don't see a reason why spiders couldn't follow a link like the above one.

    so... I am confused (
    maybe you guys know the answere I am looking for.

    Thanks a lot!
    Sheriff
    There are two major products that come out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence.

    God is just hacker who rooted the world!
  2. #2
  3. No Profile Picture
    Apprentice Deity
    Devshed Loyal (3000 - 3499 posts)

    Join Date
    Jul 1999
    Location
    Niagara Falls (On the wrong side of the gorge)
    Posts
    3,237
    Rep Power
    19
    Most spiders won't crawl pages with a ? in the URL.

    The reason is that the search engine designers believe that if the URL has a dynamic element then the data harvested from the page is more likely to be inaccuarate at a future date than a static page.
  4. #3
  5. Oheim
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2001
    Location
    Austria
    Posts
    69
    Rep Power
    14
    This means, that if you use a content management system, you have a disadvantage...
    Kinda s*****... doesn't it?

    Is there a workaround?
    There are two major products that come out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence.

    God is just hacker who rooted the world!
  6. #4
  7. No Profile Picture
    Apprentice Deity
    Devshed Loyal (3000 - 3499 posts)

    Join Date
    Jul 1999
    Location
    Niagara Falls (On the wrong side of the gorge)
    Posts
    3,237
    Rep Power
    19

IMN logo majestic logo threadwatch logo seochat tools logo