#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2004
    Posts
    8
    Rep Power
    0

    Spider that uses mysql


    I'm looking for a web crawler that will spider my site and toss the urls and page content into a mysql database so that database can , in turn, be indexed by Sphinx Site Search.

    Not wanting to reinvent the wheel, I'm wondering if anyone knows of a code snippet that will do this or can point me towards a tutorial.

    Thanks.
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2004
    Location
    Constant Limbo
    Posts
    989
    Rep Power
    363
    Google is your friend:

    linky1
    linky2
    True happiness is not getting what you want, it's wanting what you've already got.

    My Blog

IMN logo majestic logo threadwatch logo seochat tools logo