November 16th, 2004, 09:01 PM
|
|
Contributing User
|
|
Join Date: Sep 2003
Location: Washington, D.C.
Posts: 625
Time spent in forums: 2 Days 16 h 47 m 55 sec
Reputation Power: 6
|
|
|
Need simple web crawling script
I recently subscribed to careerbuilder.com, of which the subscription expires in 17 days. With the many tens of thousands of entries, it is imposible to collect all the needed information in the time alloted. I would like a script that imitates a user searching for a series of specified key words. If the key words are found, and the criteria matches, the resulting page is crawled, and stored as a row in a mysql database.
The challenging part (for me) is becoming authenticated without actually submitting the form in person. Anyone interested, I look forward to hearing from you. As stated above the subscription expires in 17 days, so the deadline is that.
jsbruns@webdevsol.com
|
Dev Shed Forums Sponsor:
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
