#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2014
    Location
    USA
    Posts
    3
    Rep Power
    0

    pull data from website into ecommerce site.


    Hi, I'm not sure if this is the right thread but here goes. I've been tasked by my boss to update his site and copy product listings from his distributors onto his e-commerce site. Two of the sites use a form to search their inventory based on specific criteria. One uses item numbers and the other has drop downs. I need to find a program that can help pull data from them in batches to be formatted for import into his site. the data must go into a comma delimited csv file and contain, product name, price, category, pic links, description, part numbers and a few others. I'm not sure exactly what I need to make this work. I used outwit hub to craw another distributor but it was all html files. this is more dynamic for the program can handle.

    If you need links, I'll provide them.
    Thanks
  2. #2
  3. No Profile Picture
    Lost in code
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 2004
    Posts
    8,304
    Rep Power
    7173
    I'm not sure about programs that would do that for you, but if you know any programming languages you could probably write a script to do it without too much trouble. It seems like the sort of thing that there might be programs for, but whether or not they would work probably depends a lot on the format of the distributors' sites.

    It might also be worth contacting the distributors and asking if they can provide the data in an easier to process format like JSON or XML. They probably can't, but it could save you a lot of work so it would be worth asking.
    PHP FAQ

    Originally Posted by Spad
    Ah USB, the only rectangular connector where you have to make 3 attempts before you get it the right way around
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2014
    Location
    USA
    Posts
    3
    Rep Power
    0
    I have little to no programming skills to speak of but can recognize certain lines of code and figure out what it might be doing. I'm not sure how to translate XML files into csv or even what JSON is. The sites in question are WSM for PWC products and Hot Products. Both use a search form rather than just linking it all as part of the navigation of sorts. They provide pdf files but everything I look at involves tables and the pdf files vary too much in the pages to be able to do it in batches and don't have the prices listed. I've played with html and css off and on so I know my way around it, just couldn't build it from scratch. My design skills look terrible.
  6. #4
  7. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2014
    Posts
    1
    Rep Power
    0
    Yes the link would help, but I don't know if you should post them in a forum... I have been using outwit hub a lot on very dynamic pages and it does work fine.
    Sam
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2014
    Location
    USA
    Posts
    3
    Rep Power
    0
    I imagine a quick google of the site should reveal the link. Outwit hub doesn't seem to like these sites though, the information is difficult to gather and the automatic bit doesn't do as I expect it would. It kept wanting to work back to the index of another site and not even do the pages I wanted. Maybe the free version limits it in that way somewhat, idk. If it could prove to be more useful, I'd have my boss buy a license.

IMN logo majestic logo threadwatch logo seochat tools logo