#1
  1. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2005
    Posts
    228
    Rep Power
    0

    How do I get search engine results programatically?


    How do I get search engine results programatically? Some search engines use AJAX to display content and so the URL does not change. This makes it difficult to get result content from a search engine programatically.

    Most search engines do not use AJAX. But two popular ones that I am interested in is YANDEX and DUCKDUCKGO. Both of these search engines use AJAX to advance beyond the first page of results. How do I find a way to programatically access these pages?
  2. #2
  3. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    16
    Rep Power
    0
    Yandex has an API for searches, it would need a fair bit of google translate or russian language skills to make sense of it all though. I hope that helps...I can't post links yet, but Google for Yandex api - I'm sure you'll be able to find it there somewhere
  4. #3
  5. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2005
    Posts
    228
    Rep Power
    0
    I use the c# WebBrowser class and I get the HTML after I use this class to navagate to a URL. I was able to use yandex by using a "&p=" to navagate to a particular page.

    The duckduckgo search engine does not work this way, though.

    I have found out how to make a specialized query for YANDEX in a modified URL. I will ask DUCKDUCKGO for advice.

    https://duckduckgo.com/api.html does not provide a solution.

    Running this, http://api.duckduckgo.com/?q=stuff+to+search&format=xml , in a browser's address bar does not provide anything as http://duckduckgo.com/?q=stuff+to+search does.

    This shouldn't be rocket science. I am able to see the HTML code. Shouldn't there be some sort of AJAX tag in the HTML that I can find to make use of?

IMN logo majestic logo threadwatch logo seochat tools logo