Page 1 of 2 12 Last
  • Jump to page:
    #1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1

    Question How To Auto Type In Search Box And Click Search Button ?


    Hi,

    I want to build a cURL script that auto searches a keyword on any site's search feature.
    In this example, imagine I want it to search ebay.com.
    How should I code it ?
    What is the code that:

    1. Types a keyword in the search box;
    2. Clicks the search button.

    I need the appropriate function names to research on them.

    I am stuck at this point:

    PHP Code:
    $keywords blah blah;
    $url "http://www.ebay.com";
    $ch curl_init();
    curl_setopt($chCURLOPT_URL"$url");
    curl_setopt($chCURLOPT_FOLLOWLOCATIONTRUE);
    curl_setopt($chCURLOPT_HEADER5);
    curl_setopt($chCURLOPT_RETURNTRANSFERTRUE);
    $result curl_exec($ch);
    curl_close($ch);
    echo 
    "$result"
    So, with my cURL, I have managed to load ebay.com homepage.
    Now, I need to tell the cURL to:

    A). Find the item (search box) that has the following name attribute or id attribute and type the $keywords value onto it.

    name="_nkw";
    id="gh-ac";


    B). Find the item (search button) that has the following name attribute or id attribute attribute and click it.

    id="gh-btn";
    value="Search";


    I would appreciate any code snippet additions to it by anyone.
    I am not that much learned to deal with API. Therefore, need the basic code lines that tell cURL or php how to identify items and type in them or click them. General code.

    Thank you very much for your help.
    Last edited by UniqueIdeaMan; October 4th, 2017 at 09:04 AM.
  2. #2
  3. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    CURL doesn't work that way. It lets you get the page, and then you close the connection and do what you want to with the returned HTML.

    To submit the form you'll need to find a few bits of data from the HTML that's returned. As a starting point (and probably not exhaustive):
    1. Get the 'action' value so you know the URL to send the values to.
    2. Get the name and value of every field in the form. A lot of forms have nonce or single-use values, ID's, honeypot fields, etc and all of these have to be included every time, and will change on every submission.
    3. The name of the actual search field so you know what field to bind your search value to.


    When you have all that you could create a second CURL call and append all of the name/value pairs to it to try and get the server to respond. It might, but it might not. eBay is another company that takes a great deal of care to stop automated systems crawing or searchong their listings, so you'll get away with it for a while, but as you get flagged, you'll be blocked.

    Comments on this post

    • UniqueIdeaMan agrees
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Thanks. But may the newbies in this forum see a small code sample (that does all the steps you mentioned) just to get us started so we do not fumble where and how to begin ??
  6. #4
  7. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    Originally Posted by UniqueIdeaMan
    Thanks. But may the newbies in this forum see a small code sample (that does all the steps you mentioned) just to get us started so we do not fumble where and how to begin ??
    Newbies? You?

    Haven't you been working on your "amazing ideas" in PHP for months now? :P Normally that would give most people a good understanding on how PHP works, and what you need to do...

    And as for a "small code sample", that's pretty much impossible for this situation. this is a big undertaking, so a small "sample" would still be 100's of lines long. How about you do what you have done before, and write soem code, see where the errors or problems are and then come back and ask for advice? As much as we do want to help, no one here is going to do your work for you.
  8. #5
  9. Banned (not really)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 1999
    Location
    Caro, Michigan
    Posts
    14,815
    Rep Power
    4554
    Still begging for code when you don't even know how HTTP works, eh?
    -- Cigars, whiskey and wild, wild women. --
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Originally Posted by Sepodati
    Still begging for code when you don't even know how HTTP works, eh?
    What ? Get cigars, whiskey and wild wild women out of your head and talk clearly. Man to man! Lol!
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Originally Posted by Catacaustic
    Newbies? You?

    Haven't you been working on your "amazing ideas" in PHP for months now? :P Normally that would give most people a good understanding on how PHP works, and what you need to do...

    And as for a "small code sample", that's pretty much impossible for this situation. this is a big undertaking, so a small "sample" would still be 100's of lines long. How about you do what you have done before, and write soem code, see where the errors or problems are and then come back and ask for advice? As much as we do want to help, no one here is going to do your work for you.
    Hey bud,

    I already started this thread with a sample code. Remember ? And, mentioned where I came to a stand still.
    Last edited by UniqueIdeaMan; October 4th, 2017 at 09:52 AM.
  14. #8
  15. Maddening Moderator
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    16,455
    Rep Power
    9645
    We're not going to just give you code. Get that in your head now. You're doing the exact same things I told you not to do elsewhere and if you keep it up then we will have the same result.

    The code you posted is the most rudimentary way of using cURL to get a page. Not only is it not the sort of thing you'll need but it doesn't even look like you put any effort into it in the first place.

    Catacaustic told you what to do: gather all the form information that you're trying to replicate and make cURL submit it in code. That probably means using CURLOPT_POST and CURL_POSTFIELDS. Learn how to use them, write some code that isn't obviously something copied and pasted from another site, and if you have problems then post what you have.
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Originally Posted by requinix
    We're not going to just give you code. Get that in your head now. You're doing the exact same things I told you not to do elsewhere and if you keep it up then we will have the same result.

    The code you posted is the most rudimentary way of using cURL to get a page. Not only is it not the sort of thing you'll need but it doesn't even look like you put any effort into it in the first place.

    Catacaustic told you what to do: gather all the form information that you're trying to replicate and make cURL submit it in code. That probably means using CURLOPT_POST and CURL_POSTFIELDS. Learn how to use them, write some code that isn't obviously something copied and pasted from another site, and if you have problems then post what you have.
    Good man! You gave me what I was looking for:
    CURLOPT_POST and CURL_POSTFIELDS

    Since I have built .exe bots before, I did understand Catacaustic's method. But, I needed the appropriate php/cURL functions to make use of the method. And he did not provide that. But, I'm glad to say: You did.
    I asked for a code snippet to sniff what functions to use. You did not give the snippet but you gave the functions names. That is a step for me, even though the snippet would have been better due to it having a sample on how to use the functions. Nevermind. I can google it. Good man. Keep it up!

    PS - Not sure what you mean by "you told me something elsewhere". You probably got your wires crossed somewhere along the line.
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Originally Posted by requinix
    We're not going to just give you code. Get that in your head now. You're doing the exact same things I told you not to do elsewhere and if you keep it up then we will have the same result.

    The code you posted is the most rudimentary way of using cURL to get a page. Not only is it not the sort of thing you'll need but it doesn't even look like you put any effort into it in the first place.

    Catacaustic told you what to do: gather all the form information that you're trying to replicate and make cURL submit it in code. That probably means using CURLOPT_POST and CURL_POSTFIELDS. Learn how to use them, write some code that isn't obviously something copied and pasted from another site, and if you have problems then post what you have.
    Ok. You don't have to give us newbies code samples to learn from. Plenty available on youtube. But atleast you can tell us which ones in your opinion are suitable and unsuitable code.
    What do you think about this code which is supposed to auto log you in ?
    https://pastebin.com/g8jX5hgy
    I found it here:
    https://www.youtube.com/watch?v=_kQN-3aNCeI
  20. #11
  21. Maddening Moderator
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    16,455
    Rep Power
    9645
    I count five reasons why it's stupid. Keep looking.
  22. #12
  23. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Originally Posted by requinix
    I count five reasons why it's stupid. Keep looking.
    Req my old pal!

    I have not tested the script yet. I just saw these mentioned:

    function login($url,$data)
    function grab_page($site)
    function post_data($site,$data)
    curl_setopt($datapost, CURLOPT_POSTFIELDS, $data);
    curl_setopt($datapost, CURLOPT_POST, TRUE);

    Note the last 2 is what you suggested I look into.
    Anyway, I will look into web automation with cURL (auto form submissions - kw searches, auto account signup, auto account login, etc. meaning actions related to web forms, auto data scraping, auto data generation, etc.) once I have completed my current task which is about to be finished soon. Then, my next project is to build web versions of my current .exe non-spamming & non-malicious genetlemen bots.
    In the meanwhile, take a rest. The storm of questions would be coming at you very very soon! Lol!

    Thanks!
    Last edited by UniqueIdeaMan; October 29th, 2017 at 08:04 PM.
  24. #13
  25. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Requinix and Catacaustic,

    Look what I'm reading.

    https://journalxtra.com/linux/bash-l...curl-and-wget/

    But, if you find anything better for this thread's topic then let me know.

    Cheers!
    Last edited by UniqueIdeaMan; October 30th, 2017 at 05:47 PM.
  26. #14
  27. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    334
    Rep Power
    1
    Hi,

    I was searching for a tutorial that teaches how cURL can auto post and came across the following code. It is supposed to fill-in WP comment form. But for some reason I see no response on the commenting section on the blog:
    https://www.atilus.com/top10internetmarketingblogs/

    Why is that ?
    If I run the script on my site then I see cURL fetches the page (the above link) but I see no response on the comment form.
    If I run the script on my Xampp (localhost) then I see a complete white blank page. Why ?

    PHP Code:
    <?php
    $postfields 
    = array();
    $postfields["action"] = "submit";
    $postfields["author"] = "programmer";
    $postfields["email"] = "email@email.com";
    $postfields["url"] = "http://www.example.com/";
    $postfields["comment"] = "Testing comment 1,2,3.";
    $postfields["comment_post_ID"] = "123";
    $postfields["_wp_unfiltered_html_comment"] = "0d870b294b";
    //Url of the form submission
    $url "https://www.atilus.com/top10internetmarketingblogs/";
    $useragent "Mozilla/5.0";
    $referer $url

    //Initialize CURL session
    $ch curl_init($url);
    //CURL options
    curl_setopt($chCURLOPT_POST1);
    //We post $postfields data
    curl_setopt($chCURLOPT_POSTFIELDS$postfields);
    //We define an useragent (Mozilla/5.0)
    curl_setopt($chCURLOPT_USERAGENT$useragent);
    //We define a refferer ($url)
    curl_setopt($chCURLOPT_REFERER$referer);
    //We get the result page in a string
    curl_setopt($chCURLOPT_RETURNTRANSFER1);
    //We exits CURL
    $result curl_exec($ch);
    curl_close($ch);

    //Finally, we display the result
    echo $result;
    ?>
    Last edited by UniqueIdeaMan; October 30th, 2017 at 05:51 PM.
  28. #15
  29. Banned (not really)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 1999
    Location
    Caro, Michigan
    Posts
    14,815
    Rep Power
    4554
    Why?
    Because you copy and paste **** without understanding it, probably.
    -- Cigars, whiskey and wild, wild women. --
Page 1 of 2 12 Last
  • Jump to page:

IMN logo majestic logo threadwatch logo seochat tools logo