Page 1 of 2 12 Last
  • Jump to page:
    #1
  1. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0

    Question Why cURL Fails To fetch Page ?


    Folks,

    Why is cURL failing to fetch the page ? All this time it worked.
    Echoes "Page fetching problem!"

    PHP Code:
    <?php 

    //Required PHP Files.
    include 'config.php';
    include 
    'header.php';

    //1). Set Banned Words.
    $banned_words = array("asshole""nut""bull****");

    $url "http://devshed.com";
    // 2). $curl is going to be data type curl resource.
    $curl curl_init();

    // 3). Set cURL options.
    curl_setopt($curlCURLOPT_URL"$url");
    curl_setopt($curlCURLOPT_SSL_VERIFYPEERfalse);
    curl_setopt($curlCURLOPT_RETURNTRANSFERtrue);

    // 4). Run cURL (execute http request).
    $result curl_exec($curl);

    if (
    curl_errno($curl))
    {
        echo 
    'Error:' curl_error($curl);
    }

    $response curl_getinfo$curl );

    //If page is fetched then replace banned words found on page.
    if($response['http_code'] == '200' )
    {   
        
    $regex '/\b';
        
    $regex .= implode('\b|\b'$banned_words);
        
    $regex .= '\b/i';
        
    $substitute 'BANNED WORD REPLACED';
        
    $clean_result preg_replace($regex$substitute$result);
        
    //Present the banned words filtered webpage.
        
    echo $clean_result;
    }
    else
    {
        
    //Show error if page fetching failed.
        
    echo "Page fetching problem!";
        exit();
    }


    ?>
  2. #2
  3. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,447
    Rep Power
    2105
    So what does the CURL call actually return? What's the status code, and the content? Look at those and you'll find the actual problem.

    Seriously, you need to go back to a very (and I mean very) basic programming course to learn some of the basic debugging techniques that they teach in the first couple of weeks. You keep on making the same mistakes over and over again, and still have got no idea on how to handle an unexpected error.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0
    Originally Posted by Catacaustic
    So what does the CURL call actually return? What's the status code, and the content? Look at those and you'll find the actual problem.

    Seriously, you need to go back to a very (and I mean very) basic programming course to learn some of the basic debugging techniques that they teach in the first couple of weeks. You keep on making the same mistakes over and over again, and still have got no idea on how to handle an unexpected error.
    You mean I should have added this line:

    PHP Code:
    echo "$response[http_code]"
    Result I get is this: 403.

    Googling, I see that the page devshed.com is forbidden. Since I can view the page on y browser and not manage to get cURL to fetch it then I'm guessing devshed.com has put a measurement in place to foil proxies.
    Switched the url to another site and that got fetched.
    last night, even google did not get fetched and so I suspected maybe I accidently deleted something from my code but could not spot the deletion. Hence, opened this thread to see if anyone else spots it.
    Now, I understand, both devshed.com and google were foiling the cURL fetch or proxy fetch.

    How well did I do ?
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jul 2003
    Posts
    4,470
    Rep Power
    653
    You really need to learn how to use search engines (that would also help you understand what is needed to build one).

    403
    There are 10 kinds of people in the world. Those that understand binary and those that don't.
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0
    Originally Posted by gw1500se
    You really need to learn how to use search engines (that would also help you understand what is needed to build one).

    403
    Already mentioned in my previous post that googling I found-out 403 means page forbidden. Server refusing to serve the page. And, I already mentioned why the refusal.
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0
    I now get error that variable $url is undefined on line 72.

    Notice: Undefined variable: url in C:\xampp\htdocs\......

    If you check line 72, it says between double quotes:
    PHP Code:
    $url "http://devshed.com"
    Even if I change the url to a url who's page cURL is able to fetch, I still get the same error.
    This does not work either, with single quotes:

    PHP Code:
    $url 'http://devshed.com'
    This is very very strange! If the $url variable has not been defined then how is cURL able to fetch the page who's url is on the $url variable value ?
    Even though page gets fetched, I still see the error! Weird!
  12. #7
  13. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,447
    Rep Power
    2105
    Originally Posted by UniqueIdeaMan
    I now get error that variable $url is undefined on line 72.
    That's strange. The code that you've given us doesn't even go up to line 72...

    Kind of makes it hard for anyone here to see anything, don't you think?

    And, there's no (major) difference between double and single quotes when you're defining a string like that, so that won't be the issue. It's most likely somewhere in the lines above or below that line. But again, that's basic debugging that you should already know.
  14. #8
  15. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0
    Originally Posted by Catacaustic
    That's strange. The code that you've given us doesn't even go up to line 72...

    Kind of makes it hard for anyone here to see anything, don't you think?

    And, there's no (major) difference between double and single quotes when you're defining a string like that, so that won't be the issue. It's most likely somewhere in the lines above or below that line. But again, that's basic debugging that you should already know.
    Yeah, I checked the submitted code and only upto line 48.
    Can't remember what happened. Myabe, I shortened the code when submitting here so you only get the relevant ones. Checking my file, I see I got upto line 122 now as more work have been added. Anyway, I'm not working on the crawler tonight and so I'll bugger you guys some other night.
  16. #9
  17. Banned (not really)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 1999
    Location
    Caro, Michigan
    Posts
    14,958
    Rep Power
    4575
    I'll bugger you guys some other night
    Well, that's a little extreme, isn't it?
    -- Cigars, whiskey and wild, wild women. --
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0
    Originally Posted by Sepodati
    Well, that's a little extreme, isn't it?
    OFFTOPIC:
    Ooops! Sorry GUYS! I thought "buggering" was a slang for "bothering" or "pestering". Didn't know it meant "penetrating your ...." Ha! Ha! Ha! Lol!
    Back in 1998, someone said "bugger off" and it came to mind in my last post. Didn't bother checking the dictionary back then as I thought it was a slang for "bog off (get lost) and stop pestering". Lol!
    Just checked:
    https://www.google.com/search?q=bugg...hrome&ie=UTF-8



    Good thing you brought it to my attention or I would probably have said that to some woman who's annoying me and probably found myself getting nicked. Lol!

    EDIT: On the other hand, maybe my 1998 assumption was not far off. It seems different countries translate it differently and on some occassions 2 different meanins in the same county:
    https://en.wikipedia.org/wiki/Bugger

    Seems like my guessing the meaning back in 1998 was spot on as I meant it the American way in one way:
    The phrase "bugger off" (bug off in American English[citation needed]) means to go, or run, away; when used as a command it means "go away" ("get lost" or "leave me alone") and can also be used in much the same type of relatively offensive manner.


    Spot on here too as I also meant it the Welsh and Anglo Indian way the other way:
    The word may be used amongst friends in an affectionate way and is used as a vernacular noun to imply that one is very fond of something (I'm a bugger for Welsh cakes).[citation needed] It can also imply a negative tendency (He's a silly bugger for losing his keys) [i.e., He's a fool for often losing his keys].[citation needed]


    Again, I meant it the Usa aswell as the Indian way ...
    In some English speaking communities the word has been in use traditionally without any profane connotations. For instance, within the Anglo-Indian community in India the word bugger has been in use, in an affectionate manner, to address or refer to a close friend or fellow schoolmate. In the United States it can be a rough synonym to whippersnapper as in calling a young boy a "little bugger"


    You know this is something really co-incidental. How come I was thinking the Welsh and Anglo Indian way when my name is used both in Wales, Indian Subcontinent and Middle East ? Makes you wonder, right ? Anyway, I do have some kind of connection with India, Britain, Middle East and Usa, Canada, New Zealand, Australia. One country by race, one country by religion, two countries by nationality (dual citizenship) and the remaining countries by sort of history (based on type of rule). And no, I did not say it in order for you to put 2 and 2 together. Lol! Now, that is a revelation you guys might not be pleased with.

    When I actually meant "bugger you guys", I meant, I'll pester you so much that you'll feel like saying "Get lost!" but I'll grip you like a vice until you give me what I want. Won't take "no" for an answer.
    You know what, might aswell use the word often here from now on. Lol!
    Last edited by UniqueIdeaMan; February 22nd, 2018 at 08:30 AM.
  20. #11
  21. Banned (not really)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 1999
    Location
    Caro, Michigan
    Posts
    14,958
    Rep Power
    4575
    I already feel like saying get lost. Get lost.
    -- Cigars, whiskey and wild, wild women. --
  22. #12
  23. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0
    Originally Posted by Sepodati
    I already feel like saying get lost. Get lost.
    So, is this part coming true for you then (meaning you suspect I'm not from your country or from your race or from your religion and best I get lost ?) Lol!

    "Lol! Now, that is a revelation you guys might not be pleased with."

    Remember, Usa, Canada, Australia, New Zealand, Middle East, Africa, India have something common with Britain. You too if you live in any of these countries. All former colonies of Britain.
    Remember, Usa, Canada, Australia, New Zealand, Britain, Africa, have something common with Middle East. You too if you live in any of these countries. Judaism, Christianity, Islam, Zoroastrianism, Sabeanism all come from Middle East.

    PS - Read my previous post again as I updated it a little.
    PPS - I did say, I won't "get lost" without a sample code.
    Last edited by UniqueIdeaMan; February 16th, 2018 at 10:12 AM.
  24. #13
  25. Banned (not really)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 1999
    Location
    Caro, Michigan
    Posts
    14,958
    Rep Power
    4575
    PHP Code:
    GOTO away
    -- Cigars, whiskey and wild, wild women. --
  26. #14
  27. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    845
    Rep Power
    0
    Originally Posted by Sepodati
    PHP Code:
    GOTO away
    Ha! Ha! Shall I give you a REP for that ? Lol!
  28. #15
  29. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2017
    Location
    Worldwide
    Posts
    48
    Rep Power
    45
    When did this PROGRAMMING HELP FORUM become a chat board?
Page 1 of 2 12 Last
  • Jump to page:

IMN logo majestic logo threadwatch logo seochat tools logo