Page 1 of 3 123 Last
  • Jump to page:
    #1
  1. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0

    Question Foreach Looping Levels - I got it right or wrong ?


    Php Buddies,

    What I am trying to do is learn to build a simple web crawler.
    So at first, I will feed it a url to start with.
    It will then fetch that page and extract all the links into a single array.
    Then it will fetch each of those links pages and extract all their links into a single array likewise. It will do this until it reaches it's max link deep level.
    Here is how I coded it:

    PHP Code:
    <?php 

    include('simple_html_dom.php'); 

    $current_link_crawling_level 0
    $link_crawling_level_max 2

    if($current_link_crawling_level == $link_crawling_level_max)
    {
        exit(); 
    }
    else
    {
        
    $url 'https://www.yahoo.com'
        
    $curl curl_init($url); 
        
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
        
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
        
    $html curl_exec($curl); 
        
        
    $current_link_crawling_level++;    

        
    //to fetch all hyperlinks from the webpage 
        
    $links = array(); 
        foreach(
    $html->find('a') as $a
        { 
            
    $links[] = $a->href
            echo 
    "Value: $value<br />\n"
            
    print_r($links); 
            
            
    $url '$value'
            
    $curl curl_init($value); 
            
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
            
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
            
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
            
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
            
    $html curl_exec($curl); 

            
    //to fetch all hyperlinks from the webpage 
            
    $links = array(); 
            foreach(
    $html->find('a') as $a
            { 
                
    $links[] = $a->href
                echo 
    "Value: $value<br />\n";
                
    print_r($links); 
                
    $current_link_crawling_level++;
            } 
        echo 
    "Value: $value<br />\n";
        
    print_r($links);  
    }

    ?>
    I have a feeling I got confused and messed it up in the foreach loops. Nestled too much. Is that the case ? Hint where I went wrong.

    Unable to test the script as I have to first sort out this error:
    Fatal error: Uncaught Error: Call to a member function find() on string in C:\xampp\h

    After that, I will be able to test it. Anyway, just looking at the script, you think I got it right or what ?

    Thanks
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2016
    Location
    Cheshire, UK
    Posts
    89
    Rep Power
    72
    Given that it's throwing a fatal error, I would have to say that I think you got it wrong.

    The error message tells you why it isn't working - as did Mr Fujin in your other post about this same error.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0
    Originally Posted by Barand
    Given that it's throwing a fatal error, I would have to say that I think you got it wrong.

    The error message tells you why it isn't working - as did Mr Fujin in your other post about this same error.
    Barand, I didn't notice. So, you are from England ? Nice to meet you, old man!
    Might aswell visit you one day on my visits to the Midlands.
    Not in UK at the moment though. .

    Anyway, apart from that error, how do you think I did building that crawler ?
    I get the feeling, I should not have used foreach loop but one of the other loops. What do you say ?
    Last edited by UniqueIdeaMan; May 20th, 2018 at 01:30 PM.
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0
    Originally Posted by Barand
    Given that it's throwing a fatal error, I would have to say that I think you got it wrong.

    The error message tells you why it isn't working - as did Mr Fujin in your other post about this same error.
    You know, I got rid of that error by adding these:

    PHP Code:
        $response_string curl_exec($curl); 

        
    $html str_get_html($response_string); 
    Like so ...

    PHP Code:
    $url 'http://php.net/manual-lookup.php?pattern=str_get_html&scope=quickref'
        
    $curl curl_init($url); 
        
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
        
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
        
    $response_string curl_exec($curl); 

        
    $html str_get_html($response_string); 
    Look over here for that:
    Fatal error: Uncaught Error: Call to a member function find() on string in C:\xampp\h

    Then I tried building my spider. Like so:
    PHP Code:

    <?php 

    include('simple_html_dom.php'); 

    $current_link_crawling_level 0
    $link_crawling_level_max 2;

    if(
    $current_link_crawling_level == $link_crawling_level_max)
    {
        echo 
    "link crawling depth level reached!"
        
    sleep(5);
        exit(); 
    }
    else
    {
        
    $url 'http://php.net/manual-lookup.php?pattern=str_get_html&scope=quickref'
        
    $curl curl_init($url); 
        
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
        
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
        
    $response_string curl_exec($curl); 

        
    $html str_get_html($response_string);
        
        
    $current_link_crawling_level++;    

        
    //to fetch all hyperlinks from the webpage 
        
    $links = array(); 
        foreach(
    $html->find('a') as $a
        { 
            
    $links[] = $a->href
            echo 
    "Value: $a<br />\n"
            
    print_r($links); 
            
            
    sleep(1);
            
            
    $url '$value'
            
    $curl curl_init($a); 
            
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
            
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
            
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
            
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
            
    $response_string curl_exec($curl); 

            
    $html str_get_html($response_string);
        
            
    $current_link_crawling_level++;    

            
    //to fetch all hyperlinks from the webpage 
            
    $links = array(); 
            foreach(
    $html->find('a') as $a
            { 
                
    $links[] = $a->href
                echo 
    "Value: $a<br />\n";
                
    print_r($links); 
                
                
    sleep(1);            
            } 
        echo 
    "Value: $a<br />\n";
        
    print_r($links); 
        }
    }

    ?>
    I get error:
    Value: php
    Array ( [0] => / )
    Fatal error: Uncaught Error: Call to a member function find() on boolean in C:\xampp\htdocs\cURL\crawler.php:64 Stack trace: #0 {main} thrown in C:\xampp\htdocs\cURL\crawler.php on line 64


    What is it wrong this time, folks ?
    And, how-about caring to show a fix to my sample ?
    Last edited by UniqueIdeaMan; May 20th, 2018 at 04:03 PM.
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2016
    Location
    Cheshire, UK
    Posts
    89
    Rep Power
    72
    As $html contains a boolean value then the most likely explanation is that str_get_html($response_string) failed and returned "false". Perhaps $response_string is empty. That's for you to check.
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0

    Thumbs up


    Originally Posted by Barand
    As $html contains a boolean value then the most likely explanation is that str_get_html($response_string) failed and returned "false". Perhaps $response_string is empty. That's for you to check.
    Thanks mate for the hint!
    However, saying all this. I get the feeling you would not have used the str_get_html($response_string).
    I'm very curious to have a peek at how you yourself would have coded it. The structure of your code I mean. For example, would you have used foreach or forloop or dowhile or anything else for that sake. I have a hunch you would've used the dowhile. But, we will never know unless we see a sample from your end.
    You're welcome to grab my code and wheeze your wand on it. I don't mind.
    I hope I have managed to talk you into getting your head into it until I see a working sample from your end.

    Anyway, reading:
    The two ways of iterating through arrays: list(), each(), and foreach loops
    Last edited by UniqueIdeaMan; May 20th, 2018 at 07:11 PM.
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0

    Question


    Originally Posted by Barand
    As $html contains a boolean value then the most likely explanation is that str_get_html($response_string) failed and returned "false". Perhaps $response_string is empty. That's for you to check.
    Barand,

    This is very very strange!
    I checked if $response_string is empty or not by echoing it. And guess what ? It's not empty because it echoes all the html of the page and I see the page fully loading on my screen. The error however still existed at the bottom of the page:

    Value: php
    Array ( [0] => / )
    Fatal error: Uncaught Error: Call to a member function find() on boolean in C:\xampp\htdocs\cURL\crawler.php:64 Stack trace: #0 {main} thrown in C:\xampp\htdocs\cURL\crawler.php on line 64

    Anyway, I then commented-out the echo and the exit(). That is as good as saying I deleted them. Guess what happened then ?
    I now see the page's extracted links echoes on screen as expected and the error gone. How did that happen ? Where's the error gone now and how did writing a line to echo the $response_string and then commenting-out that line do the job ? Absurd!
    What am I missing here, I wonder!
    The same happened with both scripts.
    Here are their updates now. Note the commented-out $response_string and the exit().
    But, on both scripts, I do get the following fatal error:
    Fatal error: Maximum execution time of 30 seconds exceeded in C:\xampp\htdocs\cURL\crawler_test.php on line

    The error occurs on this line:
    PHP Code:
    sleep(1); 
    Commenting-out all the sleeps() including this one:
    PHP Code:
    sleep(5); 
    is resulting in a bigger and longer page showing all the extracted links echoed on screen.
    But the error still remains.
    Fatal error: Maximum execution time of 30 seconds exceeded in C:\xampp\htdocs\cURL\crawler_test.php on line

    And it is referencing to this line (the 2nd $response_string and not the 1st one in the script):
    PHP Code:
    $response_string curl_exec($curl); 
    The 1st Script
    PHP Code:
    <?php 

    //13b. 

    /* FROM dom_crawler_NOTES.php file. 

    2. 
    FINDING HTML ELEMENTS BASED ON THEIR TAG NAMES 

    Suppose you wanted to find each and every link on a webpage.  
    We will be using “find” function to extract this information from the object. Here’s how to do it using Simple HTML DOM Parser : 
    */ 

    include('simple_html_dom.php'); 

    $current_link_crawling_level 0
    $link_crawling_level_max 2;

    if(
    $current_link_crawling_level == $link_crawling_level_max)
    {
        echo 
    "link crawling depth level reached!"
        
    //sleep(5);
        
    exit(); 
    }
    else
    {
        
    $url 'http://php.net/manual-lookup.php?pattern=str_get_html&scope=quickref'
        
    $curl curl_init($url); 
        
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
        
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
        
    $response_string curl_exec($curl); 
        
    //echo "$response_string";
        //exit();

        
    $html str_get_html($response_string);
        
        
    $current_link_crawling_level++;    

        
    //to fetch all hyperlinks from the webpage 
        // Hide HTML warnings
        
    libxml_use_internal_errors(true);
        
    $dom = new DOMDocument;
        if(
    $dom->loadHTML($htmlLIBXML_NOWARNING))
        {
            
    // echo Links and their anchor text
            
    echo '<pre>';
            echo 
    "Link\tAnchor\n";
            foreach(
    $dom->getElementsByTagName('a') as $link
            {
                
    $href $link->getAttribute('href');
                
    $anchor $link->nodeValue;
                echo 
    $href,"\t",$anchor,"\n";
                    
                
    //sleep(1);
                    
                
    $url 'http://php.net/manual-lookup.php?pattern=str_get_html&scope=quickref'
                
    $curl curl_init($url); 
                
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
                
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
                
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
                
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
                
    $response_string curl_exec($curl); 
                
    //echo "$response_string";
                //exit();

                
    $html str_get_html($response_string);
        
                
    $current_link_crawling_level++;    

                
    //to fetch all hyperlinks from the webpage 
                // Hide HTML warnings
                
    libxml_use_internal_errors(true);
                
    $dom = new DOMDocument;
                if(
    $dom->loadHTML($htmlLIBXML_NOWARNING))
                {
                    
    // echo Links and their anchor text
                    
    echo '<pre>';
                    echo 
    "Link\tAnchor\n";
                    foreach(
    $dom->getElementsByTagName('a') as $link
                    {
                        
    $href $link->getAttribute('href');
                        
    $anchor $link->nodeValue;
                        echo 
    $href,"\t",$anchor,"\n";
                    
                        
    //sleep(1);
                    
    }
                    echo 
    '</pre>';
                }
                else
                {
                    echo 
    "Failed to load html.";
                }
            }
        }
        else
        {
            echo 
    "Failed to load html.";
        }
    }


    ?>
    On the 2nd script, I get that error:

    Fatal error: Maximum execution time of 30 seconds exceeded in C:\xampp\htdocs\cURL\crawler_test.php on line 73

    and it is referencing to this line:
    PHP Code:
    print_r($links); 
    It is the 2nd last print_r and print_r of the final foreach loop.
    PHP Code:
    <?php 

    //13a. 

    /* FROM dom_crawler_NOTES.php file. 

    2. 
    FINDING HTML ELEMENTS BASED ON THEIR TAG NAMES 

    Suppose you wanted to find each and every link on a webpage.  
    We will be using “find” function to extract this information from the object. Here’s how to do it using Simple HTML DOM Parser : 
    */ 



    include('simple_html_dom.php'); 

    $current_link_crawling_level 0
    $link_crawling_level_max 2;

    if(
    $current_link_crawling_level == $link_crawling_level_max)
    {
        echo 
    "link crawling depth level reached!"
        
    //sleep(5);
        
    exit(); 
    }
    else
    {
        
    $url 'http://php.net/manual-lookup.php?pattern=str_get_html&scope=quickref'
        
    $curl curl_init($url); 
        
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
        
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
        
    $response_string curl_exec($curl); 
        
    //echo "$response_string";
        //exit();

        
    $html str_get_html($response_string);
        
        
    $current_link_crawling_level++;    

        
    //to fetch all hyperlinks from the webpage 
        
    $links = array(); 
        foreach(
    $html->find('a') as $a
        { 
            
    $links[] = $a->href
            echo 
    "Value: $a<br />\n"
            
    print_r($links); 
            
            
    //sleep(1);
            
            
    $url '$value'
            
    $curl curl_init($a); 
            
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
            
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
            
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
            
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
            
    //$response_string = curl_exec($curl); 
            //echo "$response_string";
            //exit();

            
    $html str_get_html($response_string);
        
            
    $current_link_crawling_level++;    

            
    //to fetch all hyperlinks from the webpage 
            
    $links = array(); 
            foreach(
    $html->find('a') as $a
            { 
                
    $links[] = $a->href
                echo 
    "Value: $a<br />\n";
                
    print_r($links); 
                
                
    //sleep(1);            
            

        echo 
    "Value: $a<br />\n";
        
    print_r($links); 
        }
    }



    ?>
    So my question is:
    Why am I getting this error referenced to 2 different lines in 2 different scripts ?
    The 1st script references is to sleep(1); while the 2nd script references that error to print_r.
    I need an answer to this one by anyone.
    And another answer on how to eliminate this error once and for all ?
    And, how can I make this:
    Maximum execution time of 30 seconds exceeded
    exceed to the time I want ? Like 180 secs (3 mins) ?
    Anyone ?

    Saying all this, I am happy now. Progressing a little, even though a snail pace. My crawler will be ready soon, God Willing!
    What do you guys think ? I'm narrowing things down to the main points or main monetary power of the crawler so my searchengine makes money for it's users day in day out from 50% of the link clicks. To erase poverty from the world.
    Catacaustic will get a big SHOCK soon!

    Barand & Mr Fujin,

    You google and clickover to a site. See nothing special.
    You use my searchengine (SE) and clickover to the same site. What do you see extraordinary ? You see face pics of all the visitors who poured into the site via my SE. You click each pic and are able to chat with the visitors.
    Now, who's SE you gonna use ? Mine or that old timer google ?
    And that is not all. When you chat with the site visitors, they earn MONEY per click.
    Now you all like me. Well, atleast the people who will earn and make a living will.
    Now you know a little why catacaustic will get a shick soon when he sees google's usership taking a dive. Lol!
    And, if the other SEs copy me then more credit goes to me and my UNIQUE IDEAS.
    I haven't revealed all my ideas here. Just something to get you guys believing me again and hoping in me again that I do finish my projects and stay curious to learn what other poverty ridding features I have YET to disclose. I hope I am whetting Mr Fujin's appettite. He can now wake-up in the morning and use my SE and earn a LOT everyday on the "side".
    What do you say, Mr Fujin ? Would that be enough reward for helping me now and then on my "php venture" ? Lol! Take care man! Or, should I say: Sir!

    EDIT: Here's another spoiler ....
    If you list your link on google and your web tracker shows a lot of visitors hit the BACK button they you won't know which of your competitions they went to. Now, will you ?
    My SE would show you where they went. Lost visitor tracking. Lost visitor contacting.
    And not only that. When your competitions headover to your site (via my SE) you would see face pics of them. Now you'd know who is a visitor and who is a competitor.
    Plus, when your competitor visits your link, you'd earn money ppc.
    Anyway, not gonna say any more or Catacaustic would copy my ideas and build it before me.
    Last edited by UniqueIdeaMan; May 21st, 2018 at 06:54 AM.
  14. #8
  15. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2016
    Location
    Cheshire, UK
    Posts
    89
    Rep Power
    72
    $url = '$value';
    Amazing. That line is so wrong it defeated the error reporting mechanism.
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2016
    Location
    Cheshire, UK
    Posts
    89
    Rep Power
    72
    Originally Posted by UniqueIdeaMan
    Barand, I didn't notice. So, you are from England ? Nice to meet you, old man!
    Might aswell visit you one day on my visits to the Midlands.
    I am out on that day.
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0

    Smile


    Originally Posted by Barand
    I am out on that day.
    Out ? Ok, good. I will return back with your cheque then. I'm afraid you will now have to fly over to the Eastern parts of the world to get that chQ as it's only deliverable by hand after id verification.

    EDIT: Or, I'll wait for you till night by spending time with your "lovely jovely" family.
  20. #11
  21. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0
    Originally Posted by Barand
    Amazing. That line is so wrong it defeated the error reporting mechanism.
    Thanks for the hint. On the other hand, I might aswell invite you over for a steak at one of my 100's of places around the world anywhere in the world!

    Take care man! Or, woman!
  22. #12
  23. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0

    Thumbs up


    Originally Posted by Barand
    Amazing. That line is so wrong it defeated the error reporting mechanism.
    Yeah, I overlooked that. Fixed it with double quotes.
    And we both over looked this 2nd line:
    PHP Code:
    $url '$value'
    $curl curl_init($a); 
    It should be:
    $url = "$a";
    $curl = curl_init($url);


    And look at the 1st script. Note that the 2nd $url is the same as the 1st. It should not be so.
    The 2nd one should be the foreach value.
    Anyway, struggling to fix the 1st script as I have not started oop yet. All these "->" is doing my head in as I'm used to the mysqli procedural stuffs (old stuffs).
    Yeah, yeah! About time I jump into pdo. Let me finish this and then I should or could.
    Last edited by UniqueIdeaMan; May 21st, 2018 at 01:01 PM.
  24. #13
  25. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2016
    Location
    Cheshire, UK
    Posts
    89
    Rep Power
    72
    Yeah, I overlooked that. Fixed it with double quotes.
    There was more tha one problem

    - $value had not been defined.
    - However, because you still do not know how to assign string variables, it was treated as a literal string instead of a variable name.
    - Assigning it as you did to $url should have failed miserably as '$value' is not a URL.
    - Having done all that, you don't then even use the $url variable in your code, so more by bad programming than by design, you got away with it..
    - Instead of using the $url in cURL call you go on to use $a, which is a dom object, instead of its url.

    How the gears don't grind to a halt and your server go up in smoke is a source of great mystery to me.
  26. #14
  27. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0
    Originally Posted by Barand
    There was more tha one problem

    - $value had not been defined.
    - However, because you still do not know how to assign string variables, it was treated as a literal string instead of a variable name.
    - Assigning it as you did to $url should have failed miserably as '$value' is not a URL.
    - Having done all that, you don't then even use the $url variable in your code, so more by bad programming than by design, you got away with it..
    - Instead of using the $url in cURL call you go on to use $a, which is a dom object, instead of its url.

    How the gears don't grind to a halt and your server go up in smoke is a source of great mystery to me.
    Originally, it was looking like this:
    PHP Code:
    $url 'http://php.net/manual-lookup.php?pattern=str_get_html&scope=quickref'
    I then switched it to:
    PHP Code:
    $url '$value'
    But forgot to switch the single quotes to double.

    And, as for your comment:
    - Instead of using the $url in cURL call you go on to use $a, which is a dom object, instead of its url.

    I did spot this mistake of mine and fixed it before you mentioned it. Check my previous post:
    Foreach Looping Levels - I got it right or wrong ?

    As for your final comment:
    How the gears don't grind to a halt and your server go up in smoke is a source of great mystery to me.

    OFFTOPIC
    Keep it up. I like farm talk. I'm in a farm myself but in the "Eastern" part of the world and no longer in the "West".
    When I was in the West, I was in a city.
    Was in a city in the East too but now in a countryside.
    And, I got broad band here in the countryside in the "3rd world". Now, that is good news is it not ?
    It's costing me around $2USD or £1.50(British Sterlings) for 4GB (3G) for 7 days broad band on mobile sim. I just use a small router (small mobile phone size) to wirelessly send the signal to my computer.
    The 3rd world is progressing very fast compared to the West when it comes to cellular fones. Internet connection not too far behind. Everything cheap over here. For under £2 or $3 you can have dinner in restaurents with a stomache full! I've grown fat in the stomache here. (Look pregnant! Lol!).
    And a 3 bedroom flat or hse would cost £30/$50 per month here in the countryside while in the cities £100/$150 per month! Beat that in the UK, USA, Canada, Australia and New Zealand.
    That is why my money is not running out fast as it would've if I was back in the West. You can save 4 times as much in the 3rd world. Whatever it would cost you a year to live on in the West, you can live 2-4yrs in the 3rd world.
    And, people in the 3rd world are not that privacy fussy. They live on approx £2/$3 per day. That is why, when I get my web proxy, social network and searchengine out and get each of them to earn atleast £1/$1.50 per day with them then I will become famous like Micheal Jackson. Let us see if I can find a php programmer here to build me something when I see I am getting no-where building them myself.
    In the West, it might cost me $2k to get my SE built. But, over here, if I pay just a lousy $250 a month to some php programmer then that should be enough to make him happy to build it. I know. Because, I saw one building a php site for some UK solicitor for around that much.
    I hope you guys would still be friends with me and help me out in my php learning and not envy me for any of this. Afterall, it's not just the East that would benefit from my php ventures but the whole world and that includes the West.

    EDIT
    Catacaustic, are you a farm boy like me and Barand or what ? Got kangaroos everywhere ? All this time I thought you were a city boy. Maybe you are. Anyway, doesn't matter. If you're a farm boy then I'll be more eager than ever to see if I can earn you enough to get you to live in the towns.
    I once drove from Melbourne to Sydney. Stopped halfway in some small town and stayed the night in some hotel. That was nearly a dacade ago and so can't remember the unimportant small town or it's hotel.
    https://www.google.com/maps/dir/Sydn...!2d-37.8136276

    Blimey, Sydney is more congested than Melbourne even though it's smaller. Looked good too. But, for some reason the melbourne people looked more professional like London people. Like Requinix.
    Guys, if you ever visit Australia, forget to taste any ham burgers because it really badly stinks! Yuck! British beef better! Infact, British & Irish food taste way better than every country I have ever set my foot on (India, Russia, Germany, France, Belgium, Luxembourg and UK (England, Wales & Scotland) and Ireland (Republic of Ireland & Northern Ireland). I haven't listed all countries such as the middle eastern ones and the ones in the Indian sub-continent. And, I'm not gonna tell you which Western country I am from and so I might aswell forget mentioning New Zealand, Canada, Usa, etc. (if I'm from any of these).
    British beef. British steak! Ah! Best in the world. Catacaustic and Benenamen, you both should try British beef.
    I wonder if benny (the young Mr. Benanamen) lives in countryside too. Mmm.
    Last edited by UniqueIdeaMan; May 23rd, 2018 at 11:22 AM.
  28. #15
  29. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jan 2017
    Posts
    817
    Rep Power
    0
    Ok.
    With one of the scripts, I am now getting this error:
    Warning: DOMDocument::loadHTML(): Empty string supplied as input in

    Don't know why the 2nd one is failing to load the html.
    PHP Code:
    if($dom->loadHTML($htmlLIBXML_NOWARNING)) 
    Here is the update:
    PHP Code:
    <?php 


    include('simple_html_dom.php'); 

    $current_link_crawling_level 0
    $link_crawling_level_max 2;

    if(
    $current_link_crawling_level == $link_crawling_level_max)
    {
        echo 
    "link crawling depth level reached!"
        
    //sleep(5);
        
    exit(); 
    }
    else
    {
        
    $url 'http://php.net/manual-lookup.php?pattern=str_get_html&scope=quickref'
        
    $curl curl_init($url); 
        
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
        
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
        
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
        
    $response_string curl_exec($curl); 
        
    //echo "$response_string";
        //exit();

        
    $html str_get_html($response_string);
        
    //echo "$html"; 
        
    $current_link_crawling_level++;    
        echo 
    "$current_link_crawling_level++";

        
    //to fetch all hyperlinks from the webpage 
        // Hide HTML warnings
        
    libxml_use_internal_errors(true);
        
    $dom = new DOMDocument;
        if(
    $dom->loadHTML($htmlLIBXML_NOWARNING))
        {
            
    // echo Links and their anchor text
            
    echo '<pre>';
            echo 
    "Link\tAnchor\n";
            foreach(
    $dom->getElementsByTagName('a') as $link
            {
                
    $href $link->getAttribute('href');
                
    $anchor $link->nodeValue;
                echo 
    $href,"\t",$anchor,"\n";
                    
                
    //sleep(1);
                    
                
    $url "$href"
                
    $curl curl_init($url); 
                
    curl_setopt($curlCURLOPT_RETURNTRANSFER1); 
                
    curl_setopt($curlCURLOPT_FOLLOWLOCATION1); 
                
    curl_setopt($curlCURLOPT_SSL_VERIFYPEER0); 
                
    curl_setopt($curlCURLOPT_SSL_VERIFYHOST0); 
                
    $response_string curl_exec($curl); 
                
    //echo "$response_string";
                //exit();

                
    $html str_get_html($response_string);
                echo 
    "$html"
                
    $current_link_crawling_level++;    
                echo 
    "$current_link_crawling_level++";

                
    //to fetch all hyperlinks from the webpage 
                // Hide HTML warnings
                
    libxml_use_internal_errors(true);
                
    $dom = new DOMDocument;
                if(
    $dom->loadHTML($htmlLIBXML_NOWARNING))
                {
                    
    // echo Links and their anchor text
                    
    echo '<pre>';
                    echo 
    "Link\tAnchor\n";
                    foreach(
    $dom->getElementsByTagName('a') as $link
                    {
                        
    $href $link->getAttribute('href');
                        
    $anchor $link->nodeValue;
                        echo 
    $href,"\t",$anchor,"\n";
                    
                        
    //sleep(1);
                    
    }
                    echo 
    '</pre>';
                }
                else
                {
                    echo 
    "Failed to load html (2nd one).";
                }
            }
        }
        else
        {
            echo 
    "Failed to load html (first one).";
        }
    }

    ?>
    What do you think is wrong ? Why is it failing to load the html ?
    I echoed the html and it got echoed. Check my code and you will see the echo is commented-out now.
Page 1 of 3 123 Last
  • Jump to page:

IMN logo majestic logo threadwatch logo seochat tools logo