Hi there,

I've been working on a little script that crawls the web, but i can't find a way to add download limit to my script.

My script
PHP Code:
$master curl_multi_init();
$curl_arr = array();

// add additional curl options here
$std_options = array(CURLOPT_RETURNTRANSFER => true,
                     
CURLOPT_FOLLOWLOCATION => true);
$options = ($custom_options) ? ($std_options $custom_options) : $std_options;

// start the first batch of requests
foreach ($urls AS $uId => $url)
{
    
$ch curl_init();
    
$options[CURLOPT_URL] = $url['url'];
    
curl_setopt_array($ch$options);
    
curl_multi_add_handle($master$ch);
    
    
// set handle so we can find back the releated data...
    
$handles[$ch] = $uId;
}

do 
{
    while((
$execrun curl_multi_exec($master$running)) == CURLM_CALL_MULTI_PERFORM);

    if(
$execrun != CURLM_OK)
        break;

    
// a request was just completed -- find out which one
    
while ($done curl_multi_info_read($master)) 
    {   
        
$info curl_getinfo($done['handle']);
        
$curHandle $handles[$done['handle']];
        
        
$urls[$curHandle]['code'] = $info['http_code'];
        
        switch (
$info['http_code'])
        {
            case 
200:
                
$output curl_multi_getcontent($done['handle']);
            break;
            
            case 
301:
            case 
302:
            break;
            
            case 
404:
            break;
            
            default:
                
$urls[$curHandle]['errno'] = $curl_errno($done['handle']);
                
$urls[$curHandle]['error'] = $curl_error($done['handle']);
            break;
        }
        
        
// remove the curl handle that just completed
        
curl_multi_remove_handle($master$done['handle']);
    }

while (
$running);

curl_multi_close($master);
print_r($urls); 
I've found a piece of PHP code that would do the job, but don't know how to add it so it works as expected.

URL: www. phpkode .com /source/s/multicurl-class-library/multicurl-class-library/MultiCurl.class.php
Line: 136
Code:
PHP Code:
if (!$active || $mrc != CURLM_OK || curl_getinfo($chCURLINFO_SIZE_DOWNLOAD) >= $this->maxSize) {
            
$this->closeSession($i);
        } 
I'm missing something, but can't seem to find the way to add this (looking for over 2 days now...) can anyone help me here?