#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2005
    Posts
    52
    Rep Power
    14

    Trouble with a cURL request in PHP


    Hi,

    I'm trying to do a cURL request with curl_init() and having problems with certain URLs that I can get to work fine through the shell. The request is for a URL with a query string that should do a 302 redirect, but when cURL tries to make the request, it first does a request for only the hostname. The problem is that the page on the base hostname, returns a 403.

    Here's what I have:

    PHP Code:
    $url 'http://example.com/cgi-bin/script.cgi?url=http://www.example2.com';

    $ch curl_init();
    curl_setopt($chCURLOPT_URL$url);
    curl_setopt($chCURLOPT_VERBOSEtrue);
    curl_setopt($chCURLOPT_HEADERtrue);
    curl_setopt($chCURLOPT_NOBODYtrue);
    curl_setopt($chCURLOPT_RETURNTRANSFERtrue);
    curl_setopt($chCURLOPT_TIMEOUT20);
    curl_setopt($chCURLOPT_AUTOREFERERtrue);
    curl_setopt($chCURLOPT_FOLLOWLOCATIONtrue);
    $ret curl_exec($ch);
    $info curl_getinfo($ch);
    curl_close($ch);

    print_r($info); 
    Here's the output:

    Code:
    * About to connect() to example.com port 80 (#0)
    *   Trying 12.12.12.12... * connected
    * Connected to example.com (12.12.12.12) port 80 (#0)
    > HEAD /cgi-bin/script.cgi?url=http://example2.com HTTP/1.1
    Host: example.com
    Accept: */*
    
    < HTTP/1.1 403 Forbidden
    < Date: Wed, 16 May 2007 01:53:02 GMT
    < Server: Apache
    < Connection: close
    < Content-Type: text/html; charset=iso-8859-1
    * Closing connection #0
    
    Array
    (
        [url] => http://example.com/cgi-bin/script.cgi?url=http://www.example2.com
        [content_type] => text/html; charset=iso-8859-1
        [http_code] => 403
        [redirect_count] => 0
    )

    If I do it through the shell, it shows the redirect just fine:

    Code:
    %curl -i 'http://example.com/cgi-bin/script.cgi?url=http://www.example2.com'
    HTTP/1.1 302 Found
    Date: Wed, 16 May 2007 02:17:22 GMT
    Server: Apache
    Location: http://www.example2.com
    Content-Length: 232
    Connection: close
    Content-Type: text/html; charset=iso-8859-1
    Any ideas on how I can make it so it doesn't do that request on the hostname and uses the entire URL, in order to not get that 403?

    Thanks.
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2005
    Posts
    52
    Rep Power
    14
    I was able to get this working using:

    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

    It was my understanding that GET was the default request method, so not sure why this changes it, but at any rate, it works now.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Jan 2005
    Posts
    1,579
    Rep Power
    279
    Just so you know, next time...

    Anytime you set the option...

    PHP Code:
    curl_setopt($chCURLOPT_NOBODYtrue); 
    CURLOPT_NOBODY automatically shifts the request to a HEAD, type request, so most servers don't allow HEAD type requests, results most times in 403 errors, so you use...

    PHP Code:
    curl_setopt($chCURLOPT_CUSTOMREQUEST'GET'); 
    So cURL, will send the request as GET or POST, but still only really make a HEAD request because the CURLOPT_NOBODY flag was set to true, which causes cURL to read only up to the end of the response header.
  6. #4
  7. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2008
    Posts
    1
    Rep Power
    0
    Originally Posted by printf
    Just so you know, next time...

    Anytime you set the option...

    PHP Code:
    curl_setopt($chCURLOPT_NOBODYtrue); 
    CURLOPT_NOBODY automatically shifts the request to a HEAD, type request, so most servers don't allow HEAD type requests, results most times in 403 errors, so you use...

    PHP Code:
    curl_setopt($chCURLOPT_CUSTOMREQUEST'GET'); 
    So cURL, will send the request as GET or POST, but still only really make a HEAD request because the CURLOPT_NOBODY flag was set to true, which causes cURL to read only up to the end of the response header.
    It really works on my PHP tools.

IMN logo majestic logo threadwatch logo seochat tools logo