Trouble with a cURL request in PHP

Dev Shed Forums Sponsor:

May 15th, 2007, 10:27 PM

tekomp

Contributing User

Join Date: Aug 2005

Posts: 52

Time spent in forums: 16 h 41 m 55 sec
Reputation Power: 7

Trouble with a cURL request in PHP

Hi,

I'm trying to do a cURL request with curl_init() and having problems with certain URLs that I can get to work fine through the shell. The request is for a URL with a query string that should do a 302 redirect, but when cURL tries to make the request, it first does a request for only the hostname. The problem is that the page on the base hostname, returns a 403.

Here's what I have:

PHP Code:


		
			
 $url = 'http://example.com/cgi-bin/script.cgi?url=http://www.example2.com';



$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $url);

curl_setopt($ch, CURLOPT_VERBOSE, true);

curl_setopt($ch, CURLOPT_HEADER, true);

curl_setopt($ch, CURLOPT_NOBODY, true);

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

curl_setopt($ch, CURLOPT_TIMEOUT, 20);

curl_setopt($ch, CURLOPT_AUTOREFERER, true);

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

$ret = curl_exec($ch);

$info = curl_getinfo($ch);

curl_close($ch);



print_r($info);

Here's the output:

Code:

* About to connect() to example.com port 80 (#0)
*   Trying 12.12.12.12... * connected
* Connected to example.com (12.12.12.12) port 80 (#0)
> HEAD /cgi-bin/script.cgi?url=http://example2.com HTTP/1.1
Host: example.com
Accept: */*

< HTTP/1.1 403 Forbidden
< Date: Wed, 16 May 2007 01:53:02 GMT
< Server: Apache
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
* Closing connection #0

Array
(
    [url] => http://example.com/cgi-bin/script.cgi?url=http://www.example2.com
    [content_type] => text/html; charset=iso-8859-1
    [http_code] => 403
    [redirect_count] => 0
)

If I do it through the shell, it shows the redirect just fine:

Code:

%curl -i 'http://example.com/cgi-bin/script.cgi?url=http://www.example2.com'
HTTP/1.1 302 Found
Date: Wed, 16 May 2007 02:17:22 GMT
Server: Apache
Location: http://www.example2.com
Content-Length: 232
Connection: close
Content-Type: text/html; charset=iso-8859-1

Any ideas on how I can make it so it doesn't do that request on the hostname and uses the entire URL, in order to not get that 403?

Thanks.

May 16th, 2007, 02:30 AM

tekomp

Contributing User

Join Date: Aug 2005

Posts: 52

Time spent in forums: 16 h 41 m 55 sec
Reputation Power: 7

I was able to get this working using:

curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

It was my understanding that GET was the default request method, so not sure why this changes it, but at any rate, it works now.

May 16th, 2007, 07:34 AM

printf

Contributing User

Dev Shed Intermediate (1500 - 1999 posts)

Join Date: Jan 2005

Posts: 1,586

Time spent in forums: 4 Weeks 1 h 34 m 20 sec
Reputation Power: 273

Just so you know, next time...

Anytime you set the option...

PHP Code:


		
			
 curl_setopt($ch, CURLOPT_NOBODY, true);

CURLOPT_NOBODY automatically shifts the request to a HEAD, type request, so most servers don't allow HEAD type requests, results most times in 403 errors, so you use...

PHP Code:


		
			
 curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

So cURL, will send the request as GET or POST, but still only really make a HEAD request because the CURLOPT_NOBODY flag was set to true, which causes cURL to read only up to the end of the response header.

February 14th, 2008, 05:46 PM

davidyin

Registered User

Join Date: Feb 2008

Posts: 1

Time spent in forums: 2 m 32 sec
Reputation Power: 0

Quote:

Originally Posted by printf

Just so you know, next time...

Anytime you set the option...

PHP Code:


		
			
 curl_setopt($ch, CURLOPT_NOBODY, true);

CURLOPT_NOBODY automatically shifts the request to a HEAD, type request, so most servers don't allow HEAD type requests, results most times in 403 errors, so you use...

PHP Code:


		
			
 curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

It really works on my PHP tools.