[php] curl error 18 - transfer closed with outstanding read data remaining

when retrieving data from a URL using curl, I sometimes (in 80% of the cases) get

error 18: transfer closed with outstanding read data remaining

Part of the returned data is then missing. The weird thing is that this does never occur when the CURLOPT_RETURNTRANSFER is set to false, that is the curl_exec function doesn't return the data but displays the content directly.

What could be the problem? Can I set some of the options to avoid such behaviour?

I got this error when my server process got an exception midway during generating the response and simply closed the connection without saying goodbye. curl still expected data from the connection and complained (rightfully).

I had this problem working with pycurl and I solved it using

c.setopt(pycurl.HTTP_VERSION, pycurl.CURL_HTTP_VERSION_1_0) 

like Eric Caron says.

Encountered similar issue, my server is behind nginx. There's no error in web server's (Python flask) log, but some error messsage in nginx log.

[crit] 31054#31054: *269464 open() "/var/cache/nginx/proxy_temp/3/45/0000000453" failed (13: Permission denied) while reading upstream

I fixed this issue by correcting the permission of directory:

/var/cache/nginx

I got this error when my server ran out of disk space and closed the connection midway during generating the response and simply closed the connection

I got this error when i was accidentally downloading a file onto itself.
(I had created a symlink in an sshfs mount of the remote directory to make it available for download, forgot to switch the working directory, and used -OJ).

I guess it won’t really »help« you when you read this, since it means your file got trashed.

Seeing this error during the use of Guzzle as well. The following header fixed it for me:

'headers' => [
    'accept-encoding' => 'gzip, deflate',
],

I issued the request with Postman which gave me a complete response and no error. Then I started adding the headers that Postman sends to the Guzzle request and this was the one that fixed it.

I've solved this error by this way.

$ch = curl_init ();
curl_setopt ( $ch, CURLOPT_URL, 'http://www.someurl/' );
curl_setopt ( $ch, CURLOPT_TIMEOUT, 30);
ob_start();
$response = curl_exec ( $ch );
$data = ob_get_clean();
if(curl_getinfo($ch, CURLINFO_HTTP_CODE) == 200 ) success;

Error still occurs, but I can handle response data in variable.

I had the same problem, but managed to fix it by suppressing the 'Expect: 100-continue' header that cURL usually sends (the following is PHP code, but should work similarly with other cURL APIs):

curl_setopt($curl, CURLOPT_HTTPHEADER, array('Expect:'));

By the way, I am sending calls to the HTTP server that is included in the JDK 6 REST stuff, which has all kinds of problems. In this case, it first sends a 100 response, and then with some requests doesn't send the subsequent 200 response correctly.

The error string is quite simply exactly what libcurl sees: since it is receiving a chunked encoding stream it knows when there is data left in a chunk to receive. When the connection is closed, libcurl knows that the last received chunk was incomplete. Then you get this error code.

There's nothing you can do to avoid this error with the request unmodified, but you can try to work around it by issuing a HTTP 1.0 request instead (since chunked encoding won't happen then) but the fact is that this is most likely a flaw in the server or in your network/setup somehow.