Why does cURL return error 23 Failed writing body

Question

It works ok as a single tool   curl  someURL  curl -o -  someURL    but it doesn t work in a pipeline   curl  someURL    tr -d   n  curl -o -  someURL    tr -d   n    it returns    23  Failed writing body   What is the problem with piping the cURL output  How to buffer the whole cURL output and then handle it

User · Answer

In Bash and zsh  and perhaps other shells   you can use process substitution  Bash zsh  to create a file on the fly  and then use that as input to the next process in the pipeline chain   For example  I was trying to parse JSON output from cURL using jq and less  but was getting the Failed writing body error     Note  this does NOT work curl https   gitlab com api v4 projects    jq   less   When I rewrote it using process substitution  it worked     this works  jq     lt  curl https   gitlab com api v4 projects     less   Note  jq uses its 2nd argument to specify an input file  Bonus  If you re using jq like me and want to keep the colorized output in less  use the following command line instead   jq -C     lt  curl https   gitlab com api v4 projects     less -r      Thanks to Kowaru for their explanation of why Failed writing body was occurring  However  their solution of using tac twice didn t work for me  I also wanted to find a solution that would scale better for large files and tries to avoid the other issues noted as comments to that answer

User · Answer

In my case  I was doing  curl  lt blabla gt    jq   grep  lt blibli gt  With jq   it worked  curl  lt blabla gt    jq     grep  lt blibli gt

User · Answer

If you are trying something similar like source  lt   curl -sS  url   and getting the  23  Failed writing body error  it is because sourcing a process substitution doesn t work in bash 3 2  the default for macOS    Instead  you can use this workaround   source  dev stdin  lt  lt  lt     curl -sS  url

User · Answer

Another possibility  if using the -o  output file  option - the destination directory does not exist   eg  if you have -o  tmp download abc txt and  tmp download does not exist   Hence  ensure any required directories are created exist beforehand  use the --create-dirs option as well as -o if necessary

User · Answer

This happens when a piped program  e g  grep  closes the read pipe before the previous program is finished writing the whole page   In curl  url    grep -qs foo  as soon as grep has what it wants it will close the read stream from curl  cURL doesn t expect this and emits the  Failed writing body  error   A workaround is to pipe the stream through an intermediary program that always reads the whole page before feeding it to the next program   E g   curl  url    tac   tac   grep -qs foo   tac is a simple Unix program that reads the entire input page and reverses the line order  hence we run it twice   Because it has to read the whole input to find the last line  it will not output anything to grep until cURL is finished  Grep will still close the read stream when it has what it s looking for  but it will only affect tac  which doesn t emit an error

User · Answer

So it was a problem of encoding  Iconv solves the problem  curl  http   www multitran ru c m exe CL 1 amp s hello amp l1 1    iconv -f windows-1251   tr -dc    print

User · Answer

For completeness and future searches   It s a matter of how cURL manages the buffer  the buffer disables the output stream with the -N option   Example  curl -s -N  URL    grep -q Welcome

User · Answer

For me  it was permission issue   Docker run is called with a user profile but root is the user inside the container  The solution was to make curl write to  tmp since that has write permission for all users    not just root   I used the -o option   -o  tmp file to download

User · Answer

You can do this instead of using -o option   curl  url   gt   file

User · Answer

The server ran out of disk space  in my case   Check for it with df -k    I was alerted to the lack of disk space when I tried piping through tac twice  as described in one of the other answers  https   stackoverflow com a 28879552 336694  It showed me the error message write error  No space left on device

User · Answer

I had the same error but from different reason  In my case I had  tmpfs  partition with only 1GB space and I was downloading big file which finally filled all memory on that partition and I got the same error as you

User · Answer

I encountered this error message while trying to install varnish cache on ubuntu  The google search landed me here for the error  23  Failed writing body  hence posting a solution that worked for me    The bug is encountered while running the command as root curl -L https   packagecloud io varnishcache varnish5 gpgkey   apt-key add -  the solution is to run apt-key add as non root   curl -L https   packagecloud io varnishcache varnish5 gpgkey   apt-key add -

User · Answer

I encountered the same problem when doing  curl -L https   packagecloud io golang-migrate migrate gpgkey   apt-key add - The above query needs to be executed using root privileges  Writing it in following way solved the issue for me  curl -L https   packagecloud io golang-migrate migrate gpgkey   sudo apt-key add - If you write sudo before curl  you will get the Failed writing body error

[macos] Why does cURL return error "(23) Failed writing body"?

Examples related to macos

Examples related to bash

Examples related to curl

Examples related to osx-lion

Examples related to pipe