[linux] How do I pipe or redirect the output of curl -v?

For some reason the output always gets printed to the terminal, regardless of whether I redirect it via 2> or > or |. Is there a way to get around this? Why is this happening?

This question is related to linux unix curl

The answer is


This simple example shows how to capture curl output, and use it in a bash script

test.sh

function main
{
  \curl -vs 'http://google.com'  2>&1
  # note: add -o /tmp/ignore.png if you want to ignore binary output, by saving it to a file. 
}

# capture output of curl to a variable
OUT=$(main)

# search output for something using grep.
echo
echo "$OUT" | grep 302 
echo
echo "$OUT" | grep title 

Your URL probably has ampersands in it. I had this problem, too, and I realized that my URL was full of ampersands (from CGI variables being passed) and so everything was getting sent to background in a weird way and thus not redirecting properly. If you put quotes around the URL it will fix it.


Just my 2 cents. The below command should do the trick, as answered earlier

curl -vs google.com 2>&1

However if need to get the output to a file,

curl -vs google.com > out.txt 2>&1

should work.


The following worked for me:

Put your curl statement in a script named abc.sh

Now run:

sh abc.sh 1>stdout_output 2>stderr_output

You will get your curl's results in stdout_output and the progress info in stderr_output.


The answer above didn't work for me, what did eventually was this syntax:

curl https://${URL} &> /dev/stdout | tee -a ${LOG}

tee puts the output on the screen, but also appends it to my log.


I found the same thing: curl by itself would print to STDOUT, but could not be piped into another program.

At first, I thought I had solved it by using xargs to echo the output first:

curl -s ... <url> | xargs -0 echo | ...

But then, as pointed out in the comments, it also works without the xargs part, so -s (silent mode) is the key to preventing extraneous progress output to STDOUT:

curl -s ... <url> | perl  -ne 'print $1 if /<sometag>([^<]+)/'

The above example grabs the simple <sometag> content (containing no embedded tags) from the XML output of the curl statement.


If you need the output in a file you can use a redirect:

curl https://vi.stackexchange.com/ -vs >curl-output.txt 2>&1

Please be sure not to flip the >curl-output.txt and 2>&1, which will not work due to bash's redirection behavior.


Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to unix

Docker CE on RHEL - Requires: container-selinux >= 2.9 What does `set -x` do? How to find files modified in last x minutes (find -mmin does not work as expected) sudo: npm: command not found How to sort a file in-place How to read a .properties file which contains keys that have a period character using Shell script gpg decryption fails with no secret key error Loop through a comma-separated shell variable Best way to find os name and version in Unix/Linux platform Resource u'tokenizers/punkt/english.pickle' not found

Examples related to curl

What is the incentive for curl to release the library for free? curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number Converting a POSTMAN request to Curl git clone error: RPC failed; curl 56 OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 10054 How to post raw body data with curl? Curl : connection refused How to use the curl command in PowerShell? Curl to return http status code along with the response How to install php-curl in Ubuntu 16.04 curl: (35) SSL connect error