[bash] How can I pipe stderr, and not stdout?

I have a program that writes information to stdout and stderr, and I need to process the stderr with grep, leaving stdout aside.

Using a temporary file, one could do it in two steps:

command > /dev/null 2> temp.file
grep 'something' temp.file

But how can this be achieved without temp files, using one command and pipes?

I try follow, find it work as well,

command > /dev/null 2>&1 | grep 'something'

This will redirect command1 stderr to command2 stdin, while leaving command1 stdout as is.

exec 3>&1
command1 2>&1 >&3 3>&- | command2 3>&-
exec 3>&-

Taken from LDP

If you are using Bash, then use:

command >/dev/null |& grep "something"

http://www.gnu.org/software/bash/manual/bashref.html#Pipelines

I just came up with a solution for sending stdout to one command and stderr to another, using named pipes.

Here goes.

mkfifo stdout-target
mkfifo stderr-target
cat < stdout-target | command-for-stdout &
cat < stderr-target | command-for-stderr &
main-command 1>stdout-target 2>stderr-target

It's probably a good idea to remove the named pipes afterward.

Or to swap the output from standard error and standard output over, use:

command 3>&1 1>&2 2>&3

This creates a new file descriptor (3) and assigns it to the same place as 1 (standard output), then assigns fd 1 (standard output) to the same place as fd 2 (standard error) and finally assigns fd 2 (standard error) to the same place as fd 3 (standard output).

Standard error is now available as standard output and the old standard output is preserved in standard error. This may be overkill, but it hopefully gives more details on Bash file descriptors (there are nine available to each process).

It's much easier to visualize things if you think about what's really going on with "redirects" and "pipes." Redirects and pipes in bash do one thing: modify where the process file descriptors 0, 1, and 2 point to (see /proc/[pid]/fd/*).

When a pipe or "|" operator is present on the command line, the first thing to happen is that bash creates a fifo and points the left side command's FD 1 to this fifo, and points the right side command's FD 0 to the same fifo.

Next, the redirect operators for each side are evaluated from left to right, and the current settings are used whenever duplication of the descriptor occurs. This is important because since the pipe was set up first, the FD1 (left side) and FD0 (right side) are already changed from what they might normally have been, and any duplication of these will reflect that fact.

Therefore, when you type something like the following:

command 2>&1 >/dev/null | grep 'something'

Here is what happens, in order:

1. a pipe (fifo) is created. "command FD1" is pointed to this pipe. "grep FD0" also is pointed to this pipe
2. "command FD2" is pointed to where "command FD1" currently points (the pipe)
3. "command FD1" is pointed to /dev/null

So, all output that "command" writes to its FD 2 (stderr) makes its way to the pipe and is read by "grep" on the other side. All output that "command" writes to its FD 1 (stdout) makes its way to /dev/null.

If instead, you run the following:

command >/dev/null 2>&1 | grep 'something'

Here's what happens:

1. a pipe is created and "command FD 1" and "grep FD 0" are pointed to it
2. "command FD 1" is pointed to /dev/null
3. "command FD 2" is pointed to where FD 1 currently points (/dev/null)

So, all stdout and stderr from "command" go to /dev/null. Nothing goes to the pipe, and thus "grep" will close out without displaying anything on the screen.

Also note that redirects (file descriptors) can be read-only (<), write-only (>), or read-write (<>).

A final note. Whether a program writes something to FD1 or FD2, is entirely up to the programmer. Good programming practice dictates that error messages should go to FD 2 and normal output to FD 1, but you will often find sloppy programming that mixes the two or otherwise ignores the convention.

You can use the rc shell.

First install the package (it's less than 1 MB).

This an example of how you would discard standard output and pipe standard error to grep in rc:

find /proc/ >[1] /dev/null |[2] grep task

You can do it without leaving Bash:

rc -c 'find /proc/ >[1] /dev/null |[2] grep task'

As you may have noticed, you can specify which file descriptor you want piped by using brackets after the pipe.

Standard file descriptors are numerated as such:

• 0 : Standard input
• 1 : Standard output
• 2 : Standard error

For those who want to redirect stdout and stderr permanently to files, grep on stderr, but keep the stdout to write messages to a tty:

# save tty-stdout to fd 3
exec 3>&1
# switch stdout and stderr, grep (-v) stderr for nasty messages and append to files
exec 2> >(grep -v "nasty_msg" >> std.err) >> std.out
# goes to the std.out
echo "my first message" >&1
# goes to the std.err
echo "a error message" >&2
# goes nowhere
echo "this nasty_msg won't appear anywhere" >&2
# goes to the tty
echo "a message on the terminal" >&3

This also works (and I find it a tiny bit easier to remember)

command 2> /dev/fd/1 | grep 'something'

More info about /dev/fd directory here

In Bash, you can also redirect to a subshell using process substitution:

command > >(stdlog pipe)  2> >(stderr pipe)

For the case at hand:

command 2> >(grep 'something') >/dev/null

Combining the best of these answers, if you do:

command 2> >(grep -v something 1>&2)

...then all stdout is preserved as stdout and all stderr is preserved as stderr, but you won't see any lines in stderr containing the string "something".

This has the unique advantage of not reversing or discarding stdout and stderr, nor smushing them together, nor using any temporary files.