[linux] Confused about stdin, stdout and stderr?

I am rather confused with the purpose of these three files. If my understanding is correct, stdin is the file in which a program writes into its requests to run a task in the process, stdout is the file into which the kernel writes its output and the process requesting it accesses the information from, and stderr is the file into which all the exceptions are entered. On opening these files to check whether these actually do occur, I found nothing seem to suggest so!

What I would want to know is what exactly is the purpose of these files, absolutely dumbed down answer with very little tech jargon!

This question is related to linux stdout stdin stderr

The answer is


Here is a lengthy article on stdin, stdout and stderr:

To summarize:

Streams Are Handled Like Files

Streams in Linux—like almost everything else—are treated as though they were files. You can read text from a file, and you can write text into a file. Both of these actions involve a stream of data. So the concept of handling a stream of data as a file isn’t that much of a stretch.

Each file associated with a process is allocated a unique number to identify it. This is known as the file descriptor. Whenever an action is required to be performed on a file, the file descriptor is used to identify the file.

These values are always used for stdin, stdout, and stderr:

0: stdin
1: stdout
2: stderr

Ironically I found this question on stack overflow and the article above because I was searching for information on abnormal / non-standard streams. So my search continues.


Using ps -aux reveals current processes, all of which are listed in /proc/ as /proc/(pid)/, by calling cat /proc/(pid)/fd/0 it prints anything that is found in the standard output of that process I think. So perhaps,

/proc/(pid)/fd/0 - Standard Output File
/proc/(pid)/fd/1 - Standard Input File
/proc/(pid)/fd/2 - Standard Error File

for examplemy terminal window

But only worked this well for /bin/bash other processes generally had nothing in 0 but many had errors written in 2


stdin

Reads input through the console (e.g. Keyboard input). Used in C with scanf

scanf(<formatstring>,<pointer to storage> ...);

stdout

Produces output to the console. Used in C with printf

printf(<string>, <values to print> ...);

stderr

Produces 'error' output to the console. Used in C with fprintf

fprintf(stderr, <string>, <values to print> ...);

Redirection

The source for stdin can be redirected. For example, instead of coming from keyboard input, it can come from a file (echo < file.txt ), or another program ( ps | grep <userid>).

The destinations for stdout, stderr can also be redirected. For example stdout can be redirected to a file: ls . > ls-output.txt, in this case the output is written to the file ls-output.txt. Stderr can be redirected with 2>.


It would be more correct to say that stdin, stdout, and stderr are "I/O streams" rather than files. As you've noticed, these entities do not live in the filesystem. But the Unix philosophy, as far as I/O is concerned, is "everything is a file". In practice, that really means that you can use the same library functions and interfaces (printf, scanf, read, write, select, etc.) without worrying about whether the I/O stream is connected to a keyboard, a disk file, a socket, a pipe, or some other I/O abstraction.

Most programs need to read input, write output, and log errors, so stdin, stdout, and stderr are predefined for you, as a programming convenience. This is only a convention, and is not enforced by the operating system.


I'm afraid your understanding is completely backwards. :)

Think of "standard in", "standard out", and "standard error" from the program's perspective, not from the kernel's perspective.

When a program needs to print output, it normally prints to "standard out". A program typically prints output to standard out with printf, which prints ONLY to standard out.

When a program needs to print error information (not necessarily exceptions, those are a programming-language construct, imposed at a much higher level), it normally prints to "standard error". It normally does so with fprintf, which accepts a file stream to use when printing. The file stream could be any file opened for writing: standard out, standard error, or any other file that has been opened with fopen or fdopen.

"standard in" is used when the file needs to read input, using fread or fgets, or getchar.

Any of these files can be easily redirected from the shell, like this:

cat /etc/passwd > /tmp/out     # redirect cat's standard out to /tmp/foo
cat /nonexistant 2> /tmp/err   # redirect cat's standard error to /tmp/error
cat < /etc/passwd              # redirect cat's standard input to /etc/passwd

Or, the whole enchilada:

cat < /etc/passwd > /tmp/out 2> /tmp/err

There are two important caveats: First, "standard in", "standard out", and "standard error" are just a convention. They are a very strong convention, but it's all just an agreement that it is very nice to be able to run programs like this: grep echo /etc/services | awk '{print $2;}' | sort and have the standard outputs of each program hooked into the standard input of the next program in the pipeline.

Second, I've given the standard ISO C functions for working with file streams (FILE * objects) -- at the kernel level, it is all file descriptors (int references to the file table) and much lower-level operations like read and write, which do not do the happy buffering of the ISO C functions. I figured to keep it simple and use the easier functions, but I thought all the same you should know the alternatives. :)


For authoritative information about these files, check out the man pages, run the command on your terminal.

$ man stdout 

But for a simple answer, each file is for:

stdout for a stream out

stdin for a stream input

stderr for printing errors or log messages.

Each unix program has each one of those streams.


stderr will not do IO Cache buffering so if our application need to print critical message info (some errors ,exceptions) to console or to file use it where as use stdout to print general log info as it use IO Cache buffering there is a chance that before writing our messages to file application may close ,leaving debugging complex


A file with associated buffering is called a stream and is declared to be a pointer to a defined type FILE. The fopen() function creates certain descriptive data for a stream and returns a pointer to designate the stream in all further transactions. Normally there are three open streams with constant pointers declared in the header and associated with the standard open files. At program startup three streams are predefined and need not be opened explicitly: standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). When opened the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device

https://www.mkssoftware.com/docs/man5/stdio.5.asp


I think people saying stderr should be used only for error messages is misleading.

It should also be used for informative messages that are meant for the user running the command and not for any potential downstream consumers of the data (i.e. if you run a shell pipe chaining several commands you do not want informative messages like "getting item 30 of 42424" to appear on stdout as they will confuse the consumer, but you might still want the user to see them.

See this for historical rationale:

"All programs placed diagnostics on the standard output. This had always caused trouble when the output was redirected into a ?le, but became intolerable when the output was sent to an unsuspecting process. Nevertheless, unwilling to violate the simplicity of the standard-input-standard-output model, people tolerated this state of affairs through v6. Shortly thereafter Dennis Ritchie cut the Gordian knot by introducing the standard error ?le. That was not quite enough. With pipelines diagnostics could come from any of several programs running simultaneously. Diagnostics needed to identify themselves."


As a complement of the answers above, here is a sum up about Redirections: Redirections cheatsheet

EDIT: This graphic is not entirely correct.

The first example does not use stdin at all, it's passing "hello" as an argument to the echo command.

The graphic also says 2>&1 has the same effect as &> however

ls Documents ABC > dirlist 2>&1
#does not give the same output as 
ls Documents ABC > dirlist &>

This is because &> requires a file to redirect to, and 2>&1 is simply sending stderr into stdout


Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to stdout

Redirect echo output in shell script to logfile Reusing output from last command in Bash Running powershell script within python script, how to make python print the powershell output while it is running Scanf/Printf double variable C Python: How to get stdout after running os.system? How to open every file in a folder Redirect all output to file using Bash on Linux? What does it mean to write to stdout in C? logger configuration to log to file and print to stdout How to get Rails.logger printing to the console/stdout when running rspec?

Examples related to stdin

sys.stdin.readline() reads without prompt, returning 'nothing in between' How to read from stdin line by line in Node Scanf/Printf double variable C How to open every file in a folder Reading from stdin How do I write to a Python subprocess' stdin? How to read from a file or STDIN in Bash? Send string to stdin How do I read a string entered by the user in C? How to read from stdin with fgets()?

Examples related to stderr

How I can print to stderr in C? Python: How to get stdout after running os.system? Redirect all output to file using Bash on Linux? How to redirect both stdout and stderr to a file How to print to stderr in Python? How to redirect stderr to null in cmd.exe Confused about stdin, stdout and stderr? How can I pipe stderr, and not stdout? How to store standard error in a variable How to redirect and append both stdout and stderr to a file with Bash?