[python] IOError: [Errno 32] Broken pipe: Python

I have a very simple Python 3 script:

f1 = open('a.txt', 'r')
print(f1.readlines())
f2 = open('b.txt', 'r')
print(f2.readlines())
f3 = open('c.txt', 'r')
print(f3.readlines())
f4 = open('d.txt', 'r')
print(f4.readlines())
f1.close()
f2.close()
f3.close()
f4.close()

But it always says:

IOError: [Errno 32] Broken pipe

I saw on the internet all the complicated ways to fix this, but I copied this code directly, so I think that there is something wrong with the code and not Python's SIGPIPE.

I am redirecting the output, so if the above script was named "open.py", then my command to run would be:

open.py | othercommand

This question is related to python python-3.x sigpipe

The answer is


The top answer (if e.errno == errno.EPIPE:) here didn't really work for me. I got:

AttributeError: 'BrokenPipeError' object has no attribute 'EPIPE'

However, this ought to work if all you care about is ignoring broken pipes on specific writes. I think it's safer than trapping SIGPIPE:

try:
    # writing, flushing, whatever goes here
except BrokenPipeError:
    exit( 0 )

You obviously have to make a decision as to whether your code is really, truly done if you hit the broken pipe, but for most purposes I think that's usually going to be true. (Don't forget to close file handles, etc.)


Closes should be done in reverse order of the opens.


I know this is not the "proper" way to do it, but if you are simply interested in getting rid of the error message, you could try this workaround:

python your_python_code.py 2> /dev/null | other_command

To bring Alex L.'s helpful answer, akhan's helpful answer, and Blckknght's helpful answer together with some additional information:

  • Standard Unix signal SIGPIPE is sent to a process writing to a pipe when there's no process reading from the pipe (anymore).

    • This is not necessarily an error condition; some Unix utilities such as head by design stop reading prematurely from a pipe, once they've received enough data.
  • By default - i.e., if the writing process does not explicitly trap SIGPIPE - the writing process is simply terminated, and its exit code is set to 141, which is calculated as 128 (to signal termination by signal in general) + 13 (SIGPIPE's specific signal number).

  • By design, however, Python itself traps SIGPIPE and translates it into a Python IOError instance with errno value errno.EPIPE, so that a Python script can catch it, if it so chooses - see Alex L.'s answer for how to do that.

  • If a Python script does not catch it, Python outputs error message IOError: [Errno 32] Broken pipe and terminates the script with exit code 1 - this is the symptom the OP saw.

  • In many cases this is more disruptive than helpful, so reverting to the default behavior is desirable:

    • Using the signal module allows just that, as stated in akhan's answer; signal.signal() takes a signal to handle as the 1st argument and a handler as the 2nd; special handler value SIG_DFL represents the system's default behavior:

      from signal import signal, SIGPIPE, SIG_DFL
      signal(SIGPIPE, SIG_DFL) 
      

The problem is due to SIGPIPE handling. You can solve this problem using the following code:

from signal import signal, SIGPIPE, SIG_DFL
signal(SIGPIPE,SIG_DFL) 

See here for background on this solution. Better answer here.


I feel obliged to point out that the method using

signal(SIGPIPE, SIG_DFL) 

is indeed dangerous (as already suggested by David Bennet in the comments) and in my case led to platform-dependent funny business when combined with multiprocessing.Manager (because the standard library relies on BrokenPipeError being raised in several places). To make a long and painful story short, this is how I fixed it:

First, you need to catch the IOError (Python 2) or BrokenPipeError (Python 3). Depending on your program you can try to exit early at that point or just ignore the exception:

from errno import EPIPE

try:
    broken_pipe_exception = BrokenPipeError
except NameError:  # Python 2
    broken_pipe_exception = IOError

try:
    YOUR CODE GOES HERE
except broken_pipe_exception as exc:
    if broken_pipe_exception == IOError:
        if exc.errno != EPIPE:
            raise

However, this isn't enough. Python 3 may still print a message like this:

Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

Unfortunately getting rid of that message is not straightforward, but I finally found http://bugs.python.org/issue11380 where Robert Collins suggests this workaround that I turned into a decorator you can wrap your main function with (yes, that's some crazy indentation):

from functools import wraps
from sys import exit, stderr, stdout
from traceback import print_exc


def suppress_broken_pipe_msg(f):
    @wraps(f)
    def wrapper(*args, **kwargs):
        try:
            return f(*args, **kwargs)
        except SystemExit:
            raise
        except:
            print_exc()
            exit(1)
        finally:
            try:
                stdout.flush()
            finally:
                try:
                    stdout.close()
                finally:
                    try:
                        stderr.flush()
                    finally:
                        stderr.close()
    return wrapper


@suppress_broken_pipe_msg
def main():
    YOUR CODE GOES HERE

A "Broken Pipe" error occurs when you try to write to a pipe that has been closed on the other end. Since the code you've shown doesn't involve any pipes directly, I suspect you're doing something outside of Python to redirect the standard output of the Python interpreter to somewhere else. This could happen if you're running a script like this:

python foo.py | someothercommand

The issue you have is that someothercommand is exiting without reading everything available on its standard input. This causes your write (via print) to fail at some point.

I was able to reproduce the error with the following command on a Linux system:

python -c 'for i in range(1000): print i' | less

If I close the less pager without scrolling through all of its input (1000 lines), Python exits with the same IOError you have reported.


This can also occur if the read end of the output from your script dies prematurely

ie open.py | otherCommand

if otherCommand exits and open.py tries to write to stdout

I had a bad gawk script that did this lovely to me.