How do I pass a string into subprocess Popen using the stdin argument

Question

If I do the following   import subprocess from cStringIO import StringIO subprocess Popen   grep   f   stdout subprocess PIPE stdin StringIO  one ntwo nthree nfour nfive nsix n    communicate   0    I get   Traceback  most recent call last     File   lt stdin gt    line 1  in     File   build toolchain mac32 python-2 4 3 lib python2 4 subprocess py   line 533  in   init        p2cread  p2cwrite    File   build toolchain mac32 python-2 4 3 lib python2 4 subprocess py   line 830  in  get handles     p2cread   stdin fileno   AttributeError   cStringIO StringI  object has no attribute  fileno    Apparently a cStringIO StringIO object doesn t quack close enough to a file duck to suit subprocess Popen   How do I work around this

User · Answer

I am using python3 and found out that you need to encode your string before you can pass it into stdin   p   Popen   grep    f    stdout PIPE  stdin PIPE  stderr PIPE  out  err   p communicate input  one ntwo nthree nfour nfive nsix n  encode    print out

User · Answer

Apparently a cStringIO StringIO object doesn t quack close enough to   a file duck to suit subprocess Popen   I m afraid not   The pipe is a low-level OS concept  so it absolutely requires a file object that is represented by an OS-level file descriptor   Your workaround is the right one

User · Answer

On Python 3 7  do this   my data    whatever you want nshould match this f  subprocess run   grep    f    text True  input my data    and you ll probably want to add capture output True to get the output of running the command as a string   On older versions of Python  replace text True with universal newlines True   subprocess run   grep    f    universal newlines True  input my data

User · Answer

Beware that Popen communicate input s may give you trouble ifsis too big  because apparently the parent process will buffer it before forking the child subprocess  meaning it needs  twice as much  used memory at that point  at least according to the  under the hood  explanation and linked documentation found here   In my particular case swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned   and no memory was left to fork it   File   opt local stow python-2 7 2 lib python2 7 subprocess py   line 1130  in  execute child     self pid   os fork   OSError   Errno 12  Cannot allocate memory

User · Answer

There s a beautiful solution if you re using Python 3 4 or better  Use the input argument instead of the stdin argument  which accepts a bytes argument   output   subprocess check output        sed    s foo bar         input b foo       This works for check output and run  but not call or check call for some reason

User · Answer

Popen communicate   documentation   Note that if you want to send data to the process   s stdin  you need to create the Popen object with stdin PIPE  Similarly  to get anything other than None in the result tuple  you need to give stdout PIPE and or stderr PIPE too  Replacing os popen       pipe   os popen cmd   w   bufsize           gt      pipe   Popen cmd  shell True  bufsize bufsize  stdin PIPE  stdin   Warning Use communicate   rather than stdin write    stdout read   or stderr read   to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process   So your example could be written as follows  from subprocess import Popen  PIPE  STDOUT  p   Popen   grep    f    stdout PIPE  stdin PIPE  stderr STDOUT      grep stdout   p communicate input b one ntwo nthree nfour nfive nsix n   0  print grep stdout decode      - gt  four   - gt  five   - gt    On Python 3 5   3 6  for encoding   you could use subprocess run  to pass input as a string to an external command and get its exit status  and its output as a string back in one call     usr bin env python3 from subprocess import run  PIPE  p   run   grep    f    stdout PIPE          input  one ntwo nthree nfour nfive nsix n   encoding  ascii   print p returncode    - gt  0 print p stdout    - gt  four   - gt  five   - gt

User · Answer

I figured out this workaround    gt  gt  gt  p   subprocess Popen   grep   f   stdout subprocess PIPE stdin subprocess PIPE   gt  gt  gt  p stdin write b one ntwo nthree nfour nfive nsix n    expects a bytes type object  gt  gt  gt  p communicate   0   four nfive n   gt  gt  gt  p stdin close     Is there a better one

User · Answer

Ex  Dialog  2-way  with a Popen        p   subprocess Popen  Your Command Here                    stdout subprocess PIPE                   stderr subprocess STDOUT                   stdin PIPE                   shell True                   bufsize 0  p stdin write  START n   out   p stdout readline   while out    line   out   line   line rstrip   n      if  WHATEVER1  in line        pr   1       p stdin write  DO 1 n         out   p stdout readline         continue    if  WHATEVER2  in line        pr   2       p stdin write  DO 2 n         out   p stdout readline         continue                     out   p stdout readline    p wait

User · Answer

p   Popen   grep    f    stdout PIPE  stdin PIPE  stderr STDOUT      p stdin write  one n   time sleep 0 5  p stdin write  two n   time sleep 0 5  p stdin write  three n   time sleep 0 5  testresult   p communicate   0  time sleep 0 5  print testresult

User · Answer

I m a bit surprised nobody suggested creating a pipe  which is in my opinion the far simplest way to pass a string to stdin of a subprocess   read  write   os pipe   os write write   stdin input here   os close write   subprocess check call   your-command    stdin read

User · Answer

from subprocess import Popen  PIPE from tempfile import SpooledTemporaryFile as tempfile f   tempfile   f write  one ntwo nthree nfour nfive nsix n   f seek 0  print Popen    bin grep   f   stdout PIPE stdin f  stdout read   f close

[python] How do I pass a string into subprocess.Popen (using the stdin argument)?

Examples related to python

Examples related to subprocess

Examples related to stdin