[python] How to run a subprocess with Python, wait for it to exit and get the full stdout as a string?

So I noticed subprocess.call while it waits for the command to finish before proceeding with the python script, I have no way of getting the stdout, except with subprocess.Popen. Are there any alternative function calls that would wait until it finishes? (I also tried Popen.wait)

NOTE: I'm trying to avoid os.system call

result = subprocess.Popen([commands...,
                        self.tmpfile.path()], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = result.communicate()
print out+"HIHIHI"

my output:

HIHIHI

NOTE: I am trying to run wine with this.

If your process gives a huge stdout and no stderr, communicate() might be the wrong way to go due to memory restrictions.

Instead,

process = subprocess.Popen(cmd, shell=True,
                           stdout=subprocess.PIPE, 
                           stderr=subprocess.PIPE)

# wait for the process to terminate
for line in process.stdout: do_something(line)
errcode = process.returncode

might be the way to go.

process.stdout is a file-like object which you can treat as any other such object, mainly:

• you can read() from it
• you can readline() from it and
• you can iterate over it.

The latter is what I do above in order to get its contents line by line.

subprocess.check_output(...)

calls the process, raises if its error code is nonzero, and otherwise returns its stdout. It's just a quick shorthand so you don't have to worry about PIPEs and things.

I'd try something like:

#!/usr/bin/python
from __future__ import print_function

import shlex
from subprocess import Popen, PIPE

def shlep(cmd):
    '''shlex split and popen
    '''
    parsed_cmd = shlex.split(cmd)
    ## if parsed_cmd[0] not in approved_commands:
    ##    raise ValueError, "Bad User!  No output for you!"
    proc = Popen(parsed_command, stdout=PIPE, stderr=PIPE)
    out, err = proc.communicate()
    return (proc.returncode, out, err)

... In other words let shlex.split() do most of the work. I would NOT attempt to parse the shell's command line, find pipe operators and set up your own pipeline. If you're going to do that then you'll basically have to write a complete shell syntax parser and you'll end up doing an awful lot of plumbing.

Of course this raises the question, why not just use Popen with the shell=True (keyword) option? This will let you pass a string (no splitting nor parsing) to the shell and still gather up the results to handle as you wish. My example here won't process any pipelines, backticks, file descriptor redirection, etc that might be in the command, they'll all appear as literal arguments to the command. Thus it is still safer then running with shell=True ... I've given a silly example of checking the command against some sort of "approved command" dictionary or set --- through it would make more sense to normalize that into an absolute path unless you intend to require that the arguments be normalized prior to passing the command string to this function.

With Python 3.8 this workes for me. For instance to execute a python script within the venv:

    import subprocess
    import sys
    res = subprocess.run([
              sys.executable,            # venv3.8/bin/python
              'main.py', '--help',], 
            stdout=PIPE, 
            text=True)
    print(res.stdout)