How to execute a program or call a system command from Python

Question

How do you call an external command  as if I d typed it at the Unix shell or Windows command prompt  from within a Python script

User · Answer

I always use fabric for this things like   from fabric operations import local result   local  ls   capture True  print  Content  n s     result      But this seem to be a good tool  sh  Python subprocess interface    Look at an example   from sh import vgdisplay print vgdisplay   print vgdisplay  -v   print vgdisplay v True

User · Answer

Here is calling an external command and return or print the command s output   Python Subprocess check output is good for     Run command with arguments and return its output as a byte string    import subprocess proc   subprocess check output  ipconfig  all   print proc

User · Answer

Use the subprocess module in the standard library  import subprocess subprocess run   quot ls quot    quot -l quot     The advantage of subprocess run over os system is that it is more flexible  you can get the stdout  stderr  the  quot real quot  status code  better error handling  etc      Even the documentation for os system recommends using subprocess instead   The subprocess module provides more powerful facilities for spawning new processes and retrieving their results  using that module is preferable to using this function  See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes   On Python 3 4 and earlier  use subprocess call instead of  run  subprocess call   quot ls quot    quot -l quot

User · Answer

Calling an external command in Python  A simple way to call an external command is using os system       And this function returns the exit value of the command  But the drawback is we won t get stdout and stderr   ret   os system  some cmd sh   if ret    0       print  some cmd sh execution returned failure    Calling an external command in Python in background  subprocess Popen provides more flexibility for running an external command rather than using os system  We can start a command in the background and wait for it to finish  And after that we can get the stdout and stderr   proc   subprocess Popen     some cmd sh    stdout subprocess PIPE  print  waiting for     str proc pid  proc wait   print  some cmd sh execution finished   out  err    proc communicate   print  some cmd sh output       out   Calling a long running external command in Python in the background and stop after some time  We can even start a long running process in the background using subprocess Popen and kill it after sometime once its task is done   proc   subprocess Popen     some long run cmd sh    stdout subprocess PIPE    Do something else   Now some long run cmd sh exeuction is no longer needed  so kill it os system  kill -15     str proc pid   print  Output     proc communicate   0

User · Answer

os system is OK  but kind of dated   It s also not very secure   Instead  try subprocess   subprocess does not call sh directly and is therefore more secure than os system   Get more information here

User · Answer

In Windows you can just import the subprocess module and run external commands by calling subprocess Popen    subprocess Popen   communicate   and subprocess Popen   wait   as below     Python script to run a command line import subprocess  def execute cmd                   Purpose    To execute a command and return exit status         Argument   cmd - command to execute         Return     exit code             process   subprocess Popen cmd  shell True  stdout subprocess PIPE  stderr subprocess PIPE       result  error    process communicate        rc   process wait        if rc    0          print  Error  failed to execute command    cmd         print error     return result   def  command    tasklist   grep python  print  This process detail   n   execute command    Output   This process detail  python exe                     604 RDP-Tcp 0                  4      5 660 K

User · Answer

If you are not using user input in the commands  you can use this   from os import getcwd from subprocess import check output from shlex import quote  def sh command       return check output quote command   shell True  cwd getcwd    universal newlines True  strip     And use it as  branch   sh  git rev-parse --abbrev-ref HEAD     shell True will spawn a shell  so you can use pipe and such shell things sh  ps aux   grep python    This is very very handy for running hardcoded commands and processing its output  The universal lines True make sure the output is returned in a string instead of binary   cwd getcwd   will make sure that the command is run with the same working directory as the interpreter  This is handy for Git commands to work like the Git branch name example above   Some recipes   free memory in megabytes  sh  free -m   split   n   1  split   1  free space on   in percent sh  df -m     split   n   1  split   4  0 -1  CPU load sum map float  sh  ps -ef -o pcpu   split   n   1      But this isn t safe for user input  from the documentation      Security Considerations      Unlike some other popen functions  this implementation will never   implicitly call a system shell  This means that all characters    including shell metacharacters  can safely be passed to child   processes  If the shell is invoked explicitly  via shell True  it is   the application   s responsibility to ensure that all whitespace and   metacharacters are quoted appropriately to avoid shell injection   vulnerabilities       When using shell True  the shlex quote   function can be used to   properly escape whitespace and shell metacharacters in strings that   are going to be used to construct shell commands    Even using the shlex quote    it is good to keep a little paranoid when using user inputs on shell commands  One option is using a hardcoded command to take some generic output and filtering by user input  Anyway using shell False will make sure that only the exactly process that you want to execute will be executed or you get a No such file or directory error   Also there is some performance impact on shell True  from my tests it seems about 20  slower than shell False  the default    In  50   timeit  check output  ls -l  split    universal newlines True    number 1000  globals globals    Out 50   2 6801227919995654  In  51   timeit  check output  ls -l   universal newlines True  shell True    number 1000  globals globals    Out 51   3 243950183999914

User · Answer

os system does not allow you to store results  so if you want to store results in some list or something  a subprocess call works

User · Answer

Use   import subprocess  p   subprocess Popen  df -h   shell True  stdout subprocess PIPE  stderr subprocess PIPE  communicate   0  print p split   n     It gives nice output which is easier to work with     Filesystem      Size  Used Avail Use  Mounted on      dev sda6        32G   21G   11G  67        none            4 0K     0  4 0K   0   sys fs cgroup     udev            1 9G  4 0K  1 9G   1   dev     tmpfs           387M  1 4M  386M   1   run     none            5 0M     0  5 0M   0   run lock     none            1 9G   58M  1 9G   3   run shm     none            100M   32K  100M   1   run user      dev sda5       340G  222G  100G  69   home

User · Answer

As of Python 3 7 0 released on June 27th 2018  https   docs python org 3 whatsnew 3 7 html   you can achieve your desired result in the most powerful while equally simple way  This answer intends to show you the essential summary of various options in a short manner  For in-depth answers  please see the other ones   TL DR in 2021 The big advantage of os system      was its simplicity  subprocess is better and still easy to use  especially as of Python 3 5  import subprocess subprocess run  quot ls -a quot   shell True   Note  This is the exact answer to your question - running a command  like in a shell   Preferred Way If possible  remove the shell overhead and run the command directly  requires a list   import subprocess subprocess run   quot help quot    subprocess run   quot ls quot    quot -a quot     Pass program arguments in a list  Don t include   quot -escaping for arguments containing spaces   Advanced Use Cases Checking The Output The following code speaks for itself  import subprocess result   subprocess run   quot ls quot    quot -a quot    capture output True  text True  if  quot stackoverflow-logo png quot  in result stdout      print  quot You re a fan  quot   else      print  quot You re not a fan  quot    result stdout is all normal program output excluding errors  Read result stderr to get them  capture output True - turns capturing on  Otherwise result stderr and result stdout would be None  Available from Python 3 7  text True - a convenience argument added in Python 3 7 which converts the received binary data to Python strings you can easily work with  Checking the returncode Do if result returncode    127  print  quot The program failed for some weird reason quot   elif result returncode    0  print  quot The program succeeded quot   else  print  quot The program failed unexpectedly quot    If you just want to check if the program succeeded  returncode    0  and otherwise throw an Exception  there is a more convenient function  result check returncode    But it s Python  so there s an even more convenient argument check which does the same thing automatically for you  result   subprocess run      check True   stderr should be inside stdout You might want to have all program output inside stdout  even errors  To accomplish this  run result   subprocess run      stderr subprocess STDOUT   result stderr will then be None and result stdout will contain everything  using shell False with an argument string shell False expects a list of arguments  You might however  split an argument string on your own using shlex  import subprocess import shlex subprocess run shlex split  quot ls -a quot     That s it  Common Problems Chances are high you just started using Python when you come across this question  Let s look at some common problems   FileNotFoundError   Errno 2  No such file or directory   ls -a    ls -a   You re running a subprocess without shell True   Either use a list    quot ls quot    quot -a quot    or set shell True   TypeError        NoneType        Check that you ve set capture output True   TypeError  a bytes-like object is required  not        You always receive byte results from your program  If you want to work with it like a normal string  set text True   subprocess CalledProcessError  Command         returned non-zero exit status 1   Your command didn t run successfully  You could disable returncode checking or check your actual program s validity   TypeError  init   got an unexpected keyword argument        You re likely using a version of Python older than 3 7 0  update it to the most recent one available  Otherwise there are other answers in this StackOverflow post showing you older alternative solutions

User · Answer

After some research  I have the following code which works very well for me  It basically prints both stdout and stderr in real time   stdout result   1 stderr result   1   def stdout thread pipe       global stdout result     while True          out   pipe stdout read 1          stdout result   pipe poll           if out       and stdout result is not None              break          if out                    sys stdout write out              sys stdout flush     def stderr thread pipe       global stderr result     while True          err   pipe stderr read 1          stderr result   pipe poll           if err       and stderr result is not None              break          if err                    sys stdout write err              sys stdout flush     def exec command command  cwd None       if cwd is not None          print           join command       in     cwd     else          print           join command             p   subprocess Popen          command  stdout subprocess PIPE  stderr subprocess PIPE  cwd cwd            out thread   threading Thread name  stdout thread   target stdout thread  args  p        err thread   threading Thread name  stderr thread   target stderr thread  args  p         err thread start       out thread start        out thread join       err thread join        return stdout result   stderr result

User · Answer

Just to add to the discussion  if you include using a Python console  you can call external commands from IPython  While in the IPython prompt  you can call shell commands by prefixing      You can also combine Python code with the shell  and assign the output of shell scripts to Python variables   For instance   In  9   mylist    ls  In  10   mylist Out 10     file1     file2     file3

User · Answer

Sultan is a recent-ish package meant for this purpose  Provides some niceties around managing user privileges and adding helpful error messages   from sultan api import Sultan  with Sultan load sudo True  hostname  myserver com   as sultan    sultan yum  install -y tree   run

User · Answer

I quite like shell command for its simplicity   It s built on top of the subprocess module   Here s an example from the documentation    gt  gt  gt  from shell command import shell call  gt  gt  gt  shell call  ls   py   setup py  shell command py  test shell command py 0  gt  gt  gt  shell call  ls -l   py   -rw-r--r-- 1 ncoghlan ncoghlan  391 2011-12-11 12 07 setup py -rw-r--r-- 1 ncoghlan ncoghlan 7855 2011-12-11 16 16 shell command py -rwxr-xr-x 1 ncoghlan ncoghlan 8463 2011-12-11 16 17 test shell command py 0

User · Answer

To fetch the network id from the OpenStack Neutron      usr bin python import os netid    nova net-list   awk    External     print  2     temp   os popen netid  read       Here temp also contains new line   n     networkId   temp rstrip   print networkId    Output of nova net-list   -------------------------------------- ------------ ------    ID                                     Label        CIDR    -------------------------------------- ------------ ------    431c9014-5b5d-4b51-a357-66020ffbb123   test1        None     27a74fcd-37c0-4789-9414-9531b7e3f126   External     None     5a2712e9-70dc-4b0e-9281-17e02f4684c9   management   None     7aa697f5-0e60-4c15-b4cc-9cb659698512   Internal     None    -------------------------------------- ------------ ------    Output of print networkId   27a74fcd-37c0-4789-9414-9531b7e3f126

User · Answer

There are a lot of different ways to run external commands in Python  and all of them have their own plus sides and drawbacks   My colleagues and me have been writing Python system administration tools  so we need to run a lot of external commands  and sometimes you want them to block or run asynchronously  time-out  update every second  etc   There are also different ways of handling the return code and errors  and you might want to parse the output  and provide new input  in an expect kind of style   Or you will need to redirect stdin  stdout and stderr to run in a different tty  e g   when using screen    So you will probably have to write a lot of wrappers around the external command  So here is a Python module which we have written which can handle almost anything you would want  and if not  it s very flexible so you can easily extend it   https   github com hpcugent vsc-base blob master lib vsc utils run py  update  It doesn t work standalone and requires some of our other tools  and got a lot of specialised functionality over the years  so it might not be a drop in replacement for you  but can give you a lot of info on how the internals of python for running commands work and idea s on how to handle certain situations

User · Answer

Use subprocess      or for a very simple command   import os os system  cat testfile

User · Answer

Here are my two cents  In my view  this is the best practice when dealing with external commands     These are the return values from the execute method     pass  stdout  stderr   execute   ls   -la     home user desktop     This is the execute method     def execute cmdArray workingDir        stdout          stderr           try          try              process   subprocess Popen cmdArray cwd workingDir  stdout subprocess PIPE  stderr subprocess PIPE  bufsize 1          except OSError              return  False       ERROR   command         join cmdArray       could not get executed             for line in iter process stdout readline  b                  try                  echoLine   line decode  utf-8               except                  echoLine   str line               stdout    echoLine          for line in iter process stderr readline  b                  try                  echoLine   line decode  utf-8               except                  echoLine   str line               stderr    echoLine      except  KeyboardInterrupt SystemExit  as err          return  False    str err        process stdout close        returnCode   process wait       if returnCode    0 or stderr                return  False  stdout  stderr      else          return  True  stdout  stderr

User · Answer

MOST OF THE CASES  For the most of cases  a short snippet of code like this is all you are going to need  import subprocess import shlex  source    quot test txt quot  destination    quot test copy txt quot   base    quot cp  source   destination   quot  cmd   base format source source  destination destination  subprocess check call shlex split cmd    It is clean and simple   subprocess check call run command with arguments and wait for command to complete  shlex split split the string cmd using shell-like syntax  REST OF THE CASES  If this do not work for some specific command  most probably you have a problem with  command-line interpreters  The operating system chose the default one which is not suitable for your type of program or could not found an adequate one on the system executable path  Example  Using the redirection operator on a Unix system input 1    quot input 1 txt quot  input 2    quot input 2 txt quot  output    quot merged txt quot  base command    quot  bin bash -c  cat  input   gt  gt   output   quot   base command format input 1  output output  subprocess check call shlex split base command    base command format input 2  output output  subprocess check call shlex split base command     As it is stated in The Zen of Python  Explicit is better than implicit  So if using a Python  gt  3 6 function  it would look something like this  import subprocess import shlex  def run command cmd interpreter  str  command  str  - gt  None      base command   f quot  cmd interpreter  -c   command   quot      subprocess check call shlex split base command

User · Answer

subprocess check call is convenient if you don t want to test return values  It throws an exception on any error

User · Answer

Use   import os  cmd    ls -al   os system cmd    os - This module provides a portable way of using operating system-dependent functionality   For the more os functions  here is the documentation

User · Answer

This is how I run my commands  This code has everything you need pretty much  from subprocess import Popen  PIPE cmd    ls -l     p   Popen cmd   shell True  stdout PIPE  stderr PIPE  out  err   p communicate   print  Return code     p returncode print out rstrip    err rstrip

User · Answer

Here s a summary of the ways to call external programs and the advantages and disadvantages of each    os system  some command with args   passes the command and arguments to your system s shell   This is nice because you can actually run multiple commands at once in this manner and set up pipes and input output redirection   For example     os system  some command  lt  input file   another command  gt  output file        However  while this is convenient  you have to manually handle the escaping of shell characters such as spaces  etc   On the other hand  this also lets you run commands which are simply shell commands and not actually external programs   See the documentation    stream   os popen  some command with args   will do the same thing as os system except that it gives you a file-like object that you can use to access standard input output for that process   There are 3 other variants of popen that all handle the i o slightly differently   If you pass everything as a string  then your command is passed to the shell  if you pass them as a list then you don t need to worry about escaping anything   See the documentation  The Popen class of the subprocess module   This is intended as a replacement for os popen but has the downside of being slightly more complicated by virtue of being so comprehensive   For example  you d say   print subprocess Popen  echo Hello World   shell True  stdout subprocess PIPE  stdout read     instead of    print os popen  echo Hello World   read     but it is nice to have all of the options there in one unified class instead of 4 different popen functions   See the documentation  The call function from the subprocess module   This is basically just like the Popen class and takes all of the same arguments  but it simply waits until the command completes and gives you the return code   For example   return code   subprocess call  echo Hello World   shell True      See the documentation  If you re on Python 3 5 or later  you can use the new subprocess run function  which is a lot like the above but even more flexible and returns a CompletedProcess object when the command finishes executing  The os module also has all of the fork exec spawn functions that you d have in a C program  but I don t recommend using them directly    The subprocess module should probably be what you use   Finally please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it  There are serious security implications if any part of the string that you pass can not be fully trusted  For example  if a user is entering some any part of the string  If you are unsure  only use these methods with constants  To give you a hint of the implications consider this code   print subprocess Popen  echo  s     user input  stdout PIPE  stdout read     and imagine that the user enters something  my mama didnt love me  amp  amp  rm -rf    which could erase the whole filesystem

User · Answer

import subprocess  p   subprocess run   ls    -ltr    capture output True  print p stdout decode    p stderr decode      Try online

User · Answer

This can be done in 2 ways  1  os system    import os  To run a system command     os system  cls    To execute a program     os system  code1 py    2  subprocess call    import subprocess  To run a system command     subprocess call  cls  shell True   To execute a program     subprocess call  code1 py  shell True

User · Answer

Python 3 5   import subprocess  p   subprocess run   ls    -ltr    capture output True  print p stdout decode    p stderr decode      Try online

User · Answer

Use subprocess call     from subprocess import call    Using list call   echo    Hello    world       Single string argument varies across platforms so better split it call  echo Hello world  split

User · Answer

I tend to use subprocess together with shlex  to handle escaping of quoted strings     gt  gt  gt  import subprocess  shlex  gt  gt  gt  command    ls -l   your path with spaces     gt  gt  gt  call params   shlex split command   gt  gt  gt  print call params   ls    -l     your path with spaces     gt  gt  gt  subprocess call call params

User · Answer

With the standard library  Use the subprocess module  Python 3    import subprocess subprocess run   ls    -l      It is the recommended standard way  However  more complicated tasks  pipes  output  input  etc   can be tedious to construct and write   Note on Python version  If you are still using Python 2  subprocess call works in a similar way   ProTip  shlex split can help you to parse the command for run  call  and other subprocess functions in case you don t want  or you can t   provide them in form of lists   import shlex import subprocess subprocess run shlex split  ls -l      With external dependencies  If you do not mind external dependencies  use plumbum   from plumbum cmd import ifconfig print ifconfig  wlan0        It is the best subprocess wrapper  It s cross-platform  i e  it works on both Windows and Unix-like systems  Install by pip install plumbum   Another popular library is sh   from sh import ifconfig print ifconfig  wlan0      However  sh dropped Windows support  so it s not as awesome as it used to be  Install by pip install sh

User · Answer

If you re writing a Python shell script and have IPython installed on your system  you can use the bang prefix to run a shell command inside IPython    ls filelist    ls

User · Answer

It can be this simple   import os cmd    your command  os system cmd

User · Answer

import os os system  your command     Note that this is dangerous  since the command isn t cleaned  I leave it up to you to google for the relevant documentation on the  os  and  sys  modules  There are a bunch of functions  exec  and spawn   that will do similar things

User · Answer

Update   subprocess run is the recommended approach as of Python 3 5 if your code does not need to maintain compatibility with earlier Python versions  It s more consistent and offers similar ease-of-use as Envoy   Piping isn t as straightforward though  See this question for how    Here s some examples from the documentation   Run a process    gt  gt  gt  subprocess run   ls    -l       Doesn t capture output CompletedProcess args   ls    -l    returncode 0    Raise on failed run    gt  gt  gt  subprocess run  exit 1   shell True  check True  Traceback  most recent call last         subprocess CalledProcessError  Command  exit 1  returned non-zero exit status 1   Capture output    gt  gt  gt  subprocess run   ls    -l     dev null    stdout subprocess PIPE  CompletedProcess args   ls    -l     dev null    returncode 0  stdout b crw-rw-rw- 1 root root 1  3 Jan 23 16 23  dev null n     Original answer   I recommend trying Envoy  It s a wrapper for subprocess  which in turn aims to replace the older modules and functions  Envoy is subprocess for humans   Example usage from the README    gt  gt  gt  r   envoy run  git config   data  data to pipe in   timeout 2    gt  gt  gt  r status code 129  gt  gt  gt  r std out  usage  git config  options    gt  gt  gt  r std err      Pipe stuff around too    gt  gt  gt  r   envoy run  uptime   pbcopy     gt  gt  gt  r command  pbcopy   gt  gt  gt  r status code 0   gt  gt  gt  r history   lt Response  uptime  gt

User · Answer

Here are my two cents  In my view  this is the best practice when dealing with external commands     These are the return values from the execute method     pass  stdout  stderr   execute   ls   -la     home user desktop     This is the execute method     def execute cmdArray workingDir        stdout          stderr           try          try              process   subprocess Popen cmdArray cwd workingDir  stdout subprocess PIPE  stderr subprocess PIPE  bufsize 1          except OSError              return  False       ERROR   command         join cmdArray       could not get executed             for line in iter process stdout readline  b                  try                  echoLine   line decode  utf-8               except                  echoLine   str line               stdout    echoLine          for line in iter process stderr readline  b                  try                  echoLine   line decode  utf-8               except                  echoLine   str line               stderr    echoLine      except  KeyboardInterrupt SystemExit  as err          return  False    str err        process stdout close        returnCode   process wait       if returnCode    0 or stderr                return  False  stdout  stderr      else          return  True  stdout  stderr

User · Answer

As an example  in Linux    import subprocess subprocess run  mkdir test dir   shell True    This creates test dir in the current directory  Note that this also works   import subprocess subprocess call  mkdir test dir   shell True    The equivalent code using os system is   import os os system  mkdir test dir     Best practice would be to use subprocess instead of os  with  run favored over  call   All you need to know about subprocess is here  Also  note that all Python documentation is available for download from here  I downloaded the PDF packed as  zip  I mention this because there s a nice overview of the os module in tutorial pdf  page 81   Besides  it s an authoritative resource for Python coders

User · Answer

MOST OF THE CASES  For the most of cases  a short snippet of code like this is all you are going to need  import subprocess import shlex  source    quot test txt quot  destination    quot test copy txt quot   base    quot cp  source   destination   quot  cmd   base format source source  destination destination  subprocess check call shlex split cmd    It is clean and simple   subprocess check call run command with arguments and wait for command to complete  shlex split split the string cmd using shell-like syntax  REST OF THE CASES  If this do not work for some specific command  most probably you have a problem with  command-line interpreters  The operating system chose the default one which is not suitable for your type of program or could not found an adequate one on the system executable path  Example  Using the redirection operator on a Unix system input 1    quot input 1 txt quot  input 2    quot input 2 txt quot  output    quot merged txt quot  base command    quot  bin bash -c  cat  input   gt  gt   output   quot   base command format input 1  output output  subprocess check call shlex split base command    base command format input 2  output output  subprocess check call shlex split base command     As it is stated in The Zen of Python  Explicit is better than implicit  So if using a Python  gt  3 6 function  it would look something like this  import subprocess import shlex  def run command cmd interpreter  str  command  str  - gt  None      base command   f quot  cmd interpreter  -c   command   quot      subprocess check call shlex split base command

User · Answer

Here s a summary of the ways to call external programs and the advantages and disadvantages of each    os system  some command with args   passes the command and arguments to your system s shell   This is nice because you can actually run multiple commands at once in this manner and set up pipes and input output redirection   For example     os system  some command  lt  input file   another command  gt  output file        However  while this is convenient  you have to manually handle the escaping of shell characters such as spaces  etc   On the other hand  this also lets you run commands which are simply shell commands and not actually external programs   See the documentation    stream   os popen  some command with args   will do the same thing as os system except that it gives you a file-like object that you can use to access standard input output for that process   There are 3 other variants of popen that all handle the i o slightly differently   If you pass everything as a string  then your command is passed to the shell  if you pass them as a list then you don t need to worry about escaping anything   See the documentation  The Popen class of the subprocess module   This is intended as a replacement for os popen but has the downside of being slightly more complicated by virtue of being so comprehensive   For example  you d say   print subprocess Popen  echo Hello World   shell True  stdout subprocess PIPE  stdout read     instead of    print os popen  echo Hello World   read     but it is nice to have all of the options there in one unified class instead of 4 different popen functions   See the documentation  The call function from the subprocess module   This is basically just like the Popen class and takes all of the same arguments  but it simply waits until the command completes and gives you the return code   For example   return code   subprocess call  echo Hello World   shell True      See the documentation  If you re on Python 3 5 or later  you can use the new subprocess run function  which is a lot like the above but even more flexible and returns a CompletedProcess object when the command finishes executing  The os module also has all of the fork exec spawn functions that you d have in a C program  but I don t recommend using them directly    The subprocess module should probably be what you use   Finally please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it  There are serious security implications if any part of the string that you pass can not be fully trusted  For example  if a user is entering some any part of the string  If you are unsure  only use these methods with constants  To give you a hint of the implications consider this code   print subprocess Popen  echo  s     user input  stdout PIPE  stdout read     and imagine that the user enters something  my mama didnt love me  amp  amp  rm -rf    which could erase the whole filesystem

User · Answer

I have written a wrapper to handle errors and redirecting output and other stuff   import shlex import psutil import subprocess  def call cmd cmd  stdout sys stdout  quiet False  shell False  raise exceptions True  use shlex True  timeout None          Exec command by command line like  ln -ls   var log               if not quiet          print  Run  s   str cmd       if use shlex and isinstance cmd   str  unicode            cmd   shlex split cmd      if timeout is None          process   subprocess Popen cmd  stdout stdout  stderr sys stderr  shell shell          retcode   process wait       else          process   subprocess Popen cmd  stdout stdout  stderr sys stderr  shell shell          p   psutil Process process pid          finish  alive   psutil wait procs  p   timeout          if len alive   gt  0              ps   p children               ps insert 0  p              print  waiting for timeout again due to child process check               finish  alive   psutil wait procs ps  0          if len alive   gt  0              print  process    will be killed  format  p pid for p in alive                for p in alive                  p kill               if raise exceptions                  print  External program timeout at        format timeout  cmd                   raise CalledProcessTimeout 1  cmd          retcode   process wait       if retcode and raise exceptions          print  External program failed  s   str cmd           raise subprocess CalledProcessError retcode  cmd    You can call it like this   cmd    ln -ls   var log   stdout    out txt  call cmd cmd  stdout

User · Answer

Use   import subprocess  p   subprocess Popen  df -h   shell True  stdout subprocess PIPE  stderr subprocess PIPE  communicate   0  print p split   n     It gives nice output which is easier to work with     Filesystem      Size  Used Avail Use  Mounted on      dev sda6        32G   21G   11G  67        none            4 0K     0  4 0K   0   sys fs cgroup     udev            1 9G  4 0K  1 9G   1   dev     tmpfs           387M  1 4M  386M   1   run     none            5 0M     0  5 0M   0   run lock     none            1 9G   58M  1 9G   3   run shm     none            100M   32K  100M   1   run user      dev sda5       340G  222G  100G  69   home

User · Answer

Python 3 5   import subprocess  p   subprocess run   ls    -ltr    capture output True  print p stdout decode    p stderr decode      Try online

User · Answer

There are lots of different libraries which allow you to call external commands with Python  For each library I ve given a description and shown an example of calling an external command  The command I used as the example is ls -l  list all files   If you want to find out more about any of the libraries I ve listed and linked the documentation for each of them      Sources     subprocess  https   docs python org 3 5 library subprocess html shlex  https   docs python org 3 library shlex html os  https   docs python org 3 5 library os html sh  https   amoffat github io sh  plumbum  https   plumbum readthedocs io en latest  pexpect  https   pexpect readthedocs io en stable  fabric  http   www fabfile org  envoy  https   github com kennethreitz envoy commands  https   docs python org 2 library commands html                  These are all the libraries            Hopefully this will help you make a decision on which library to use        subprocess   Subprocess allows you to call external commands and connect them to their input output error pipes  stdin  stdout  and stderr   Subprocess is the default choice for running commands  but sometimes other modules are better   subprocess run   ls    -l      Run command subprocess run   ls    -l    stdout subprocess PIPE    This will run the command and return any output subprocess run shlex split  ls -l      You can also use the shlex library to split the command      os   os is used for  operating system dependent functionality   It can also be used to call external commands with os system and os popen  Note  There is also a subprocess popen   os will always run the shell and is a simple alternative for people who don t need to  or don t know how to use subprocess run   os system  ls -l     run command os popen  ls -l   read     This will run the command and return any output      sh   sh is a subprocess interface which lets you call programs as if they were functions  This is useful if you want to run a command multiple times   sh ls  -l     Run command normally ls cmd   sh Command  ls     Save command as a variable ls cmd     Run command as if it were a function      plumbum   plumbum is a library for  script-like  Python programs  You can call programs like functions as in sh  Plumbum is useful if you want to run a pipeline without the shell   ls cmd   plumbum local  ls -l     get command ls cmd     run command      pexpect   pexpect lets you spawn child applications  control them and find patterns in their output  This is a better alternative to subprocess for commands that expect a tty on Unix   pexpect run  ls -l     Run command as normal child   pexpect spawn  scp foo user example com       Spawns child application child expect  Password      When this is the output child sendline  mypassword        fabric   fabric is a Python 2 5 and 2 7 library  It allows you to execute local and remote shell commands  Fabric is simple alternative for running commands in a secure shell  SSH   fabric operations local  ls -l     Run command as normal fabric operations local  ls -l   capture   True    Run command and receive output      envoy   envoy is known as  subprocess for humans   It is used as a convenience wrapper around the subprocess module   r   envoy run  ls -l     Run command r std out   get output      commands   commands contains wrapper functions for os popen  but it has been removed from Python 3 since subprocess is a better alternative   The edit was based on J F  Sebastian s comment

User · Answer

There is another difference here which is not mentioned previously   subprocess Popen executes the  lt command  as a subprocess  In my case  I need to execute file  lt a  which needs to communicate with another program   lt b     I tried subprocess  and execution was successful  However  lt b  could not communicate with  lt a   Everything is normal when I run both from the terminal   One more    NOTE  kwrite behaves different from other applications  If you try the below with Firefox  the results will not be the same    If you try os system  kwrite    program flow freezes until the user closes kwrite  To overcome that I tried instead os system konsole -e kwrite   This time program continued to flow  but kwrite became the subprocess of the console   Anyone runs the kwrite not being a subprocess  i e  in the system monitor it must appear at the leftmost edge of the tree

User · Answer

I would recommend the following method  run  and it will help us in getting STDOUT  STDERR and exit status as dictionary  The caller of this can read the dictionary return by  run  method to know the actual state of process      def run  cmd          print    DEBUG exec  0    format cmd         p   subprocess Popen cmd  stdout subprocess PIPE  stderr subprocess PIPE  universal newlines True  shell True          out  err    p communicate          ret          p wait          out          filter None  out split   n           err          filter None  err split   n           ret          True if ret    0 else False        return dict   output   out   error   err   status   ret      end

User · Answer

Invoke is a Python  2 7 and 3 4   task execution tool  amp  library  It provides a clean  high level API for running shell commands   gt  gt  gt  from invoke import run  gt  gt  gt  cmd    pip install -r requirements txt   gt  gt  gt  result   run cmd  hide True  warn True   gt  gt  gt  print result ok  True  gt  gt  gt  print result stdout splitlines   -1   Successfully installed invocations-0 13 0 pep8-1 5 7 spec-1 3 1

User · Answer

The subprocess module described above by Eli is very powerful  but the syntax to make a bog-standard system call and inspect its output  is unnecessarily prolix   The easiest way to make a system call is with the commands module  Linux only     gt  import commands  gt  commands getstatusoutput  grep matter alice-in-wonderland txt    0    Then it doesn t matter which way you go   said the Cat      The first item in the tuple is the return code of the process  The second item is its standard output  and standard error  merged      The Python devs have  deprecated  the commands module  but that doesn t mean you shouldn t use it  Only that they re not developing it anymore  which is okay  because it s already perfect  at its small but important function

User · Answer

I d recommend using the subprocess module instead of os system because it does shell escaping for you and is therefore much safer   subprocess call   ping    localhost

User · Answer

There is another difference here which is not mentioned previously   subprocess Popen executes the  lt command  as a subprocess  In my case  I need to execute file  lt a  which needs to communicate with another program   lt b     I tried subprocess  and execution was successful  However  lt b  could not communicate with  lt a   Everything is normal when I run both from the terminal   One more    NOTE  kwrite behaves different from other applications  If you try the below with Firefox  the results will not be the same    If you try os system  kwrite    program flow freezes until the user closes kwrite  To overcome that I tried instead os system konsole -e kwrite   This time program continued to flow  but kwrite became the subprocess of the console   Anyone runs the kwrite not being a subprocess  i e  in the system monitor it must appear at the leftmost edge of the tree

User · Answer

There are lots of different libraries which allow you to call external commands with Python  For each library I ve given a description and shown an example of calling an external command  The command I used as the example is ls -l  list all files   If you want to find out more about any of the libraries I ve listed and linked the documentation for each of them      Sources     subprocess  https   docs python org 3 5 library subprocess html shlex  https   docs python org 3 library shlex html os  https   docs python org 3 5 library os html sh  https   amoffat github io sh  plumbum  https   plumbum readthedocs io en latest  pexpect  https   pexpect readthedocs io en stable  fabric  http   www fabfile org  envoy  https   github com kennethreitz envoy commands  https   docs python org 2 library commands html                  These are all the libraries            Hopefully this will help you make a decision on which library to use        subprocess   Subprocess allows you to call external commands and connect them to their input output error pipes  stdin  stdout  and stderr   Subprocess is the default choice for running commands  but sometimes other modules are better   subprocess run   ls    -l      Run command subprocess run   ls    -l    stdout subprocess PIPE    This will run the command and return any output subprocess run shlex split  ls -l      You can also use the shlex library to split the command      os   os is used for  operating system dependent functionality   It can also be used to call external commands with os system and os popen  Note  There is also a subprocess popen   os will always run the shell and is a simple alternative for people who don t need to  or don t know how to use subprocess run   os system  ls -l     run command os popen  ls -l   read     This will run the command and return any output      sh   sh is a subprocess interface which lets you call programs as if they were functions  This is useful if you want to run a command multiple times   sh ls  -l     Run command normally ls cmd   sh Command  ls     Save command as a variable ls cmd     Run command as if it were a function      plumbum   plumbum is a library for  script-like  Python programs  You can call programs like functions as in sh  Plumbum is useful if you want to run a pipeline without the shell   ls cmd   plumbum local  ls -l     get command ls cmd     run command      pexpect   pexpect lets you spawn child applications  control them and find patterns in their output  This is a better alternative to subprocess for commands that expect a tty on Unix   pexpect run  ls -l     Run command as normal child   pexpect spawn  scp foo user example com       Spawns child application child expect  Password      When this is the output child sendline  mypassword        fabric   fabric is a Python 2 5 and 2 7 library  It allows you to execute local and remote shell commands  Fabric is simple alternative for running commands in a secure shell  SSH   fabric operations local  ls -l     Run command as normal fabric operations local  ls -l   capture   True    Run command and receive output      envoy   envoy is known as  subprocess for humans   It is used as a convenience wrapper around the subprocess module   r   envoy run  ls -l     Run command r std out   get output      commands   commands contains wrapper functions for os popen  but it has been removed from Python 3 since subprocess is a better alternative   The edit was based on J F  Sebastian s comment

User · Answer

This is how I run my commands  This code has everything you need pretty much  from subprocess import Popen  PIPE cmd    ls -l     p   Popen cmd   shell True  stdout PIPE  stderr PIPE  out  err   p communicate   print  Return code     p returncode print out rstrip    err rstrip

User · Answer

Use the subprocess module in the standard library  import subprocess subprocess run   quot ls quot    quot -l quot     The advantage of subprocess run over os system is that it is more flexible  you can get the stdout  stderr  the  quot real quot  status code  better error handling  etc      Even the documentation for os system recommends using subprocess instead   The subprocess module provides more powerful facilities for spawning new processes and retrieving their results  using that module is preferable to using this function  See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes   On Python 3 4 and earlier  use subprocess call instead of  run  subprocess call   quot ls quot    quot -l quot

User · Answer

If you need to call a shell command from a Python notebook  like Jupyter  Zeppelin  Databricks  or Google Cloud Datalab  you can just use the   prefix   For example    ls -ilF

User · Answer

Here is calling an external command and return or print the command s output   Python Subprocess check output is good for     Run command with arguments and return its output as a byte string    import subprocess proc   subprocess check output  ipconfig  all   print proc

User · Answer

For Python 3 5  it is recommended that you use the run function from the subprocess module  This returns a CompletedProcess object  from which you can easily obtain the output as well as return code   from subprocess import PIPE  run  command     echo    hello   result   run command  stdout PIPE  stderr PIPE  universal newlines True  print result returncode  result stdout  result stderr

User · Answer

As of Python 3 7 0 released on June 27th 2018  https   docs python org 3 whatsnew 3 7 html   you can achieve your desired result in the most powerful while equally simple way  This answer intends to show you the essential summary of various options in a short manner  For in-depth answers  please see the other ones   TL DR in 2021 The big advantage of os system      was its simplicity  subprocess is better and still easy to use  especially as of Python 3 5  import subprocess subprocess run  quot ls -a quot   shell True   Note  This is the exact answer to your question - running a command  like in a shell   Preferred Way If possible  remove the shell overhead and run the command directly  requires a list   import subprocess subprocess run   quot help quot    subprocess run   quot ls quot    quot -a quot     Pass program arguments in a list  Don t include   quot -escaping for arguments containing spaces   Advanced Use Cases Checking The Output The following code speaks for itself  import subprocess result   subprocess run   quot ls quot    quot -a quot    capture output True  text True  if  quot stackoverflow-logo png quot  in result stdout      print  quot You re a fan  quot   else      print  quot You re not a fan  quot    result stdout is all normal program output excluding errors  Read result stderr to get them  capture output True - turns capturing on  Otherwise result stderr and result stdout would be None  Available from Python 3 7  text True - a convenience argument added in Python 3 7 which converts the received binary data to Python strings you can easily work with  Checking the returncode Do if result returncode    127  print  quot The program failed for some weird reason quot   elif result returncode    0  print  quot The program succeeded quot   else  print  quot The program failed unexpectedly quot    If you just want to check if the program succeeded  returncode    0  and otherwise throw an Exception  there is a more convenient function  result check returncode    But it s Python  so there s an even more convenient argument check which does the same thing automatically for you  result   subprocess run      check True   stderr should be inside stdout You might want to have all program output inside stdout  even errors  To accomplish this  run result   subprocess run      stderr subprocess STDOUT   result stderr will then be None and result stdout will contain everything  using shell False with an argument string shell False expects a list of arguments  You might however  split an argument string on your own using shlex  import subprocess import shlex subprocess run shlex split  quot ls -a quot     That s it  Common Problems Chances are high you just started using Python when you come across this question  Let s look at some common problems   FileNotFoundError   Errno 2  No such file or directory   ls -a    ls -a   You re running a subprocess without shell True   Either use a list    quot ls quot    quot -a quot    or set shell True   TypeError        NoneType        Check that you ve set capture output True   TypeError  a bytes-like object is required  not        You always receive byte results from your program  If you want to work with it like a normal string  set text True   subprocess CalledProcessError  Command         returned non-zero exit status 1   Your command didn t run successfully  You could disable returncode checking or check your actual program s validity   TypeError  init   got an unexpected keyword argument        You re likely using a version of Python older than 3 7 0  update it to the most recent one available  Otherwise there are other answers in this StackOverflow post showing you older alternative solutions

User · Answer

I d recommend using the subprocess module instead of os system because it does shell escaping for you and is therefore much safer   subprocess call   ping    localhost

User · Answer

If you need the output from the command you are calling  then you can use subprocess check output  Python 2 7      gt  gt  gt  subprocess check output   ls    -l     dev null     crw-rw-rw- 1 root root 1  3 Oct 18  2007  dev null n    Also note the shell parameter      If shell is True  the specified command will be executed through the shell  This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes  filename wildcards  environment variable expansion  and expansion of   to a user   s home directory  However  note that Python itself offers implementations of many shell-like features  in particular  glob  fnmatch  os walk    os path expandvars    os path expanduser    and shutil

User · Answer

Shameless plug  I wrote a library for this  P https   github com houqp shell py  It s basically a wrapper for popen and shlex for now  It also supports piping commands so you can chain commands easier in Python  So you can do things like   ex  echo hello shell py      awk   print  2

User · Answer

With the standard library  Use the subprocess module  Python 3    import subprocess subprocess run   ls    -l      It is the recommended standard way  However  more complicated tasks  pipes  output  input  etc   can be tedious to construct and write   Note on Python version  If you are still using Python 2  subprocess call works in a similar way   ProTip  shlex split can help you to parse the command for run  call  and other subprocess functions in case you don t want  or you can t   provide them in form of lists   import shlex import subprocess subprocess run shlex split  ls -l      With external dependencies  If you do not mind external dependencies  use plumbum   from plumbum cmd import ifconfig print ifconfig  wlan0        It is the best subprocess wrapper  It s cross-platform  i e  it works on both Windows and Unix-like systems  Install by pip install plumbum   Another popular library is sh   from sh import ifconfig print ifconfig  wlan0      However  sh dropped Windows support  so it s not as awesome as it used to be  Install by pip install sh

User · Answer

Here s a summary of the ways to call external programs and the advantages and disadvantages of each    os system  some command with args   passes the command and arguments to your system s shell   This is nice because you can actually run multiple commands at once in this manner and set up pipes and input output redirection   For example     os system  some command  lt  input file   another command  gt  output file        However  while this is convenient  you have to manually handle the escaping of shell characters such as spaces  etc   On the other hand  this also lets you run commands which are simply shell commands and not actually external programs   See the documentation    stream   os popen  some command with args   will do the same thing as os system except that it gives you a file-like object that you can use to access standard input output for that process   There are 3 other variants of popen that all handle the i o slightly differently   If you pass everything as a string  then your command is passed to the shell  if you pass them as a list then you don t need to worry about escaping anything   See the documentation  The Popen class of the subprocess module   This is intended as a replacement for os popen but has the downside of being slightly more complicated by virtue of being so comprehensive   For example  you d say   print subprocess Popen  echo Hello World   shell True  stdout subprocess PIPE  stdout read     instead of    print os popen  echo Hello World   read     but it is nice to have all of the options there in one unified class instead of 4 different popen functions   See the documentation  The call function from the subprocess module   This is basically just like the Popen class and takes all of the same arguments  but it simply waits until the command completes and gives you the return code   For example   return code   subprocess call  echo Hello World   shell True      See the documentation  If you re on Python 3 5 or later  you can use the new subprocess run function  which is a lot like the above but even more flexible and returns a CompletedProcess object when the command finishes executing  The os module also has all of the fork exec spawn functions that you d have in a C program  but I don t recommend using them directly    The subprocess module should probably be what you use   Finally please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it  There are serious security implications if any part of the string that you pass can not be fully trusted  For example  if a user is entering some any part of the string  If you are unsure  only use these methods with constants  To give you a hint of the implications consider this code   print subprocess Popen  echo  s     user input  stdout PIPE  stdout read     and imagine that the user enters something  my mama didnt love me  amp  amp  rm -rf    which could erase the whole filesystem

User · Answer

Shameless plug  I wrote a library for this  P https   github com houqp shell py  It s basically a wrapper for popen and shlex for now  It also supports piping commands so you can chain commands easier in Python  So you can do things like   ex  echo hello shell py      awk   print  2

User · Answer

There are a number of ways of Calling an external Command from Python  There are some Functions and Modules with the Good Helper Functions that can make it really easy  But the Recommended among all is the Subprocess Module   import subprocess as s s call   command exe            Call function will start the external process  pass some command line arguments and wait for it to finish  When it finishes you continue executing  Arguments in Call function are passed through the list  The first argument in the list is the command typically in the form of an executable file and subsequent arguments in the list whatever you want to pass   If you have called processes from the command line in the windows before you ll be aware that you often need to quote arguments  You need to put quotations mark around it  if there s a space then there s a backslash and there are some complicated rules but you can avoid a whole lot of that in python by using subprocess module because it is a list and each item is known to be a distinct and python can get quoting correctly for you   In the end  after the list  there are a number of optional parameters one of these is a shell and if you set shell equals to true then your command is going to be run as if you have typed in at the command prompt   s call   command exe          shell True    This gives you access to functionality like piping  you can redirect to files  you can call multiple commands in one thing   One more thing  if your script relies on the process succeeding then you want to check the result and the result can be checked with the check call helper function   s check call        It is exactly the same as a call function  it takes the same arguments  takes the same list  you can pass in any of the extra arguments but it going to wait for the functions to complete  And if the exit code of the function is anything other then zero  it will through an exception in the python script   Finally  if you want tighter control Popen constructor which is also from the subprocess module  It also takes the same arguments as incall  amp  check call function but it returns an object representing the running process   p s Popen          It does not wait for the running process to finish also it s not going to throw any exception immediately but it gives you an object that will let you do things like wait for it to finish  let you communicate to it  you can redirect standard input  standard output if you want to display output somewhere else and a lot more

User · Answer

As some of the answers were related to the previous versions of Python or were using os system module  I post this answer for people like me who intend to use subprocess in Python nbsp 3 5   The following did the trick for me on Linux   import subprocess    subprocess run   returns a completed process object that can be inspected c   subprocess run   ls    -ltrh    stdout subprocess PIPE  stderr subprocess PIPE  print c stdout decode  utf-8      As mentioned in the documentation  PIPE values are byte sequences and for properly showing them decoding should be considered  For later versions of Python  text True and encoding  utf-8  are added to kwargs of subprocess run     The output of the abovementioned code is   total 113M -rwxr-xr-x  1 farzad farzad  307 Jan 15  2018 vpnscript -rwxrwxr-x  1 farzad farzad  204 Jan 15  2018 ex drwxrwxr-x  4 farzad farzad 4 0K Jan 22  2018 scripts        Some other lines

User · Answer

The subprocess module described above by Eli is very powerful  but the syntax to make a bog-standard system call and inspect its output  is unnecessarily prolix   The easiest way to make a system call is with the commands module  Linux only     gt  import commands  gt  commands getstatusoutput  grep matter alice-in-wonderland txt    0    Then it doesn t matter which way you go   said the Cat      The first item in the tuple is the return code of the process  The second item is its standard output  and standard error  merged      The Python devs have  deprecated  the commands module  but that doesn t mean you shouldn t use it  Only that they re not developing it anymore  which is okay  because it s already perfect  at its small but important function

User · Answer

Use subprocess      or for a very simple command   import os os system  cat testfile

User · Answer

A simple way is to use the os module   import os os system  ls     Alternatively  you can also use the subprocess module   import subprocess subprocess check call  ls     If you want the result to be stored in a variable try   import subprocess r   subprocess check output  ls

User · Answer

Use   import os  cmd    ls -al   os system cmd    os - This module provides a portable way of using operating system-dependent functionality   For the more os functions  here is the documentation

User · Answer

Very simplest way to run any command and get the result back   from commands import getstatusoutput  try      return getstatusoutput  ls -ltr   except Exception  e      return None

User · Answer

To fetch the network id from the OpenStack Neutron      usr bin python import os netid    nova net-list   awk    External     print  2     temp   os popen netid  read       Here temp also contains new line   n     networkId   temp rstrip   print networkId    Output of nova net-list   -------------------------------------- ------------ ------    ID                                     Label        CIDR    -------------------------------------- ------------ ------    431c9014-5b5d-4b51-a357-66020ffbb123   test1        None     27a74fcd-37c0-4789-9414-9531b7e3f126   External     None     5a2712e9-70dc-4b0e-9281-17e02f4684c9   management   None     7aa697f5-0e60-4c15-b4cc-9cb659698512   Internal     None    -------------------------------------- ------------ ------    Output of print networkId   27a74fcd-37c0-4789-9414-9531b7e3f126

User · Answer

I would recommend the following method  run  and it will help us in getting STDOUT  STDERR and exit status as dictionary  The caller of this can read the dictionary return by  run  method to know the actual state of process      def run  cmd          print    DEBUG exec  0    format cmd         p   subprocess Popen cmd  stdout subprocess PIPE  stderr subprocess PIPE  universal newlines True  shell True          out  err    p communicate          ret          p wait          out          filter None  out split   n           err          filter None  err split   n           ret          True if ret    0 else False        return dict   output   out   error   err   status   ret      end

User · Answer

I d recommend using the subprocess module instead of os system because it does shell escaping for you and is therefore much safer   subprocess call   ping    localhost

User · Answer

Check the  pexpect  Python library  too   It allows for interactive controlling of external programs commands  even ssh  ftp  telnet  etc  You can just type something like   child   pexpect spawn  ftp 192 168 0 24    child expect    i name         child sendline  anonymous    child expect    i password

User · Answer

A simple way is to use the os module   import os os system  ls     Alternatively  you can also use the subprocess module   import subprocess subprocess check call  ls     If you want the result to be stored in a variable try   import subprocess r   subprocess check output  ls

User · Answer

import os os system  your command     Note that this is dangerous  since the command isn t cleaned  I leave it up to you to google for the relevant documentation on the  os  and  sys  modules  There are a bunch of functions  exec  and spawn   that will do similar things

User · Answer

I always use fabric for this things like   from fabric operations import local result   local  ls   capture True  print  Content  n s     result      But this seem to be a good tool  sh  Python subprocess interface    Look at an example   from sh import vgdisplay print vgdisplay   print vgdisplay  -v   print vgdisplay v True

User · Answer

For Python 3 5  it is recommended that you use the run function from the subprocess module  This returns a CompletedProcess object  from which you can easily obtain the output as well as return code   from subprocess import PIPE  run  command     echo    hello   result   run command  stdout PIPE  stderr PIPE  universal newlines True  print result returncode  result stdout  result stderr

User · Answer

If you are not using user input in the commands  you can use this   from os import getcwd from subprocess import check output from shlex import quote  def sh command       return check output quote command   shell True  cwd getcwd    universal newlines True  strip     And use it as  branch   sh  git rev-parse --abbrev-ref HEAD     shell True will spawn a shell  so you can use pipe and such shell things sh  ps aux   grep python    This is very very handy for running hardcoded commands and processing its output  The universal lines True make sure the output is returned in a string instead of binary   cwd getcwd   will make sure that the command is run with the same working directory as the interpreter  This is handy for Git commands to work like the Git branch name example above   Some recipes   free memory in megabytes  sh  free -m   split   n   1  split   1  free space on   in percent sh  df -m     split   n   1  split   4  0 -1  CPU load sum map float  sh  ps -ef -o pcpu   split   n   1      But this isn t safe for user input  from the documentation      Security Considerations      Unlike some other popen functions  this implementation will never   implicitly call a system shell  This means that all characters    including shell metacharacters  can safely be passed to child   processes  If the shell is invoked explicitly  via shell True  it is   the application   s responsibility to ensure that all whitespace and   metacharacters are quoted appropriately to avoid shell injection   vulnerabilities       When using shell True  the shlex quote   function can be used to   properly escape whitespace and shell metacharacters in strings that   are going to be used to construct shell commands    Even using the shlex quote    it is good to keep a little paranoid when using user inputs on shell commands  One option is using a hardcoded command to take some generic output and filtering by user input  Anyway using shell False will make sure that only the exactly process that you want to execute will be executed or you get a No such file or directory error   Also there is some performance impact on shell True  from my tests it seems about 20  slower than shell False  the default    In  50   timeit  check output  ls -l  split    universal newlines True    number 1000  globals globals    Out 50   2 6801227919995654  In  51   timeit  check output  ls -l   universal newlines True  shell True    number 1000  globals globals    Out 51   3 243950183999914

User · Answer

import os cmd    ls -al  os system cmd    If you want to return the results of the command  you can use os popen  However  this is deprecated since version 2 6 in favor of the subprocess module  which other answers have covered well

User · Answer

You can use Popen  and then you can check the procedure s status   from subprocess import Popen  proc   Popen   ls    -l    if proc poll   is None      proc kill     Check out subprocess Popen

User · Answer

Typical implementation   import subprocess  p   subprocess Popen  ls   shell True  stdout subprocess PIPE  stderr subprocess STDOUT  for line in p stdout readlines        print line  retval   p wait     You are free to do what you want with the stdout data in the pipe   In fact  you can simply omit those parameters  stdout  and stderr   and it ll behave like os system

User · Answer

I wrote a small library to help with this use case   https   pypi org project citizenshell   It can be installed using   pip install citizenshell   And then used as follows   from citizenshell import sh assert sh  echo Hello World       Hello World    You can separate stdout from stderr and extract the exit code as follows   result   sh   gt  amp 2 echo error  amp  amp  echo output  amp  amp  exit 13   assert result stdout        output   assert result stderr        error   assert result exit code      13   And the cool thing is that you don t have to wait for the underlying shell to exit before starting processing the output   for line in sh  for i in 1 2 3 4  do echo -n  It is    date   H  M  S  sleep 1  done   wait False      print   gt  gt  gt    line         will print the lines as they are available thanks to the wait False   gt  gt  gt  It is 14 24 52   gt  gt  gt  It is 14 24 53   gt  gt  gt  It is 14 24 54   gt  gt  gt  It is 14 24 55    More examples can be found at https   github com meuter citizenshell

User · Answer

Under Linux  in case you would like to call an external command that will execute independently  will keep running after the python script terminates   you can use a simple queue as task spooler or the at command  An example with task spooler   import os os system  ts  lt your-command gt      Notes about task spooler  ts      You could set the number of concurrent processes to be run   slots   with   ts -S  lt number-of-slots gt  Installing ts doesn t requires admin privileges  You can download and compile it from source with a simple make  add it to your path and you re done

User · Answer

There is also Plumbum   gt  gt  gt  from plumbum import local  gt  gt  gt  ls   local  ls    gt  gt  gt  ls LocalCommand  lt LocalPath  bin ls gt    gt  gt  gt  ls   u build py ndist ndocs nLICENSE nplumbum nREADME rst nsetup py ntests ntodo txt n   gt  gt  gt  notepad   local  c   windows  notepad exe    gt  gt  gt  notepad                                       Notepad window pops up u                                                 Notepad window is closed by user  command returns

User · Answer

os popen   is the easiest and the most safest way to execute a command  You can execute any command that you run on the command line  In addition you will also be able to capture the output of the command using os popen   read    You can do it like this   import os output   os popen  Your Command Here   read   print  output    An example where you list all the files in the current directory   import os output   os popen  ls   read   print  output    Outputs list of files in the directory

User · Answer

Use the subprocess module in the standard library  import subprocess subprocess run   quot ls quot    quot -l quot     The advantage of subprocess run over os system is that it is more flexible  you can get the stdout  stderr  the  quot real quot  status code  better error handling  etc      Even the documentation for os system recommends using subprocess instead   The subprocess module provides more powerful facilities for spawning new processes and retrieving their results  using that module is preferable to using this function  See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes   On Python 3 4 and earlier  use subprocess call instead of  run  subprocess call   quot ls quot    quot -l quot

User · Answer

Some hints on detaching the child process from the calling one  starting the child process in background    Suppose you want to start a long task from a CGI script  That is  the child process should live longer than the CGI script execution process   The classical example from the subprocess module documentation is   import subprocess import sys    Some code here  pid   subprocess Popen  sys executable   longtask py      Call subprocess    Some more code here   The idea here is that you do not want to wait in the line  call subprocess  until the longtask py is finished  But it is not clear what happens after the line  some more code here  from the example   My target platform was FreeBSD  but the development was on Windows  so I faced the problem on Windows first   On Windows  Windows nbsp XP   the parent process will not finish until the longtask py has finished its work  It is not what you want in a CGI script  The problem is not specific to Python  in the PHP community the problems are the same   The solution is to pass DETACHED PROCESS Process Creation Flag to the underlying CreateProcess function in Windows API  If you happen to have installed pywin32  you can import the flag from the win32process module  otherwise you should define it yourself   DETACHED PROCESS   0x00000008  pid   subprocess Popen  sys executable   longtask py                           creationflags DETACHED PROCESS  pid      UPD 2015 10 27  eryksun in a comment below notes  that the semantically correct flag is CREATE NEW CONSOLE  0x00000010      On FreeBSD we have another problem  when the parent process is finished  it finishes the child processes as well  And that is not what you want in a CGI script either  Some experiments showed that the problem seemed to be in sharing sys stdout  And the working solution was the following   pid   subprocess Popen  sys executable   longtask py    stdout subprocess PIPE  stderr subprocess PIPE  stdin subprocess PIPE    I have not checked the code on other platforms and do not know the reasons of the behaviour on FreeBSD  If anyone knows  please share your ideas  Googling on starting background processes in Python does not shed any light yet

User · Answer

Typical implementation   import subprocess  p   subprocess Popen  ls   shell True  stdout subprocess PIPE  stderr subprocess STDOUT  for line in p stdout readlines        print line  retval   p wait     You are free to do what you want with the stdout data in the pipe   In fact  you can simply omit those parameters  stdout  and stderr   and it ll behave like os system

User · Answer

i use this for 3 6   import subprocess def execute cmd                   Purpose    To execute a command and return exit status         Argument   cmd - command to execute         Return     result  exit code             process   subprocess Popen cmd  shell True  stdout subprocess PIPE  stderr subprocess PIPE       result  error    process communicate       rc   process wait       if rc    0          print   Error  failed to execute command     cmd          print  error rstrip   decode  utf-8        return result rstrip   decode  utf-8    serror rstrip   decode  utf-8     def

User · Answer

Use subprocess call     from subprocess import call    Using list call   echo    Hello    world       Single string argument varies across platforms so better split it call  echo Hello world  split

User · Answer

Calling an external command in Python   Simple  use subprocess run  which returns a CompletedProcess object    gt  gt  gt  import subprocess  gt  gt  gt  completed process   subprocess run  python --version   Python 3 6 1    Anaconda 4 4 0  64-bit   gt  gt  gt  completed process CompletedProcess args  python --version   returncode 0    Why   As of Python 3 5  the documentation recommends subprocess run      The recommended approach to invoking subprocesses is to use the run   function for all use cases it can handle  For more advanced use cases  the underlying Popen interface can be used directly    Here s an example of the simplest possible usage - and it does exactly as asked    gt  gt  gt  import subprocess  gt  gt  gt  completed process   subprocess run  python --version   Python 3 6 1    Anaconda 4 4 0  64-bit   gt  gt  gt  completed process CompletedProcess args  python --version   returncode 0    run waits for the command to successfully finish  then returns a CompletedProcess object  It may instead raise TimeoutExpired  if you give it a timeout  argument  or CalledProcessError  if it fails and you pass check True    As you might infer from the above example  stdout and stderr both get piped to your own stdout and stderr by default   We can inspect the returned object and see the command that was given and the returncode    gt  gt  gt  completed process args  python --version   gt  gt  gt  completed process returncode 0   Capturing output  If you want to capture the output  you can pass subprocess PIPE to the appropriate stderr or stdout    gt  gt  gt  cp   subprocess run  python --version                            stderr subprocess PIPE                           stdout subprocess PIPE   gt  gt  gt  cp stderr b Python 3 6 1    Anaconda 4 4 0  64-bit  r n   gt  gt  gt  cp stdout b      I find it interesting and slightly counterintuitive that the version info gets put to stderr instead of stdout    Pass a command list  One might easily move from manually providing a command string  like the question suggests  to providing a string built programmatically  Don t build strings programmatically  This is a potential security issue  It s better to assume you don t trust the input     gt  gt  gt  import textwrap  gt  gt  gt  args     python   textwrap   file     gt  gt  gt  cp   subprocess run args  stdout subprocess PIPE   gt  gt  gt  cp stdout b Hello there  r n  This is indented  r n    Note  only args should be passed positionally   Full Signature  Here s the actual signature in the source and as shown by help run     def run  popenargs  input None  timeout None  check False    kwargs      The popenargs and kwargs are given to the Popen constructor  input can be a string of bytes  or unicode  if specify encoding or universal newlines True  that will be piped to the subprocess s stdin   The documentation describes timeout  and check True better than I could      The timeout argument is passed to Popen communicate    If the timeout   expires  the child process will be killed and waited for  The   TimeoutExpired exception will be re-raised after the child process has   terminated       If check is true  and the process exits with a non-zero exit code  a   CalledProcessError exception will be raised  Attributes of that   exception hold the arguments  the exit code  and stdout and stderr if   they were captured    and this example for check True is better than one I could come up with     gt  gt  gt  subprocess run  exit 1   shell True  check True  Traceback  most recent call last         subprocess CalledProcessError  Command  exit 1  returned non-zero exit status 1    Expanded Signature  Here s an expanded signature  as given in the documentation    subprocess run args     stdin None  input None  stdout None  stderr None   shell False  cwd None  timeout None  check False  encoding None   errors None     Note that this indicates that only the args list should be passed positionally  So pass the remaining arguments as keyword arguments   Popen  When use Popen instead  I would struggle to find use-case based on the arguments alone  Direct usage of Popen would  however  give you access to its methods  including poll   send signal    terminate   and  wait    Here s the Popen signature as given in the source  I think this is the most precise encapsulation of the information  as opposed to help Popen     def   init   self  args  bufsize -1  executable None               stdin None  stdout None  stderr None               preexec fn None  close fds  PLATFORM DEFAULT CLOSE FDS               shell False  cwd None  env None  universal newlines False               startupinfo None  creationflags 0               restore signals True  start new session False               pass fds        encoding None  errors None     But more informative is the Popen documentation    subprocess Popen args  bufsize -1  executable None  stdin None                   stdout None  stderr None  preexec fn None  close fds True                   shell False  cwd None  env None  universal newlines False                   startupinfo None  creationflags 0  restore signals True                   start new session False  pass fds        encoding None  errors None        Execute a child program in a new process  On POSIX  the class uses   os execvp  -like behavior to execute the child program  On Windows    the class uses the Windows CreateProcess   function  The arguments to   Popen are as follows    Understanding the remaining documentation on Popen will be left as an exercise for the reader

User · Answer

As an example  in Linux    import subprocess subprocess run  mkdir test dir   shell True    This creates test dir in the current directory  Note that this also works   import subprocess subprocess call  mkdir test dir   shell True    The equivalent code using os system is   import os os system  mkdir test dir     Best practice would be to use subprocess instead of os  with  run favored over  call   All you need to know about subprocess is here  Also  note that all Python documentation is available for download from here  I downloaded the PDF packed as  zip  I mention this because there s a nice overview of the os module in tutorial pdf  page 81   Besides  it s an authoritative resource for Python coders

User · Answer

Under Linux  in case you would like to call an external command that will execute independently  will keep running after the python script terminates   you can use a simple queue as task spooler or the at command  An example with task spooler   import os os system  ts  lt your-command gt      Notes about task spooler  ts      You could set the number of concurrent processes to be run   slots   with   ts -S  lt number-of-slots gt  Installing ts doesn t requires admin privileges  You can download and compile it from source with a simple make  add it to your path and you re done

User · Answer

Use subprocess      or for a very simple command   import os os system  cat testfile

User · Answer

import subprocess  p   subprocess run   ls    -ltr    capture output True  print p stdout decode    p stderr decode      Try online

User · Answer

I tend to use subprocess together with shlex  to handle escaping of quoted strings     gt  gt  gt  import subprocess  shlex  gt  gt  gt  command    ls -l   your path with spaces     gt  gt  gt  call params   shlex split command   gt  gt  gt  print call params   ls    -l     your path with spaces     gt  gt  gt  subprocess call call params

User · Answer

There are a number of ways of Calling an external Command from Python  There are some Functions and Modules with the Good Helper Functions that can make it really easy  But the Recommended among all is the Subprocess Module   import subprocess as s s call   command exe            Call function will start the external process  pass some command line arguments and wait for it to finish  When it finishes you continue executing  Arguments in Call function are passed through the list  The first argument in the list is the command typically in the form of an executable file and subsequent arguments in the list whatever you want to pass   If you have called processes from the command line in the windows before you ll be aware that you often need to quote arguments  You need to put quotations mark around it  if there s a space then there s a backslash and there are some complicated rules but you can avoid a whole lot of that in python by using subprocess module because it is a list and each item is known to be a distinct and python can get quoting correctly for you   In the end  after the list  there are a number of optional parameters one of these is a shell and if you set shell equals to true then your command is going to be run as if you have typed in at the command prompt   s call   command exe          shell True    This gives you access to functionality like piping  you can redirect to files  you can call multiple commands in one thing   One more thing  if your script relies on the process succeeding then you want to check the result and the result can be checked with the check call helper function   s check call        It is exactly the same as a call function  it takes the same arguments  takes the same list  you can pass in any of the extra arguments but it going to wait for the functions to complete  And if the exit code of the function is anything other then zero  it will through an exception in the python script   Finally  if you want tighter control Popen constructor which is also from the subprocess module  It also takes the same arguments as incall  amp  check call function but it returns an object representing the running process   p s Popen          It does not wait for the running process to finish also it s not going to throw any exception immediately but it gives you an object that will let you do things like wait for it to finish  let you communicate to it  you can redirect standard input  standard output if you want to display output somewhere else and a lot more

User · Answer

If you need to call a shell command from a Python notebook  like Jupyter  Zeppelin  Databricks  or Google Cloud Datalab  you can just use the   prefix   For example    ls -ilF

User · Answer

Sultan is a recent-ish package meant for this purpose  Provides some niceties around managing user privileges and adding helpful error messages   from sultan api import Sultan  with Sultan load sudo True  hostname  myserver com   as sultan    sultan yum  install -y tree   run

User · Answer

Using the Popen function of the subprocess Python module is the simplest way of running Linux commands  In that  the Popen communicate   function will give your commands output  For example  import subprocess     process   subprocess Popen         Pass command and arguments to the function stdout  stderr   process communicate       Get command output and error

User · Answer

Very simplest way to run any command and get the result back   from commands import getstatusoutput  try      return getstatusoutput  ls -ltr   except Exception  e      return None

User · Answer

Calling an external command in Python   Simple  use subprocess run  which returns a CompletedProcess object    gt  gt  gt  import subprocess  gt  gt  gt  completed process   subprocess run  python --version   Python 3 6 1    Anaconda 4 4 0  64-bit   gt  gt  gt  completed process CompletedProcess args  python --version   returncode 0    Why   As of Python 3 5  the documentation recommends subprocess run      The recommended approach to invoking subprocesses is to use the run   function for all use cases it can handle  For more advanced use cases  the underlying Popen interface can be used directly    Here s an example of the simplest possible usage - and it does exactly as asked    gt  gt  gt  import subprocess  gt  gt  gt  completed process   subprocess run  python --version   Python 3 6 1    Anaconda 4 4 0  64-bit   gt  gt  gt  completed process CompletedProcess args  python --version   returncode 0    run waits for the command to successfully finish  then returns a CompletedProcess object  It may instead raise TimeoutExpired  if you give it a timeout  argument  or CalledProcessError  if it fails and you pass check True    As you might infer from the above example  stdout and stderr both get piped to your own stdout and stderr by default   We can inspect the returned object and see the command that was given and the returncode    gt  gt  gt  completed process args  python --version   gt  gt  gt  completed process returncode 0   Capturing output  If you want to capture the output  you can pass subprocess PIPE to the appropriate stderr or stdout    gt  gt  gt  cp   subprocess run  python --version                            stderr subprocess PIPE                           stdout subprocess PIPE   gt  gt  gt  cp stderr b Python 3 6 1    Anaconda 4 4 0  64-bit  r n   gt  gt  gt  cp stdout b      I find it interesting and slightly counterintuitive that the version info gets put to stderr instead of stdout    Pass a command list  One might easily move from manually providing a command string  like the question suggests  to providing a string built programmatically  Don t build strings programmatically  This is a potential security issue  It s better to assume you don t trust the input     gt  gt  gt  import textwrap  gt  gt  gt  args     python   textwrap   file     gt  gt  gt  cp   subprocess run args  stdout subprocess PIPE   gt  gt  gt  cp stdout b Hello there  r n  This is indented  r n    Note  only args should be passed positionally   Full Signature  Here s the actual signature in the source and as shown by help run     def run  popenargs  input None  timeout None  check False    kwargs      The popenargs and kwargs are given to the Popen constructor  input can be a string of bytes  or unicode  if specify encoding or universal newlines True  that will be piped to the subprocess s stdin   The documentation describes timeout  and check True better than I could      The timeout argument is passed to Popen communicate    If the timeout   expires  the child process will be killed and waited for  The   TimeoutExpired exception will be re-raised after the child process has   terminated       If check is true  and the process exits with a non-zero exit code  a   CalledProcessError exception will be raised  Attributes of that   exception hold the arguments  the exit code  and stdout and stderr if   they were captured    and this example for check True is better than one I could come up with     gt  gt  gt  subprocess run  exit 1   shell True  check True  Traceback  most recent call last         subprocess CalledProcessError  Command  exit 1  returned non-zero exit status 1    Expanded Signature  Here s an expanded signature  as given in the documentation    subprocess run args     stdin None  input None  stdout None  stderr None   shell False  cwd None  timeout None  check False  encoding None   errors None     Note that this indicates that only the args list should be passed positionally  So pass the remaining arguments as keyword arguments   Popen  When use Popen instead  I would struggle to find use-case based on the arguments alone  Direct usage of Popen would  however  give you access to its methods  including poll   send signal    terminate   and  wait    Here s the Popen signature as given in the source  I think this is the most precise encapsulation of the information  as opposed to help Popen     def   init   self  args  bufsize -1  executable None               stdin None  stdout None  stderr None               preexec fn None  close fds  PLATFORM DEFAULT CLOSE FDS               shell False  cwd None  env None  universal newlines False               startupinfo None  creationflags 0               restore signals True  start new session False               pass fds        encoding None  errors None     But more informative is the Popen documentation    subprocess Popen args  bufsize -1  executable None  stdin None                   stdout None  stderr None  preexec fn None  close fds True                   shell False  cwd None  env None  universal newlines False                   startupinfo None  creationflags 0  restore signals True                   start new session False  pass fds        encoding None  errors None        Execute a child program in a new process  On POSIX  the class uses   os execvp  -like behavior to execute the child program  On Windows    the class uses the Windows CreateProcess   function  The arguments to   Popen are as follows    Understanding the remaining documentation on Popen will be left as an exercise for the reader

User · Answer

Typical implementation   import subprocess  p   subprocess Popen  ls   shell True  stdout subprocess PIPE  stderr subprocess STDOUT  for line in p stdout readlines        print line  retval   p wait     You are free to do what you want with the stdout data in the pipe   In fact  you can simply omit those parameters  stdout  and stderr   and it ll behave like os system

User · Answer

There are many ways to call a command    For example    if and exe needs two parameters  In cmd we can call sample exe use this     and exe 2 3 and it show 5 on screen   If we use a Python script to call and exe  we should do like     os system cmd        os system   and exe           2           3     os popen cmd        os popen   and exe           2           3     subprocess Popen cmd        subprocess Popen   and exe           2           3       It s too hard  so we can join cmd with a space   import os cmd       join exename parameters  os popen cmd

User · Answer

Update   subprocess run is the recommended approach as of Python 3 5 if your code does not need to maintain compatibility with earlier Python versions  It s more consistent and offers similar ease-of-use as Envoy   Piping isn t as straightforward though  See this question for how    Here s some examples from the documentation   Run a process    gt  gt  gt  subprocess run   ls    -l       Doesn t capture output CompletedProcess args   ls    -l    returncode 0    Raise on failed run    gt  gt  gt  subprocess run  exit 1   shell True  check True  Traceback  most recent call last         subprocess CalledProcessError  Command  exit 1  returned non-zero exit status 1   Capture output    gt  gt  gt  subprocess run   ls    -l     dev null    stdout subprocess PIPE  CompletedProcess args   ls    -l     dev null    returncode 0  stdout b crw-rw-rw- 1 root root 1  3 Jan 23 16 23  dev null n     Original answer   I recommend trying Envoy  It s a wrapper for subprocess  which in turn aims to replace the older modules and functions  Envoy is subprocess for humans   Example usage from the README    gt  gt  gt  r   envoy run  git config   data  data to pipe in   timeout 2    gt  gt  gt  r status code 129  gt  gt  gt  r std out  usage  git config  options    gt  gt  gt  r std err      Pipe stuff around too    gt  gt  gt  r   envoy run  uptime   pbcopy     gt  gt  gt  r command  pbcopy   gt  gt  gt  r status code 0   gt  gt  gt  r history   lt Response  uptime  gt

User · Answer

If you need the output from the command you are calling  then you can use subprocess check output  Python 2 7      gt  gt  gt  subprocess check output   ls    -l     dev null     crw-rw-rw- 1 root root 1  3 Oct 18  2007  dev null n    Also note the shell parameter      If shell is True  the specified command will be executed through the shell  This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes  filename wildcards  environment variable expansion  and expansion of   to a user   s home directory  However  note that Python itself offers implementations of many shell-like features  in particular  glob  fnmatch  os walk    os path expandvars    os path expanduser    and shutil

User · Answer

Check the  pexpect  Python library  too   It allows for interactive controlling of external programs commands  even ssh  ftp  telnet  etc  You can just type something like   child   pexpect spawn  ftp 192 168 0 24    child expect    i name         child sendline  anonymous    child expect    i password

User · Answer

There is also Plumbum   gt  gt  gt  from plumbum import local  gt  gt  gt  ls   local  ls    gt  gt  gt  ls LocalCommand  lt LocalPath  bin ls gt    gt  gt  gt  ls   u build py ndist ndocs nLICENSE nplumbum nREADME rst nsetup py ntests ntodo txt n   gt  gt  gt  notepad   local  c   windows  notepad exe    gt  gt  gt  notepad                                       Notepad window pops up u                                                 Notepad window is closed by user  command returns

User · Answer

import os cmd    ls -al  os system cmd    If you want to return the results of the command  you can use os popen  However  this is deprecated since version 2 6 in favor of the subprocess module  which other answers have covered well

User · Answer

After some research  I have the following code which works very well for me  It basically prints both stdout and stderr in real time   stdout result   1 stderr result   1   def stdout thread pipe       global stdout result     while True          out   pipe stdout read 1          stdout result   pipe poll           if out       and stdout result is not None              break          if out                    sys stdout write out              sys stdout flush     def stderr thread pipe       global stderr result     while True          err   pipe stderr read 1          stderr result   pipe poll           if err       and stderr result is not None              break          if err                    sys stdout write err              sys stdout flush     def exec command command  cwd None       if cwd is not None          print           join command       in     cwd     else          print           join command             p   subprocess Popen          command  stdout subprocess PIPE  stderr subprocess PIPE  cwd cwd            out thread   threading Thread name  stdout thread   target stdout thread  args  p        err thread   threading Thread name  stderr thread   target stderr thread  args  p         err thread start       out thread start        out thread join       err thread join        return stdout result   stderr result

User · Answer

os popen   is the easiest and the most safest way to execute a command  You can execute any command that you run on the command line  In addition you will also be able to capture the output of the command using os popen   read    You can do it like this   import os output   os popen  Your Command Here   read   print  output    An example where you list all the files in the current directory   import os output   os popen  ls   read   print  output    Outputs list of files in the directory

User · Answer

Using the Popen function of the subprocess Python module is the simplest way of running Linux commands  In that  the Popen communicate   function will give your commands output  For example  import subprocess     process   subprocess Popen         Pass command and arguments to the function stdout  stderr   process communicate       Get command output and error

User · Answer

Calling an external command in Python  A simple way to call an external command is using os system       And this function returns the exit value of the command  But the drawback is we won t get stdout and stderr   ret   os system  some cmd sh   if ret    0       print  some cmd sh execution returned failure    Calling an external command in Python in background  subprocess Popen provides more flexibility for running an external command rather than using os system  We can start a command in the background and wait for it to finish  And after that we can get the stdout and stderr   proc   subprocess Popen     some cmd sh    stdout subprocess PIPE  print  waiting for     str proc pid  proc wait   print  some cmd sh execution finished   out  err    proc communicate   print  some cmd sh output       out   Calling a long running external command in Python in the background and stop after some time  We can even start a long running process in the background using subprocess Popen and kill it after sometime once its task is done   proc   subprocess Popen     some long run cmd sh    stdout subprocess PIPE    Do something else   Now some long run cmd sh exeuction is no longer needed  so kill it os system  kill -15     str proc pid   print  Output     proc communicate   0

User · Answer

os system is OK  but kind of dated   It s also not very secure   Instead  try subprocess   subprocess does not call sh directly and is therefore more secure than os system   Get more information here

User · Answer

Invoke is a Python  2 7 and 3 4   task execution tool  amp  library  It provides a clean  high level API for running shell commands   gt  gt  gt  from invoke import run  gt  gt  gt  cmd    pip install -r requirements txt   gt  gt  gt  result   run cmd  hide True  warn True   gt  gt  gt  print result ok  True  gt  gt  gt  print result stdout splitlines   -1   Successfully installed invocations-0 13 0 pep8-1 5 7 spec-1 3 1

User · Answer

Typical implementation   import subprocess  p   subprocess Popen  ls   shell True  stdout subprocess PIPE  stderr subprocess STDOUT  for line in p stdout readlines        print line  retval   p wait     You are free to do what you want with the stdout data in the pipe   In fact  you can simply omit those parameters  stdout  and stderr   and it ll behave like os system

User · Answer

Use subprocess      or for a very simple command   import os os system  cat testfile

User · Answer

Some hints on detaching the child process from the calling one  starting the child process in background    Suppose you want to start a long task from a CGI script  That is  the child process should live longer than the CGI script execution process   The classical example from the subprocess module documentation is   import subprocess import sys    Some code here  pid   subprocess Popen  sys executable   longtask py      Call subprocess    Some more code here   The idea here is that you do not want to wait in the line  call subprocess  until the longtask py is finished  But it is not clear what happens after the line  some more code here  from the example   My target platform was FreeBSD  but the development was on Windows  so I faced the problem on Windows first   On Windows  Windows nbsp XP   the parent process will not finish until the longtask py has finished its work  It is not what you want in a CGI script  The problem is not specific to Python  in the PHP community the problems are the same   The solution is to pass DETACHED PROCESS Process Creation Flag to the underlying CreateProcess function in Windows API  If you happen to have installed pywin32  you can import the flag from the win32process module  otherwise you should define it yourself   DETACHED PROCESS   0x00000008  pid   subprocess Popen  sys executable   longtask py                           creationflags DETACHED PROCESS  pid      UPD 2015 10 27  eryksun in a comment below notes  that the semantically correct flag is CREATE NEW CONSOLE  0x00000010      On FreeBSD we have another problem  when the parent process is finished  it finishes the child processes as well  And that is not what you want in a CGI script either  Some experiments showed that the problem seemed to be in sharing sys stdout  And the working solution was the following   pid   subprocess Popen  sys executable   longtask py    stdout subprocess PIPE  stderr subprocess PIPE  stdin subprocess PIPE    I have not checked the code on other platforms and do not know the reasons of the behaviour on FreeBSD  If anyone knows  please share your ideas  Googling on starting background processes in Python does not shed any light yet

User · Answer

It can be this simple   import os cmd    your command  os system cmd

User · Answer

You can use Popen  and then you can check the procedure s status   from subprocess import Popen  proc   Popen   ls    -l    if proc poll   is None      proc kill     Check out subprocess Popen

User · Answer

import os cmd    ls -al  os system cmd    If you want to return the results of the command  you can use os popen  However  this is deprecated since version 2 6 in favor of the subprocess module  which other answers have covered well

User · Answer

I d recommend using the subprocess module instead of os system because it does shell escaping for you and is therefore much safer   subprocess call   ping    localhost

User · Answer

Often  I use the following function for external commands  and this is especially handy for long running processes  The below method tails process output while it is running and returns the output  raises an exception if process fails   It comes out if the process is done using the poll   method on the process   import subprocess sys  def exec long running proc command  args       cmd           format command      join str arg  if     not in arg else arg replace           for arg in args       print cmd      process   subprocess Popen cmd  shell True  stdout subprocess PIPE  stderr subprocess STDOUT         Poll process for new output until finished     while True          nextline   process stdout readline   decode  UTF-8           if nextline       and process poll   is not None              break         sys stdout write nextline          sys stdout flush        output   process communicate   0      exitCode   process returncode      if  exitCode    0           return output     else          raise Exception command  exitCode  output    You can invoke it like this   exec long running proc command    hive   args   -f   hql path

User · Answer

import os cmd    ls -al  os system cmd    If you want to return the results of the command  you can use os popen  However  this is deprecated since version 2 6 in favor of the subprocess module  which other answers have covered well

User · Answer

I quite like shell command for its simplicity   It s built on top of the subprocess module   Here s an example from the documentation    gt  gt  gt  from shell command import shell call  gt  gt  gt  shell call  ls   py   setup py  shell command py  test shell command py 0  gt  gt  gt  shell call  ls -l   py   -rw-r--r-- 1 ncoghlan ncoghlan  391 2011-12-11 12 07 setup py -rw-r--r-- 1 ncoghlan ncoghlan 7855 2011-12-11 16 16 shell command py -rwxr-xr-x 1 ncoghlan ncoghlan 8463 2011-12-11 16 17 test shell command py 0

User · Answer

os system does not allow you to store results  so if you want to store results in some list or something  a subprocess call works

User · Answer

Often  I use the following function for external commands  and this is especially handy for long running processes  The below method tails process output while it is running and returns the output  raises an exception if process fails   It comes out if the process is done using the poll   method on the process   import subprocess sys  def exec long running proc command  args       cmd           format command      join str arg  if     not in arg else arg replace           for arg in args       print cmd      process   subprocess Popen cmd  shell True  stdout subprocess PIPE  stderr subprocess STDOUT         Poll process for new output until finished     while True          nextline   process stdout readline   decode  UTF-8           if nextline       and process poll   is not None              break         sys stdout write nextline          sys stdout flush        output   process communicate   0      exitCode   process returncode      if  exitCode    0           return output     else          raise Exception command  exitCode  output    You can invoke it like this   exec long running proc command    hive   args   -f   hql path

User · Answer

import os os system  your command     Note that this is dangerous  since the command isn t cleaned  I leave it up to you to google for the relevant documentation on the  os  and  sys  modules  There are a bunch of functions  exec  and spawn   that will do similar things

User · Answer

os system is OK  but kind of dated   It s also not very secure   Instead  try subprocess   subprocess does not call sh directly and is therefore more secure than os system   Get more information here

User · Answer

i use this for 3 6   import subprocess def execute cmd                   Purpose    To execute a command and return exit status         Argument   cmd - command to execute         Return     result  exit code             process   subprocess Popen cmd  shell True  stdout subprocess PIPE  stderr subprocess PIPE       result  error    process communicate       rc   process wait       if rc    0          print   Error  failed to execute command     cmd          print  error rstrip   decode  utf-8        return result rstrip   decode  utf-8    serror rstrip   decode  utf-8     def

User · Answer

import os os system  your command     Note that this is dangerous  since the command isn t cleaned  I leave it up to you to google for the relevant documentation on the  os  and  sys  modules  There are a bunch of functions  exec  and spawn   that will do similar things

User · Answer

Use the subprocess module in the standard library  import subprocess subprocess run   quot ls quot    quot -l quot     The advantage of subprocess run over os system is that it is more flexible  you can get the stdout  stderr  the  quot real quot  status code  better error handling  etc      Even the documentation for os system recommends using subprocess instead   The subprocess module provides more powerful facilities for spawning new processes and retrieving their results  using that module is preferable to using this function  See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes   On Python 3 4 and earlier  use subprocess call instead of  run  subprocess call   quot ls quot    quot -l quot

User · Answer

I have written a wrapper to handle errors and redirecting output and other stuff   import shlex import psutil import subprocess  def call cmd cmd  stdout sys stdout  quiet False  shell False  raise exceptions True  use shlex True  timeout None          Exec command by command line like  ln -ls   var log               if not quiet          print  Run  s   str cmd       if use shlex and isinstance cmd   str  unicode            cmd   shlex split cmd      if timeout is None          process   subprocess Popen cmd  stdout stdout  stderr sys stderr  shell shell          retcode   process wait       else          process   subprocess Popen cmd  stdout stdout  stderr sys stderr  shell shell          p   psutil Process process pid          finish  alive   psutil wait procs  p   timeout          if len alive   gt  0              ps   p children               ps insert 0  p              print  waiting for timeout again due to child process check               finish  alive   psutil wait procs ps  0          if len alive   gt  0              print  process    will be killed  format  p pid for p in alive                for p in alive                  p kill               if raise exceptions                  print  External program timeout at        format timeout  cmd                   raise CalledProcessTimeout 1  cmd          retcode   process wait       if retcode and raise exceptions          print  External program failed  s   str cmd           raise subprocess CalledProcessError retcode  cmd    You can call it like this   cmd    ln -ls   var log   stdout    out txt  call cmd cmd  stdout

User · Answer

I wrote a small library to help with this use case   https   pypi org project citizenshell   It can be installed using   pip install citizenshell   And then used as follows   from citizenshell import sh assert sh  echo Hello World       Hello World    You can separate stdout from stderr and extract the exit code as follows   result   sh   gt  amp 2 echo error  amp  amp  echo output  amp  amp  exit 13   assert result stdout        output   assert result stderr        error   assert result exit code      13   And the cool thing is that you don t have to wait for the underlying shell to exit before starting processing the output   for line in sh  for i in 1 2 3 4  do echo -n  It is    date   H  M  S  sleep 1  done   wait False      print   gt  gt  gt    line         will print the lines as they are available thanks to the wait False   gt  gt  gt  It is 14 24 52   gt  gt  gt  It is 14 24 53   gt  gt  gt  It is 14 24 54   gt  gt  gt  It is 14 24 55    More examples can be found at https   github com meuter citizenshell

User · Answer

As some of the answers were related to the previous versions of Python or were using os system module  I post this answer for people like me who intend to use subprocess in Python nbsp 3 5   The following did the trick for me on Linux   import subprocess    subprocess run   returns a completed process object that can be inspected c   subprocess run   ls    -ltrh    stdout subprocess PIPE  stderr subprocess PIPE  print c stdout decode  utf-8      As mentioned in the documentation  PIPE values are byte sequences and for properly showing them decoding should be considered  For later versions of Python  text True and encoding  utf-8  are added to kwargs of subprocess run     The output of the abovementioned code is   total 113M -rwxr-xr-x  1 farzad farzad  307 Jan 15  2018 vpnscript -rwxrwxr-x  1 farzad farzad  204 Jan 15  2018 ex drwxrwxr-x  4 farzad farzad 4 0K Jan 22  2018 scripts        Some other lines

User · Answer

Here s a summary of the ways to call external programs and the advantages and disadvantages of each    os system  some command with args   passes the command and arguments to your system s shell   This is nice because you can actually run multiple commands at once in this manner and set up pipes and input output redirection   For example     os system  some command  lt  input file   another command  gt  output file        However  while this is convenient  you have to manually handle the escaping of shell characters such as spaces  etc   On the other hand  this also lets you run commands which are simply shell commands and not actually external programs   See the documentation    stream   os popen  some command with args   will do the same thing as os system except that it gives you a file-like object that you can use to access standard input output for that process   There are 3 other variants of popen that all handle the i o slightly differently   If you pass everything as a string  then your command is passed to the shell  if you pass them as a list then you don t need to worry about escaping anything   See the documentation  The Popen class of the subprocess module   This is intended as a replacement for os popen but has the downside of being slightly more complicated by virtue of being so comprehensive   For example  you d say   print subprocess Popen  echo Hello World   shell True  stdout subprocess PIPE  stdout read     instead of    print os popen  echo Hello World   read     but it is nice to have all of the options there in one unified class instead of 4 different popen functions   See the documentation  The call function from the subprocess module   This is basically just like the Popen class and takes all of the same arguments  but it simply waits until the command completes and gives you the return code   For example   return code   subprocess call  echo Hello World   shell True      See the documentation  If you re on Python 3 5 or later  you can use the new subprocess run function  which is a lot like the above but even more flexible and returns a CompletedProcess object when the command finishes executing  The os module also has all of the fork exec spawn functions that you d have in a C program  but I don t recommend using them directly    The subprocess module should probably be what you use   Finally please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it  There are serious security implications if any part of the string that you pass can not be fully trusted  For example  if a user is entering some any part of the string  If you are unsure  only use these methods with constants  To give you a hint of the implications consider this code   print subprocess Popen  echo  s     user input  stdout PIPE  stdout read     and imagine that the user enters something  my mama didnt love me  amp  amp  rm -rf    which could erase the whole filesystem

User · Answer

There are a lot of different ways to run external commands in Python  and all of them have their own plus sides and drawbacks   My colleagues and me have been writing Python system administration tools  so we need to run a lot of external commands  and sometimes you want them to block or run asynchronously  time-out  update every second  etc   There are also different ways of handling the return code and errors  and you might want to parse the output  and provide new input  in an expect kind of style   Or you will need to redirect stdin  stdout and stderr to run in a different tty  e g   when using screen    So you will probably have to write a lot of wrappers around the external command  So here is a Python module which we have written which can handle almost anything you would want  and if not  it s very flexible so you can easily extend it   https   github com hpcugent vsc-base blob master lib vsc utils run py  update  It doesn t work standalone and requires some of our other tools  and got a lot of specialised functionality over the years  so it might not be a drop in replacement for you  but can give you a lot of info on how the internals of python for running commands work and idea s on how to handle certain situations

User · Answer

subprocess check call is convenient if you don t want to test return values  It throws an exception on any error

User · Answer

If you re writing a Python shell script and have IPython installed on your system  you can use the bang prefix to run a shell command inside IPython    ls filelist    ls

User · Answer

In Windows you can just import the subprocess module and run external commands by calling subprocess Popen    subprocess Popen   communicate   and subprocess Popen   wait   as below     Python script to run a command line import subprocess  def execute cmd                   Purpose    To execute a command and return exit status         Argument   cmd - command to execute         Return     exit code             process   subprocess Popen cmd  shell True  stdout subprocess PIPE  stderr subprocess PIPE       result  error    process communicate        rc   process wait        if rc    0          print  Error  failed to execute command    cmd         print error     return result   def  command    tasklist   grep python  print  This process detail   n   execute command    Output   This process detail  python exe                     604 RDP-Tcp 0                  4      5 660 K

User · Answer

There are many ways to call a command    For example    if and exe needs two parameters  In cmd we can call sample exe use this     and exe 2 3 and it show 5 on screen   If we use a Python script to call and exe  we should do like     os system cmd        os system   and exe           2           3     os popen cmd        os popen   and exe           2           3     subprocess Popen cmd        subprocess Popen   and exe           2           3       It s too hard  so we can join cmd with a space   import os cmd       join exename parameters  os popen cmd

User · Answer

Just to add to the discussion  if you include using a Python console  you can call external commands from IPython  While in the IPython prompt  you can call shell commands by prefixing      You can also combine Python code with the shell  and assign the output of shell scripts to Python variables   For instance   In  9   mylist    ls  In  10   mylist Out 10     file1     file2     file3

User · Answer

os system is OK  but kind of dated   It s also not very secure   Instead  try subprocess   subprocess does not call sh directly and is therefore more secure than os system   Get more information here

User · Answer

This can be done in 2 ways  1  os system    import os  To run a system command     os system  cls    To execute a program     os system  code1 py    2  subprocess call    import subprocess  To run a system command     subprocess call  cls  shell True   To execute a program     subprocess call  code1 py  shell True

[python] How to execute a program or call a system command from Python

Examples related to python

Examples related to shell

Examples related to terminal

Examples related to subprocess

Examples related to command