[python] OSError: [Errno 8] Exec format error

I am having hard time parsing the arguments to subprocess.Popen. I am trying to execute a script on my Unix server. The script syntax when running on shell prompt is as follows: /usr/local/bin/script hostname = <hostname> -p LONGLIST. No matter how I try, the script is not running inside subprocess.Popen

The space before and after "=" is mandatory.

import subprocess
Out = subprocess.Popen(['/usr/local/bin/script', 'hostname = ', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)

The above does not work.

And when I use shell=False, I get OSError: [Errno 8] Exec format error

Have you tried this?

Out = subprocess.Popen('/usr/local/bin/script hostname = actual_server_name -p LONGLIST'.split(), shell=False,stdout=subprocess.PIPE,stderr=subprocess.PIPE) 

Edited per the apt comment from @J.F.Sebastian

It wouldn't be wrong to mention that Pexpect does throw a similar error

#python -c "import pexpect; p=pexpect.spawn('/usr/local/ssl/bin/openssl_1.1.0f  version'); p.interact()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/pexpect.py", line 430, in __init__
    self._spawn (command, args)
  File "/usr/lib/python2.7/site-packages/pexpect.py", line 560, in _spawn
    os.execv(self.command, self.args)
OSError: [Errno 8] Exec format error

Over here, the openssl_1.1.0f file at the specified path has exec command specified in it and is running the actual openssl binary when called.

Usually, I wouldn't mention this unless I have the root cause, but this problem was not there earlier. Unable to find the similar problem, the closest explanation to make it work is the same as the one provided by @jfs above.

what worked for me is both

• adding /bin/bash at the beginning of the command or file you are
  facing the problem with, or
• adding shebang #!/bin/sh as the first line.

for ex.

#python -c "import pexpect; p=pexpect.spawn('/bin/bash /usr/local/ssl/bin/openssl_1.1.0f  version'); p.interact()"
OpenSSL 1.1.0f  25 May 2017

OSError: [Errno 8] Exec format error can happen if there is no shebang line at the top of the shell script and you are trying to execute the script directly. Here's an example that reproduces the issue:

>>> with open('a','w') as f: f.write('exit 0') # create the script
... 
>>> import os
>>> os.chmod('a', 0b111101101) # rwxr-xr-x make it executable                       
>>> os.execl('./a', './a')     # execute it                                            
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/os.py", line 312, in execl
    execv(file, args)
OSError: [Errno 8] Exec format error

To fix it, just add the shebang e.g., if it is a shell script; prepend #!/bin/sh at the top of your script:

>>> with open('a','w') as f: f.write('#!/bin/sh\nexit 0')
... 
>>> os.execl('./a', './a')

It executes exit 0 without any errors.

On POSIX systems, shell parses the command line i.e., your script won't see spaces around = e.g., if script is:

#!/usr/bin/env python
import sys
print(sys.argv)

then running it in the shell:

$ /usr/local/bin/script hostname = '<hostname>' -p LONGLIST

produces:

['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST']

Note: no spaces around '='. I've added quotes around <hostname> to escape the redirection metacharacters <>.

To emulate the shell command in Python, run:

from subprocess import check_call

cmd = ['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST']
check_call(cmd)

Note: no shell=True. And you don't need to escape <> because no shell is run.

"Exec format error" might indicate that your script has invalid format, run:

$ file /usr/local/bin/script

to find out what it is. Compare the architecture with the output of:

$ uname -m

I will hijack this thread to point out that this error may also happen when target of Popen is not executable. Learnt it hard way when by accident I have had override a perfectly executable binary file with zip file.

If you think the space before and after "=" is mandatory, try it as separate item in the list.

Out = subprocess.Popen(['/usr/local/bin/script', 'hostname', '=', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)