Python vs Bash - In which kind of tasks each one outruns the other performance-wise

Question

Obviously Python is more user friendly  a quick search on google shows many results that say that  as Python is byte-compiled is usually faster  I even found this that claims that you can see an improvement of over 2000  on dictionary-based operations   What is your experience on this matter  In which kind of task each one is a clear winner

User · Answer

Performance-wise bash outperforms python in the process startup time.

Here are some measurements from my core i7 laptop running Linux Mint:

Starting process                       Startup time

empty /bin/sh script                   1.7 ms
empty /bin/bash script                 2.8 ms
empty python script                    11.1 ms
python script with a few libs*         110 ms

*Python loaded libs are: os, os.path, json, time, requests, threading, subprocess

This shows a huge difference however bash execution time degrades quickly if it has to do anything sensible since it usually must call external processes.

If you care about performance use bash only for:

really simple and frequently called scripts
scripts that mainly call other processes
when you need minimal friction between manual administrative actions and scripting - fast check a few commands and place them in the file.sh

User · Answer

There are 2 scenario s where Bash performance is at least equal I believe    Scripting of command line utilities Scripts which take only a short time to execute  where starting the Python interpreter takes more time than the operation itself   That said  I usually don t really concern myself with performance of the scripting language itself  If performance is a real issue you don t script but program  possibly in Python

User · Answer

Typical mainframe flow       Input Disk Tape User  runtime  -- gt  Job Control Language  JCL  -- gt  Output Disk Tape Screen Printer                                                                                                    v                                                                -- gt  COBOL Program --------     Typical Linux flow     Input Disk SSD User  runtime  -- gt  sh bash ksh zsh     ---------- gt  Output Disk SSD Screen Printer                                                                                                    v                                                                -- gt  Python script --------                                                                                                     v                                                                -- gt  awk script -----------                                                                                                     v                                                                -- gt  sed script -----------                                                                                                     v                                                                -- gt  C C   program --------                                                                                                     v                                                                --- Java program ---------                                                                                                     v                                                                                              Shells are the glue of Linux  Linux shells like sh ksh bash     provide input output flow-control designation facilities much like the old mainframe Job Control Language    but on steroids  They are Turing complete languages in their own right while being optimized to efficiently pass data and control to and from other executing processes written in any language the O S supports   Most Linux applications  regardless what language the bulk of the program is written in  depend on shell scripts and Bash has become the most common  Clicking an icon on the desktop usually runs a short Bash script  That script  either directly or indirectly  knows where all the files needed are and sets variables and command line parameters  finally calling the program  That s a shell s simplest use   Linux as we know it however would hardly be Linux without the thousands of shell scripts that startup the system  respond to events  control execution priorities and compile  configure and run programs  Many of these are quite large and complex   Shells provide an infrastructure that lets us use pre-built components that are linked together at run time rather than compile time  Those components are free-standing programs in their own right that can be used alone or in other combinations without recompiling  The syntax for calling them is indistinguishable from that of a Bash builtin command  and there are in fact numerous builtin commands for which there is also a stand-alone executable on the system  often having additional options   There is no language-wide difference between Python and Bash in performance  It entirely depends on how each is coded and which external tools are called   Any of the well known tools like awk  sed  grep  bc  dc  tr  etc  will leave doing those operations in either language in the dust  Bash then is preferred for anything without a graphical user interface since it is easier and more efficient to call and pass data back from a tool like those with Bash than Python   Performance  It depends on which programs the Bash shell script calls and their suitability for the subtask they are given whether the overall throughput and or responsiveness will be better or worse than the equivalent Python  To complicate matters Python  like most languages  can also call other executables  though it is more cumbersome and thus not as often used   User Interface  One area where Python is the clear winner is user interface  That makes it an excellent language for building local or client-server applications as it natively supports GTK graphics and is far more intuitive than Bash   Bash only understands text  Other tools must be called for a GUI and data passed back from them  A Python script is one option  Faster but less flexible options are the binaries like YAD  Zenity  and GTKDialog   While shells like Bash work well with GUIs like Yad  GtkDialog  embedded XML-like interface to GTK  functions   dialog  and xmessage  Python is much more capable and so better for complex GUI windows   Summary  Building with shell scripts is like assembling a computer with off-the-shelf components the way desktop PCs are   Building with Python  C   or most any other language is more like building a computer by soldering the chips  libraries  and other electronic parts together the way smartphones are   The best results are usually obtained by using a combination of languages where each can do what they do best  One developer calls this  polyglot programming

User · Answer

I m posting this late answer primarily because Google likes this question   I believe the issue and context really should be about the workflow  not the tools   The overall philosophy is always  Use the right tool for the job    But before this comes one that many often forget when they get lost in the tools   Get the job done    When I have a problem that isn t completely defined  I almost always start with Bash   I have solved some gnarly problems in large Bash scripts that are both readable and maintainable   But when does the problem start to exceed what Bash should be asked to do   I have some checks I use to give me warnings    Am I wishing Bash had 2D  or higher  arrays   If yes  it s time to realize that Bash is not a great data processing language  Am I doing more work preparing data for other utilities than I am actually running those utilities   If yes  time again to realize Bash is not a great data processing language  Is my script simply getting too large to manage   If yes  it is important to realize that while Bash can import script libraries  it lacks a package system like other languages   It s really a  roll your own  language compared to most others   Then again  it has a enormous amount of functionality built-in  some say too much       The list goes on   Bottom-line  when you are working harder to keep your scripts running that you do adding features  it s time to leave Bash   Let s assume you ve decided to move your work to Python   If your Bash scripts are clean  the initial conversion is quite straightforward   There are even several converters   translators that will do the first pass for you   The next question is  What do you give up moving to Python    All calls to external utilities must be wrapped in something from the subprocess module  or equivalent    There are multiple ways to do this  and until 3 7 it took some effort to get it right  3 7 improved subprocess run   to handle all common cases on its own   Surprisingly  Python has no standard platform-independent non-blocking utility  with timeout  for polling the keyboard  stdin    The Bash read command is an awesome tool for simple user interaction   My most common use is to show a spinner until the user presses a key  while also running a polling function  with each spinner step  to make sure things are still running well   This is a harder problem than it would appear at first  so I often simply make a call to Bash  Expensive  but it does precisely what I need  If you are developing on an embedded or memory-constrained system  Python s memory footprint can be many times larger than Bash s  depending on the task at hand    Plus  there is almost always an instance of Bash already in memory  which may not be the case for Python  For scripts that run once and exit quickly  Python s startup time can be much longer than Bash s   But if the script contains significant calculations  Python quickly pulls ahead  Python has the most comprehensive package system on the planet   When Bash gets even slightly complex  Python probably has a package that makes whole chunks of Bash become a single call   However  finding the right package s  to use is the biggest and most daunting part of becoming a Pythonista   Fortunately  Google and StackExchange are your friends

User · Answer

When you writing scripts performance does not matter  in most cases   If you care about performance  Python vs Bash  is a false question     Python    easier to write   easier to maintain   easier code reuse  try to find universal error-proof way to include files with common code in sh  I dare you    you can do OOP with it too    easier arguments parsing  well  not easier  exactly  it still will be too wordy to my taste  but python have argparse facility built in  - ugly ugly  subprocess   try to chain commands and not to cry a river how ugly your code will become  especially if you care about exit codes     Bash    ubiquity  as was said earlier  indeed    simple commands chaining  that s how you glue together different   commands in a simple way  Also Bash  not sh  have some improvements  like pipefail  so chaining is really short and expressive    do not require 3rd-party programs to be installed  can be executed right away  - god  it s full of gotchas  IFS  CDPATH   thousands of them   If one writing a script bigger than 100 LOC  choose Python If one need path manipulation in script  choose Python 3  If one need somewhat like alias but slightly complicated  choose Bash sh    Anyway  one should try both sides to get the idea what are they capable of    Maybe answer can be extended with packaging and IDE support points  but I m not familiar with this sides     As always you have to choose from turd sandwich and giant douche  And remember  just a few years ago Perl was new hope  Where it is now

User · Answer

If you are looking to cobble together a quick utility with minimal effort  bash is good   For a wrapper round an application  bash is invaluable   Anything that may have you coming back over and over to add improvements is probably  though not always  better suited to a language like Python as Bash code comprising over a 1000 lines gets very painful to maintain  Bash code is also irritating to debug when it gets long         Part of the problem with these kind of questions is  from my experience  that shell scripts are usually all custom tasks   There have been very few shell scripting tasks that I have come across where there is already a solution freely available

User · Answer

Bash is primarily a batch   shell scripting language with far less support for various data types and all sorts of quirks around control structures -- not to mention compatibility issues    Which is faster  Neither  because you are not comparing apples to apples here  If you had to sort an ascii text file and you were using tools like zcat  sort  uniq  and sed then you will smoke Python performance wise    However  if you need a proper programming environment that supports floating point and various control flow  then Python wins hands down  If you wrote say a recursive algorithm in Bash and Python  the Python version will win in an order of magnitude or more

User · Answer

Developer efficiency matters much more to me in scenarios where both bash and Python are sensible choices   Some tasks lend themselves well to bash  and others to Python   It also isn t unusual for me to start something as a bash script and change it to Python as it evolves over several weeks     A big advantage Python has is in corner cases around filename handling  while it has glob  shutil  subprocess  and others for common scripting needs

User · Answer

I don t know if this is accurate  but I have found that python ruby works much better for scripts that have a lot of mathematical computations  Otherwise you have to use dc or some other  arbitrary precision calculator   It just becomes a very big pain  With python you have much more control over floats vs ints and it is much easier to perform a lot of computations and sometimes   In particular  I would never work with a bash script to handle binary information or bytes  Instead I would use something like python  maybe  or C   or even Node JS

User · Answer

Performance wise both can do equally the same  so the question becomes which saves more development time   Bash relies on calling other commands  and piping them for creating new ones  This has the advantage that you can quickly create new programs just with the code borrowed from other people  no matter what programming language they used   This also has the side effect of resisting change in sub-commands pretty well  as the interface between them is just plain text   Additionally Bash is very permissive on how you can write on it  This means it will work well for a wider variety of context  but it also relies on the programmer having the intention of coding in a clean safe manner  Otherwise Bash won t stop you from building a mess   Python is more structured on style  so a messy programmer won t be as messy  It will also work on operating systems outside Linux  making it instantly more appropriate if you need that kind of portability   But it isn t as simple for calling other commands  So if your operating system is Unix most likely you will find that developing on Bash is the fastest way to develop   When to use Bash    It s a non graphical program  or the engine of a graphical one  It s only for Unix    When to use Python    It s a graphical program  It shall work on Windows

User · Answer

Generally  bash works better than python only in those environments where python is not available      Seriously  I have to deal with both languages daily  and will take python instantly over bash if given the choice  Alas  I am forced to use bash on certain  small  platforms because someone has  mistakenly  IMHO  decided that python is  too large  to fit   While it is true that bash might be faster than python for some select tasks  it can never be as quick to develop with  or as easy to maintain  at least after you get past 10 lines of code or so   Bash s sole strong point wrt python or ruby or lua  etc   is its ubiquity

[python] Python vs Bash - In which kind of tasks each one outruns the other performance-wise?

Examples related to python

Examples related to performance

Examples related to bash