[linux] Multithreading in Bash

I would like to introduce multithreading feature in my shell script.

I have a script which calls the function read_cfg() with different arguments. Each of these function calls are independent.

Would it be possible to instantiate these function calls (not scripts) parallelly. Please let me how can we achieve that.. ?

This question is related to linux multithreading bash shell

The answer is


Sure, just add & after the command:

read_cfg cfgA &
read_cfg cfgB &
read_cfg cfgC &
wait

all those jobs will then run in the background simultaneously. The optional wait command will then wait for all the jobs to finish.

Each command will run in a separate process, so it's technically not "multithreading", but I believe it solves your problem.


Bash job control involves multiple processes, not multiple threads.

You can execute a command in background with the & suffix.

You can wait for completion of a background command with the wait command.

You can execute multiple commands in parallel by separating them with |. This provides also a synchronization mechanism, since stdout of a command at left of | is connected to stdin of command at right.


You can run several copies of your script in parallel, each copy for different input data, e.g. to process all *.cfg files on 4 cores:

    ls *.cfg | xargs -P 4 -n 1 read_cfg.sh

The read_cfg.sh script takes just one parameters (as enforced by -n)


Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to multithreading

How can compare-and-swap be used for a wait-free mutual exclusion for any shared data structure? Waiting until the task finishes What is the difference between Task.Run() and Task.Factory.StartNew() Why is setState in reactjs Async instead of Sync? What exactly is std::atomic? Calling async method on button click WAITING at sun.misc.Unsafe.park(Native Method) How to use background thread in swift? What is the use of static synchronized method in java? Locking pattern for proper use of .NET MemoryCache

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to shell

Comparing a variable with a string python not working when redirecting from bash script Get first line of a shell command's output How to run shell script file using nodejs? Run bash command on jenkins pipeline Way to create multiline comments in Bash? How to do multiline shell script in Ansible How to check if a file exists in a shell script How to check if an environment variable exists and get its value? Curl to return http status code along with the response docker entrypoint running bash script gets "permission denied"