Waiting for background processes to finish before exiting script

Question

How do I make sure that all my background processes have finished execution before I exit my script  TCL Bash    I was thinking of writing all my background process pids to a pidfile  And then at the end pgrep the pidfile to see if any processes are still running before I exit    Is there some simpler way to do this  And is there a TCL specific way to do this

User · Answer

GNU parallel and xargs These two tools that can make scripts simpler  and also control the maximum number of threads  thread pool   E g   seq 10   xargs -P4 -I     echo       or  seq 10   parallel -j4  echo       See also  how to write a process-pool bash shell

User · Answer

You can use kill -0 for checking whether a particular pid is running or not   Assuming  you have list of pid numbers in a file called pid in pwd  while true  do      if   -s pid     then         for pid in  cat pid          do               echo  Checking the  pid              kill -0   pid  2 gt  dev null    sed -i     pid  d  pid         done     else         echo  All your process completed     Do what you want here    here all your pids are in finished stated         break     fi done

User · Answer

If you want to wait for jobs to finish  use wait   This will make the shell wait until all background jobs complete   However  if any of your jobs daemonize themselves  they are no longer children of the shell and wait will have no effect  as far as the shell is concerned  the child is already done   Indeed  when a process daemonizes itself  it does so by terminating and spawning a new process that inherits its role       bin sh   sleep 5  echo waking up after 5 seconds     amp    sleep 1  echo waking up after 1 second     amp  wait echo all jobs are done

User · Answer

WARNING  Long script ahead   A while ago  I faced a similar problem  from a Tcl script  launch a number of processes  then wait for all of them to finish  Here is a demo script I wrote to solve this problem   main tcl     usr bin env tclsh    Launches many processes and wait for them to finish    This script will works on systems that has the ps command such as   BSD  Linux  and OS X  package require Tclx    For process-management utilities  proc updatePidList  stat        global pidList     global allFinished        Parse the process ID of the just-finished process     lassign  stat processId howProcessEnded exitCode        Remove this process ID from the list of process IDs     set pidList  lindex  intersect3  pidList  processId  0      set processCount  llength  pidList         Occasionally  a child process quits but the signal was lost  This       block of code will go through the list of remaining process IDs       and remove those that has finished     set updatedPidList        foreach pid  pidList           if    catch  exec ps  pid  errmsg                 lappend updatedPidList  pid                      set pidList  updatedPidList        Show the remaining processes     if   processCount  gt  0            puts  Waiting for  llength  pidList  processes        else           set allFinished 1         puts  All finished             A signal handler that gets called when a child process finished    This handler needs to exit quickly  so it delegates the real works to   the proc updatePidList proc childTerminated            Restart the handler     signal -restart trap SIGCHLD childTerminated        Update the list of process IDs     while    catch  wait -nohang  stat   amp  amp   stat ne               after idle  list updatePidList  stat               Main starts here    puts  Main begins  set NUMBER OF PROCESSES TO LAUNCH 10 set pidList    set allFinished 0    When a child process exits  call proc childTerminated signal -restart trap SIGCHLD childTerminated    Spawn many processes for  set i 0    i  lt   NUMBER OF PROCESSES TO LAUNCH   incr i        set childId  exec tclsh child tcl  i  amp       puts  child   i  pid  childId      lappend pidList  childId     after 1000      Do some processing puts  list of processes   pidList  puts  Waiting for child processes to finish    Do some more processing if required    After all done  wait for all to finish before exiting vwait allFinished  puts  Main ends    child tcl     usr bin env tclsh   child script  simulate some lengthy operations  proc randomInteger  min max        return  expr int rand       max -  min   1    1000    min      set duration  randomInteger 10 30  puts    child   argv runs for  duration miliseconds  after  duration puts    child   argv ends    Sample output for running main tcl  Main begins child  0  pid 64525   child  0 runs for 17466 miliseconds child  1  pid 64526   child  1 runs for 14181 miliseconds child  2  pid 64527   child  2 runs for 10856 miliseconds child  3  pid 64528   child  3 runs for 7464 miliseconds child  4  pid 64529   child  4 runs for 4034 miliseconds child  5  pid 64531   child  5 runs for 1068 miliseconds child  6  pid 64532   child  6 runs for 18571 miliseconds   child  5 ends child  7  pid 64534   child  7 runs for 15374 miliseconds child  8  pid 64535   child  8 runs for 11996 miliseconds   child  4 ends child  9  pid 64536   child  9 runs for 8694 miliseconds list of processes  64525 64526 64527 64528 64529 64531 64532 64534 64535 64536 Waiting for child processes to finish Waiting for 8 processes Waiting for 8 processes   child  3 ends Waiting for 7 processes   child  2 ends Waiting for 6 processes   child  1 ends Waiting for 5 processes   child  0 ends Waiting for 4 processes   child  9 ends Waiting for 3 processes   child  8 ends Waiting for 2 processes   child  7 ends Waiting for 1 processes   child  6 ends All finished Main ends

User · Answer

Even if you do not have the pid  you can trigger  wait   after triggering all background processes  For  eg  in commandfile sh- bteq  lt  input file1 sql  gt  output file1 sql  amp  bteq  lt  input file2 sql  gt  output file2 sql  amp  bteq  lt  input file3 sql  gt  output file3 sql  amp  wait  Then when this is triggered  as - subprocess call   sh    commandfile sh    print  all background processes done     This will be printed only after all the background processes are done

[bash] Waiting for background processes to finish before exiting script

Examples related to bash

Examples related to shell

Examples related to unix

Examples related to tcl

Examples related to grep