How to process SIGTERM signal gracefully

Question

Let s assume we have such a trivial daemon written in python   def mainloop        while True            1  do           2  some           3  important           4  job           5  sleep  mainloop     and we daemonize it using start-stop-daemon which by default sends SIGTERM  TERM  signal on --stop   Let s suppose the current step performed is  2  And at this very moment we re sending TERM signal   What happens is that the execution terminates immediately   I ve found that I can handle the signal event using signal signal signal SIGTERM  handler  but the thing is that it still interrupts the current execution and passes the control to handler   So  my question is - is it possible to not interrupt the current execution but handle the TERM signal in a separated thread     so that I was able to set shutdown flag   True so that mainloop   had a chance to stop gracefully

User · Answer

I think you are near to a possible solution.

Execute mainloop in a separate thread and extend it with the property shutdown_flag. The signal can be caught with signal.signal(signal.SIGTERM, handler) in the main thread (not in a separate thread). The signal handler should set shutdown_flag to True and wait for the thread to end with thread.join()

User · Answer

First  I m not certain that you need a second thread to set the shutdown flag  Why not set it directly in the SIGTERM handler   An alternative is to raise an exception from the SIGTERM handler  which will be propagated up the stack   Assuming you ve got proper exception handling  e g  with with contextmanager and try      finally  blocks  this should be a fairly graceful shutdown  similar to if you were to Ctrl C your program   Example program signals-test py      usr bin python  from time import sleep import signal import sys   def sigterm handler  signo   stack frame         Raises SystemExit 0       sys exit 0   if sys argv 1      handle signal       signal signal signal SIGTERM  sigterm handler   try      print  Hello      i   0     while True          i    1         print  Iteration   i    i         sleep 1  finally      print  Goodbye    Now see the Ctrl C behaviour       signals-test py default Hello Iteration  1 Iteration  2 Iteration  3 Iteration  4  CGoodbye Traceback  most recent call last     File    signals-test py   line 21  in  lt module gt      sleep 1  KeyboardInterrupt   echo    1   This time I send it SIGTERM after 4 iterations with kill   ps aux   grep signals-test   awk   python   print  2          signals-test py default Hello Iteration  1 Iteration  2 Iteration  3 Iteration  4 Terminated   echo    143   This time I enable my custom SIGTERM handler and send it SIGTERM       signals-test py handle signal Hello Iteration  1 Iteration  2 Iteration  3 Iteration  4 Goodbye   echo    0

User · Answer

Found easiest way for me  Here an example with fork for clarity that this way is useful for flow control   import signal import time import sys import os  def handle exit sig  frame       raise SystemExit   def main        time sleep 120   signal signal signal SIGTERM  handle exit   p   os fork   if p    0      main       os  exit    try      os waitpid p  0  except  KeyboardInterrupt  SystemExit       print  exit handled       os kill p  15      os waitpid p  0

User · Answer

Based on the previous answers  I have created a context manager which protects from sigint and sigterm   import logging import signal import sys   class TerminateProtected          Protect a piece of code from being killed by SIGINT or SIGTERM      It can still be killed by a force kill       Example          with TerminateProtected                run func 1               run func 2        Both functions will be executed even if a sigterm or sigkill has been received              killed   False      def  handler self  signum  frame           logging error  Received SIGINT or SIGTERM  Finishing this block  then exiting            self killed   True      def   enter   self           self old sigint   signal signal signal SIGINT  self  handler          self old sigterm   signal signal signal SIGTERM  self  handler       def   exit   self  type  value  traceback           if self killed              sys exit 0          signal signal signal SIGINT  self old sigint          signal signal signal SIGTERM  self old sigterm    if   name         main         print  Try pressing ctrl c while the sleep is running        from time import sleep     with TerminateProtected            sleep 10          print  Finished anyway        print  This only prints if there was no sigint or sigterm

User · Answer

The simplest solution I have found  taking inspiration by responses above is  class SignalHandler       def   init   self              register signal handlers         signal signal signal SIGINT  self exit gracefully          signal signal signal SIGTERM  self exit gracefully           self logger   Logger level ERROR       def exit gracefully self  signum  frame           self logger info  captured signal  d    signum          traceback print stack frame                  do your resources clean up here                raise SystemExit

User · Answer

A class based clean to use solution   import signal import time  class GracefulKiller    kill now   False   def   init   self       signal signal signal SIGINT  self exit gracefully      signal signal signal SIGTERM  self exit gracefully     def exit gracefully self signum  frame       self kill now   True  if   name         main       killer   GracefulKiller     while not killer kill now      time sleep 1      print  doing something in a loop          print  End of the program  I was killed gracefully

User · Answer

Here is a simple example without threads or classes   import signal  run   True  def handler stop signals signum  frame       global run     run   False  signal signal signal SIGINT  handler stop signals  signal signal signal SIGTERM  handler stop signals   while run      pass   do stuff including other IO stuff

[python] How to process SIGTERM signal gracefully?

Examples related to python

Examples related to daemon

Examples related to sigterm

Examples related to start-stop-daemon