How do you create a daemon in Python

Question

Searching on Google reveals x2 code snippets  The first result is to this code recipe which has a lot of documentation and explanation  along with some useful discussion underneath    However  another code sample  whilst not containing so much documentation  includes sample code for passing commands such as start  stop and restart  It also creates a PID file which can be handy for checking if the daemon is already running etc   These samples both explain how to create the daemon  Are there any additional things that need to be considered  Is one sample better than the other  and why

User · Answer

Note the python-daemon package which solves a lot of problems behind daemons out of the box.

Among other features it enables to (from Debian package description):

Detach the process into its own process group.
Set process environment appropriate for running inside a chroot.
Renounce suid and sgid privileges.
Close all open file descriptors.
Change the working directory, uid, gid, and umask.
Set appropriate signal handlers.
Open new file descriptors for stdin, stdout, and stderr.
Manage a specified PID lock file.
Register cleanup functions for at-exit processing.

User · Answer

80  of the time  when folks say  daemon   they only want a server   Since the question is perfectly unclear on this point  it s hard to say what the possible domain of answers could be   Since a server is adequate  start there   If an actual  daemon  is actually needed  this is rare   read up on nohup as a way to daemonize a server   Until such time as an actual daemon is actually required  just write a simple server   Also look at the WSGI reference implementation   Also look at the Simple HTTP Server    Are there any additional things that need to be considered     Yes   About a million things   What protocol   How many requests   How long to service each request   How frequently will they arrive   Will you use a dedicated process   Threads   Subprocesses   Writing a daemon is a big job

User · Answer

YapDi is a relatively new python module that popped up in Hacker News  Looks pretty useful  can be used to convert a python script into daemon mode from inside the script

User · Answer

This function will transform an application to a daemon   import sys import os  def daemonize        try          pid   os fork           if pid  gt  0                exit first parent             sys exit 0      except OSError as err          sys stderr write   Fork  1 failed   0  n  format err           sys exit 1        decouple from parent environment     os chdir          os setsid       os umask 0        do second fork     try          pid   os fork           if pid  gt  0                exit from second parent             sys exit 0      except OSError as err          sys stderr write   Fork  2 failed   0  n  format err           sys exit 1        redirect standard file descriptors     sys stdout flush       sys stderr flush       si   open os devnull   r       so   open os devnull   w       se   open os devnull   w       os dup2 si fileno    sys stdin fileno        os dup2 so fileno    sys stdout fileno        os dup2 se fileno    sys stderr fileno

User · Answer

Here s my basic  Howdy World  Python daemon that I start with  when I m developing a new daemon application      usr bin python import time from daemon import runner  class App        def   init   self           self stdin path     dev null          self stdout path     dev tty          self stderr path     dev tty          self pidfile path      tmp foo pid          self pidfile timeout   5     def run self           while True              print  Howdy   Gig em   Whoop                time sleep 10   app   App   daemon runner   runner DaemonRunner app  daemon runner do action     Note that you ll need the python-daemon library   You can install it by   pip install python-daemon   Then just start it with   howdy py start  and stop it with   howdy py stop

User · Answer

80  of the time  when folks say  daemon   they only want a server   Since the question is perfectly unclear on this point  it s hard to say what the possible domain of answers could be   Since a server is adequate  start there   If an actual  daemon  is actually needed  this is rare   read up on nohup as a way to daemonize a server   Until such time as an actual daemon is actually required  just write a simple server   Also look at the WSGI reference implementation   Also look at the Simple HTTP Server    Are there any additional things that need to be considered     Yes   About a million things   What protocol   How many requests   How long to service each request   How frequently will they arrive   Will you use a dedicated process   Threads   Subprocesses   Writing a daemon is a big job

User · Answer

One more to thing to think about when daemonizing in python   If your are using python logging and you want to continue using it after daemonizing  make sure to call close   on the handlers  particularly the file handlers    If you don t do this the handler can still think it has files open  and your messages will simply disappear - in other words make sure the logger knows its files are closed   This assumes when you daemonise you are closing ALL the open file descriptors indiscriminatingly - instead you could try closing all but the log files  but it s usually simpler to close all then reopen the ones you want

User · Answer

The easiest way to create daemon with Python is to use the Twisted event-driven framework   It handles all of the stuff necessary for daemonization for you   It uses the Reactor Pattern to handle concurrent requests

User · Answer

One more to thing to think about when daemonizing in python   If your are using python logging and you want to continue using it after daemonizing  make sure to call close   on the handlers  particularly the file handlers    If you don t do this the handler can still think it has files open  and your messages will simply disappear - in other words make sure the logger knows its files are closed   This assumes when you daemonise you are closing ALL the open file descriptors indiscriminatingly - instead you could try closing all but the log files  but it s usually simpler to close all then reopen the ones you want

User · Answer

Note the python-daemon package which solves a lot of problems behind daemons out of the box.

Among other features it enables to (from Debian package description):

Detach the process into its own process group.
Set process environment appropriate for running inside a chroot.
Renounce suid and sgid privileges.
Close all open file descriptors.
Change the working directory, uid, gid, and umask.
Set appropriate signal handlers.
Open new file descriptors for stdin, stdout, and stderr.
Manage a specified PID lock file.
Register cleanup functions for at-exit processing.

User · Answer

since python-daemon has not yet supported python 3 x  and from what can be read on the mailing list  it may never will  i have written a new implementation of PEP 3143  pep3143daemon  pep3143daemon should support at least python 2 6  2 7 and 3 x  It also contains a PidFile class   The library only depends on the standard library and on the six module   It can be used as a drop in replacement for python-daemon   Here is the documentation

User · Answer

An alternative -- create a normal  non-daemonized Python program then externally daemonize it using supervisord   This can save a lot of headaches  and is  nix- and language-portable

User · Answer

The easiest way to create daemon with Python is to use the Twisted event-driven framework   It handles all of the stuff necessary for daemonization for you   It uses the Reactor Pattern to handle concurrent requests

User · Answer

Probably not a direct answer to the question  but systemd can be used to run your application as a daemon   Here is an example    Unit  Description Python daemon After syslog target After network target   Service  Type simple User  lt run as user gt  Group  lt run as group group gt  ExecStart  usr bin python  lt python script home gt  script py    Give the script some time to startup TimeoutSec 300   Install  WantedBy multi-user target   I prefer this method because a lot of the work is done for you  and then your daemon script behaves similarly to the rest of your system   -Orby

User · Answer

There are many fiddly things to take care of when becoming a well-behaved daemon process   prevent core dumps  many daemons run as root  and core dumps can contain sensitive information   behave correctly inside a chroot gaol  set UID  GID  working directory  umask  and other process parameters appropriately for the use case  relinquish elevated suid  sgid privileges  close all open file descriptors  with exclusions depending on the use case  behave correctly if started inside an already-detached context  such as init  inetd  etc   set up signal handlers for sensible daemon behaviour  but also with specific handlers determined by the use case  redirect the standard streams stdin  stdout  stderr since a daemon process no longer has a controlling terminal  handle a PID file as a cooperative advisory lock  which is a whole can of worms in itself with many contradictory but valid ways to behave  allow proper cleanup when the process is terminated  actually become a daemon process without leading to zombies   Some of these are standard  as described in canonical Unix literature  Advanced Programming in the UNIX Environment  by the late W  Richard Stevens  Addison-Wesley  1992   Others  such as stream redirection and PID file handling  are conventional behaviour most daemon users would expect but that are less standardised  All of these are covered by the PEP 3143    Standard daemon process library    specification  The python-daemon reference implementation works on Python 2 7 or later  and Python 3 2 or later

User · Answer

The easiest way to create daemon with Python is to use the Twisted event-driven framework   It handles all of the stuff necessary for daemonization for you   It uses the Reactor Pattern to handle concurrent requests

User · Answer

Though you may prefer the pure Python solution provided by the python-daemon module  there is a daemon 3  function in libc -- at least  on BSD and Linux -- which will do the right thing   Calling it from python is easy   import ctypes  ctypes CDLL None  daemon 0  0    Read the man-page for the arguments  meanings   The only remaining thing to do is creation  and locking  of the PID-file  But that you can handle yourself

User · Answer

I modified a few lines in Sander Marechal s code sample  mentioned by  JeffBauer in the accepted answer  to add a quit   method that gets executed before the daemon is stopped  This is sometimes very useful   Here it is   Note  I don t use the  python-daemon  module because the documentation is still missing  see also many other SO questions  and is rather obscure  how to start stop properly a daemon from command line with this module

User · Answer

80  of the time  when folks say  daemon   they only want a server   Since the question is perfectly unclear on this point  it s hard to say what the possible domain of answers could be   Since a server is adequate  start there   If an actual  daemon  is actually needed  this is rare   read up on nohup as a way to daemonize a server   Until such time as an actual daemon is actually required  just write a simple server   Also look at the WSGI reference implementation   Also look at the Simple HTTP Server    Are there any additional things that need to be considered     Yes   About a million things   What protocol   How many requests   How long to service each request   How frequently will they arrive   Will you use a dedicated process   Threads   Subprocesses   Writing a daemon is a big job

User · Answer

Current solution  A reference implementation of PEP 3143  Standard daemon process library  is now available as python-daemon   Historical answer  Sander Marechal s code sample is superior to the original  which was originally posted in 2004   I once contributed a daemonizer for Pyro  but would probably use Sander s code if I had to do it over

User · Answer

The easiest way to create daemon with Python is to use the Twisted event-driven framework   It handles all of the stuff necessary for daemonization for you   It uses the Reactor Pattern to handle concurrent requests

User · Answer

Here s my basic  Howdy World  Python daemon that I start with  when I m developing a new daemon application      usr bin python import time from daemon import runner  class App        def   init   self           self stdin path     dev null          self stdout path     dev tty          self stderr path     dev tty          self pidfile path      tmp foo pid          self pidfile timeout   5     def run self           while True              print  Howdy   Gig em   Whoop                time sleep 10   app   App   daemon runner   runner DaemonRunner app  daemon runner do action     Note that you ll need the python-daemon library   You can install it by   pip install python-daemon   Then just start it with   howdy py start  and stop it with   howdy py stop

User · Answer

I am afraid the daemon module mentioned by  Dustin didn t work for me  Instead I installed python-daemon and used the following code     filename myDaemon py import sys import daemon sys path append   home ubuntu samplemodule     till   init   py from samplemodule import moduleclass   with daemon DaemonContext        moduleclass do running     I have do running   function and whatever I was doing in   main     in module py I copied in it    Running is easy   gt  python myDaemon py   just for completeness here is samplemodule directory content   gt ls samplemodule   init   py   init   pyc moduleclass py   The content of moduleclass py can be  class moduleclass             def do running        m   moduleclass         do whatever daemon is required to do

User · Answer

This function will transform an application to a daemon   import sys import os  def daemonize        try          pid   os fork           if pid  gt  0                exit first parent             sys exit 0      except OSError as err          sys stderr write   Fork  1 failed   0  n  format err           sys exit 1        decouple from parent environment     os chdir          os setsid       os umask 0        do second fork     try          pid   os fork           if pid  gt  0                exit from second parent             sys exit 0      except OSError as err          sys stderr write   Fork  2 failed   0  n  format err           sys exit 1        redirect standard file descriptors     sys stdout flush       sys stderr flush       si   open os devnull   r       so   open os devnull   w       se   open os devnull   w       os dup2 si fileno    sys stdin fileno        os dup2 so fileno    sys stdout fileno        os dup2 se fileno    sys stderr fileno

User · Answer

since python-daemon has not yet supported python 3 x  and from what can be read on the mailing list  it may never will  i have written a new implementation of PEP 3143  pep3143daemon  pep3143daemon should support at least python 2 6  2 7 and 3 x  It also contains a PidFile class   The library only depends on the standard library and on the six module   It can be used as a drop in replacement for python-daemon   Here is the documentation

User · Answer

Current solution  A reference implementation of PEP 3143  Standard daemon process library  is now available as python-daemon   Historical answer  Sander Marechal s code sample is superior to the original  which was originally posted in 2004   I once contributed a daemonizer for Pyro  but would probably use Sander s code if I had to do it over

User · Answer

I am afraid the daemon module mentioned by  Dustin didn t work for me  Instead I installed python-daemon and used the following code     filename myDaemon py import sys import daemon sys path append   home ubuntu samplemodule     till   init   py from samplemodule import moduleclass   with daemon DaemonContext        moduleclass do running     I have do running   function and whatever I was doing in   main     in module py I copied in it    Running is easy   gt  python myDaemon py   just for completeness here is samplemodule directory content   gt ls samplemodule   init   py   init   pyc moduleclass py   The content of moduleclass py can be  class moduleclass             def do running        m   moduleclass         do whatever daemon is required to do

User · Answer

Current solution  A reference implementation of PEP 3143  Standard daemon process library  is now available as python-daemon   Historical answer  Sander Marechal s code sample is superior to the original  which was originally posted in 2004   I once contributed a daemonizer for Pyro  but would probably use Sander s code if I had to do it over

User · Answer

YapDi is a relatively new python module that popped up in Hacker News  Looks pretty useful  can be used to convert a python script into daemon mode from inside the script

User · Answer

There are many fiddly things to take care of when becoming a well-behaved daemon process   prevent core dumps  many daemons run as root  and core dumps can contain sensitive information   behave correctly inside a chroot gaol  set UID  GID  working directory  umask  and other process parameters appropriately for the use case  relinquish elevated suid  sgid privileges  close all open file descriptors  with exclusions depending on the use case  behave correctly if started inside an already-detached context  such as init  inetd  etc   set up signal handlers for sensible daemon behaviour  but also with specific handlers determined by the use case  redirect the standard streams stdin  stdout  stderr since a daemon process no longer has a controlling terminal  handle a PID file as a cooperative advisory lock  which is a whole can of worms in itself with many contradictory but valid ways to behave  allow proper cleanup when the process is terminated  actually become a daemon process without leading to zombies   Some of these are standard  as described in canonical Unix literature  Advanced Programming in the UNIX Environment  by the late W  Richard Stevens  Addison-Wesley  1992   Others  such as stream redirection and PID file handling  are conventional behaviour most daemon users would expect but that are less standardised  All of these are covered by the PEP 3143    Standard daemon process library    specification  The python-daemon reference implementation works on Python 2 7 or later  and Python 3 2 or later

User · Answer

After a few years and many attempts  I tried all the answers given here  but all of them had minor drawbacks at the end   now I realize that there is a better way than wanting to start  stop  restart a daemon directly from Python  use the OS tools instead   For example  for Linux  instead of doing python myapp start and python myapp stop  I do this to start the app   screen -S myapp python myapp py     CTRL A  D to detach   or screen -dmS myapp python myapp py to start and detach it in one command   Then   screen -r myapp   to attach to this terminal again  Once in the terminal  it s possible to use CTRL C to stop it

User · Answer

80  of the time  when folks say  daemon   they only want a server   Since the question is perfectly unclear on this point  it s hard to say what the possible domain of answers could be   Since a server is adequate  start there   If an actual  daemon  is actually needed  this is rare   read up on nohup as a way to daemonize a server   Until such time as an actual daemon is actually required  just write a simple server   Also look at the WSGI reference implementation   Also look at the Simple HTTP Server    Are there any additional things that need to be considered     Yes   About a million things   What protocol   How many requests   How long to service each request   How frequently will they arrive   Will you use a dedicated process   Threads   Subprocesses   Writing a daemon is a big job

User · Answer

Probably not a direct answer to the question  but systemd can be used to run your application as a daemon   Here is an example    Unit  Description Python daemon After syslog target After network target   Service  Type simple User  lt run as user gt  Group  lt run as group group gt  ExecStart  usr bin python  lt python script home gt  script py    Give the script some time to startup TimeoutSec 300   Install  WantedBy multi-user target   I prefer this method because a lot of the work is done for you  and then your daemon script behaves similarly to the rest of your system   -Orby

User · Answer

After a few years and many attempts  I tried all the answers given here  but all of them had minor drawbacks at the end   now I realize that there is a better way than wanting to start  stop  restart a daemon directly from Python  use the OS tools instead   For example  for Linux  instead of doing python myapp start and python myapp stop  I do this to start the app   screen -S myapp python myapp py     CTRL A  D to detach   or screen -dmS myapp python myapp py to start and detach it in one command   Then   screen -r myapp   to attach to this terminal again  Once in the terminal  it s possible to use CTRL C to stop it

User · Answer

I modified a few lines in Sander Marechal s code sample  mentioned by  JeffBauer in the accepted answer  to add a quit   method that gets executed before the daemon is stopped  This is sometimes very useful   Here it is   Note  I don t use the  python-daemon  module because the documentation is still missing  see also many other SO questions  and is rather obscure  how to start stop properly a daemon from command line with this module

User · Answer

Though you may prefer the pure Python solution provided by the python-daemon module  there is a daemon 3  function in libc -- at least  on BSD and Linux -- which will do the right thing   Calling it from python is easy   import ctypes  ctypes CDLL None  daemon 0  0    Read the man-page for the arguments  meanings   The only remaining thing to do is creation  and locking  of the PID-file  But that you can handle yourself

User · Answer

An alternative -- create a normal  non-daemonized Python program then externally daemonize it using supervisord   This can save a lot of headaches  and is  nix- and language-portable

User · Answer

Current solution  A reference implementation of PEP 3143  Standard daemon process library  is now available as python-daemon   Historical answer  Sander Marechal s code sample is superior to the original  which was originally posted in 2004   I once contributed a daemonizer for Pyro  but would probably use Sander s code if I had to do it over

[python] How do you create a daemon in Python?

Examples related to python

Examples related to daemon