[python] How to make a Python script run like a service or daemon in Linux

I have written a Python script that checks a certain e-mail address and passes new e-mails to an external program. How can I get this script to execute 24/7, such as turning it into daemon or service in Linux. Would I also need a loop that never ends in the program, or can it be done by just having the code re executed multiple times?

Here's a nice class that is taken from here:

#!/usr/bin/env python

import sys, os, time, atexit
from signal import SIGTERM

class Daemon:
        """
        A generic daemon class.

        Usage: subclass the Daemon class and override the run() method
        """
        def __init__(self, pidfile, stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'):
                self.stdin = stdin
                self.stdout = stdout
                self.stderr = stderr
                self.pidfile = pidfile

        def daemonize(self):
                """
                do the UNIX double-fork magic, see Stevens' "Advanced
                Programming in the UNIX Environment" for details (ISBN 0201563177)
                http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16
                """
                try:
                        pid = os.fork()
                        if pid > 0:
                                # exit first parent
                                sys.exit(0)
                except OSError, e:
                        sys.stderr.write("fork #1 failed: %d (%s)\n" % (e.errno, e.strerror))
                        sys.exit(1)

                # decouple from parent environment
                os.chdir("/")
                os.setsid()
                os.umask(0)

                # do second fork
                try:
                        pid = os.fork()
                        if pid > 0:
                                # exit from second parent
                                sys.exit(0)
                except OSError, e:
                        sys.stderr.write("fork #2 failed: %d (%s)\n" % (e.errno, e.strerror))
                        sys.exit(1)

                # redirect standard file descriptors
                sys.stdout.flush()
                sys.stderr.flush()
                si = file(self.stdin, 'r')
                so = file(self.stdout, 'a+')
                se = file(self.stderr, 'a+', 0)
                os.dup2(si.fileno(), sys.stdin.fileno())
                os.dup2(so.fileno(), sys.stdout.fileno())
                os.dup2(se.fileno(), sys.stderr.fileno())

                # write pidfile
                atexit.register(self.delpid)
                pid = str(os.getpid())
                file(self.pidfile,'w+').write("%s\n" % pid)

        def delpid(self):
                os.remove(self.pidfile)

        def start(self):
                """
                Start the daemon
                """
                # Check for a pidfile to see if the daemon already runs
                try:
                        pf = file(self.pidfile,'r')
                        pid = int(pf.read().strip())
                        pf.close()
                except IOError:
                        pid = None

                if pid:
                        message = "pidfile %s already exist. Daemon already running?\n"
                        sys.stderr.write(message % self.pidfile)
                        sys.exit(1)

                # Start the daemon
                self.daemonize()
                self.run()

        def stop(self):
                """
                Stop the daemon
                """
                # Get the pid from the pidfile
                try:
                        pf = file(self.pidfile,'r')
                        pid = int(pf.read().strip())
                        pf.close()
                except IOError:
                        pid = None

                if not pid:
                        message = "pidfile %s does not exist. Daemon not running?\n"
                        sys.stderr.write(message % self.pidfile)
                        return # not an error in a restart

                # Try killing the daemon process       
                try:
                        while 1:
                                os.kill(pid, SIGTERM)
                                time.sleep(0.1)
                except OSError, err:
                        err = str(err)
                        if err.find("No such process") > 0:
                                if os.path.exists(self.pidfile):
                                        os.remove(self.pidfile)
                        else:
                                print str(err)
                                sys.exit(1)

        def restart(self):
                """
                Restart the daemon
                """
                self.stop()
                self.start()

        def run(self):
                """
                You should override this method when you subclass Daemon. It will be called after the process has been
                daemonized by start() or restart().
                """

cron is clearly a great choice for many purposes. However it doesn't create a service or daemon as you requested in the OP. cron just runs jobs periodically (meaning the job starts and stops), and no more often than once / minute. There are issues with cron -- for example, if a prior instance of your script is still running the next time the cron schedule comes around and launches a new instance, is that OK? cron doesn't handle dependencies; it just tries to start a job when the schedule says to.

If you find a situation where you truly need a daemon (a process that never stops running), take a look at supervisord. It provides a simple way to wrapper a normal, non-daemonized script or program and make it operate like a daemon. This is a much better way than creating a native Python daemon.

Assuming that you would really want your loop to run 24/7 as a background service

For a solution that doesn't involve injecting your code with libraries, you can simply create a service template, since you are using linux:

[Unit]
Description = <Your service description here>
After = network.target # Assuming you want to start after network interfaces are made available
 
[Service]
Type = simple
ExecStart = python <Path of the script you want to run>
User = # User to run the script as
Group = # Group to run the script as
Restart = on-failure # Restart when there are errors
SyslogIdentifier = <Name of logs for the service>
RestartSec = 5
TimeoutStartSec = infinity
 
[Install]
WantedBy = multi-user.target # Make it accessible to other users

Place that file in your daemon service folder (usually /etc/systemd/system/), in a *.service file, and install it using the following systemctl commands (will likely require sudo privileges):

systemctl enable <service file name without .service extension>

systemctl daemon-reload

systemctl start <service file name without .service extension>

You can then check that your service is running by using the command:

systemctl | grep running

Use whatever service manager your system offers - for example under Ubuntu use upstart. This will handle all the details for you such as start on boot, restart on crash, etc.

I would recommend this solution. You need to inherit and override method run.

import sys
import os
from signal import SIGTERM
from abc import ABCMeta, abstractmethod



class Daemon(object):
    __metaclass__ = ABCMeta


    def __init__(self, pidfile):
        self._pidfile = pidfile


    @abstractmethod
    def run(self):
        pass


    def _daemonize(self):
        # decouple threads
        pid = os.fork()

        # stop first thread
        if pid > 0:
            sys.exit(0)

        # write pid into a pidfile
        with open(self._pidfile, 'w') as f:
            print >> f, os.getpid()


    def start(self):
        # if daemon is started throw an error
        if os.path.exists(self._pidfile):
            raise Exception("Daemon is already started")

        # create and switch to daemon thread
        self._daemonize()

        # run the body of the daemon
        self.run()


    def stop(self):
        # check the pidfile existing
        if os.path.exists(self._pidfile):
            # read pid from the file
            with open(self._pidfile, 'r') as f:
                pid = int(f.read().strip())

            # remove the pidfile
            os.remove(self._pidfile)

            # kill daemon
            os.kill(pid, SIGTERM)

        else:
            raise Exception("Daemon is not started")


    def restart(self):
        self.stop()
        self.start()

You can also make the python script run as a service using a shell script. First create a shell script to run the python script like this (scriptname arbitary name)

#!/bin/sh
script='/home/.. full path to script'
/usr/bin/python $script &

now make a file in /etc/init.d/scriptname

#! /bin/sh

PATH=/bin:/usr/bin:/sbin:/usr/sbin
DAEMON=/home/.. path to shell script scriptname created to run python script
PIDFILE=/var/run/scriptname.pid

test -x $DAEMON || exit 0

. /lib/lsb/init-functions

case "$1" in
  start)
     log_daemon_msg "Starting feedparser"
     start_daemon -p $PIDFILE $DAEMON
     log_end_msg $?
   ;;
  stop)
     log_daemon_msg "Stopping feedparser"
     killproc -p $PIDFILE $DAEMON
     PID=`ps x |grep feed | head -1 | awk '{print $1}'`
     kill -9 $PID       
     log_end_msg $?
   ;;
  force-reload|restart)
     $0 stop
     $0 start
   ;;
  status)
     status_of_proc -p $PIDFILE $DAEMON atd && exit 0 || exit $?
   ;;
 *)
   echo "Usage: /etc/init.d/atd {start|stop|restart|force-reload|status}"
   exit 1
  ;;
esac

exit 0

Now you can start and stop your python script using the command /etc/init.d/scriptname start or stop.

You can use fork() to detach your script from the tty and have it continue to run, like so:

import os, sys
fpid = os.fork()
if fpid!=0:
  # Running as daemon now. PID is fpid
  sys.exit(0)

Of course you also need to implement an endless loop, like

while 1:
  do_your_check()
  sleep(5)

Hope this get's you started.

A simple and supported version is Daemonize.

Install it from Python Package Index (PyPI):

$ pip install daemonize

and then use like:

...
import os, sys
from daemonize import Daemonize
...
def main()
      # your code here

if __name__ == '__main__':
        myname=os.path.basename(sys.argv[0])
        pidfile='/tmp/%s' % myname       # any name
        daemon = Daemonize(app=myname,pid=pidfile, action=main)
        daemon.start()

how about using $nohup command on linux?

I use it for running my commands on my Bluehost server.

Please advice if I am wrong.

Ubuntu has a very simple way to manage a service. For python the difference is that ALL the dependencies (packages) have to be in the same directory, where the main file is run from.

I just manage to create such a service to provide weather info to my clients. Steps:

• Create your python application project as you normally do.

• Install all dependencies locally like: sudo pip3 install package_name -t .

• Create your command line variables and handle them in code (if you need any)

• Create the service file. Something (minimalist) like:

    [Unit]
    Description=1Droid Weather meddleware provider
  
    [Service]
    Restart=always
    User=root
    WorkingDirectory=/home/ubuntu/weather
    ExecStart=/usr/bin/python3 /home/ubuntu/weather/main.py httpport=9570  provider=OWMap
  
    [Install]
    WantedBy=multi-user.target
  

• Save the file as myweather.service (for example)

• Make sure that your app runs if started in the current directory

    python3  main.py httpport=9570  provider=OWMap
  

• The service file produced above and named myweather.service (important to have the extension .service) will be treated by the system as the name of your service. That is the name that you will use to interact with your service.

• Copy the service file:

    sudo cp myweather.service /lib/systemd/system/myweather.service
  

• Refresh demon registry:

    sudo systemctl daemon-reload
  

• Stop the service (if it was running)

    sudo service myweatherr stop
  

• Start the service:

    sudo service myweather start
  

• Check the status (log file with where your print statements go):

    tail -f /var/log/syslog
  

• Or check the status with:

    sudo service myweather status
  

• Back to the start with another iteration if needed

This service is now running and even if you log out it will not be affected. And YES if the host is shutdown and restarted this service will be restarted...information for my mobile android app...

You should use the python-daemon library, it takes care of everything.

From PyPI: Library to implement a well-behaved Unix daemon process.

to creating some thing that is running like service you can use this thing :

The first thing that you must do is installing the Cement framework: Cement frame work is a CLI frame work that you can deploy your application on it.

command line interface of the app :

interface.py

 from cement.core.foundation import CementApp
 from cement.core.controller import CementBaseController, expose
 from YourApp import yourApp

 class Meta:
    label = 'base'
    description = "your application description"
    arguments = [
        (['-r' , '--run'],
          dict(action='store_true', help='Run your application')),
        (['-v', '--version'],
          dict(action='version', version="Your app version")),
        ]
        (['-s', '--stop'],
          dict(action='store_true', help="Stop your application")),
        ]

    @expose(hide=True)
    def default(self):
        if self.app.pargs.run:
            #Start to running the your app from there !
            YourApp.yourApp()
        if self.app.pargs.stop:
            #Stop your application
            YourApp.yourApp.stop()

 class App(CementApp):
       class Meta:
       label = 'Uptime'
       base_controller = 'base'
       handlers = [MyBaseController]

 with App() as app:
       app.run()

YourApp.py class:

 import threading

 class yourApp:
     def __init__:
        self.loger = log_exception.exception_loger()
        thread = threading.Thread(target=self.start, args=())
        thread.daemon = True
        thread.start()

     def start(self):
        #Do every thing you want
        pass
     def stop(self):
        #Do some things to stop your application

Keep in mind that your app must run on a thread to be daemon

To run the app just do this in command line

python interface.py --help

If you are using terminal(ssh or something) and you want to keep a long-time script working after you log out from the terminal, you can try this:

screen

apt-get install screen

create a virtual terminal inside( namely abc): screen -dmS abc

now we connect to abc: screen -r abc

So, now we can run python script: python keep_sending_mails.py

from now on, you can directly close your terminal, however, the python script will keep running rather than being shut down

Since this keep_sending_mails.py's PID is a child process of the virtual screen rather than the terminal(ssh)

If you want to go back check your script running status, you can use screen -r abc again

First, read up on mail aliases. A mail alias will do this inside the mail system without you having to fool around with daemons or services or anything of the sort.

You can write a simple script that will be executed by sendmail each time a mail message is sent to a specific mailbox.

See http://www.feep.net/sendmail/tutorial/intro/aliases.html

If you really want to write a needlessly complex server, you can do this.

nohup python myscript.py &

That's all it takes. Your script simply loops and sleeps.

import time
def do_the_work():
    # one round of polling -- checking email, whatever.
while True:
    time.sleep( 600 ) # 10 min.
    try:
        do_the_work()
    except:
        pass