Set up a scheduled job

Question

I ve been working on a web app using Django  and I m curious if there is a way to schedule a job to run periodically    Basically I just want to run through the database and make some calculations updates on an automatic  regular basis  but I can t seem to find any documentation on doing this   Does anyone know how to set this up   To clarify   I know I can set up a cron job to do this  but I m curious if there is some feature in Django that provides this functionality    I d like people to be able to deploy this app themselves without having to do much config  preferably zero    I ve considered triggering these actions  retroactively  by simply checking if a job should have been run since the last time a request was sent to the site  but I m hoping for something a bit cleaner

User · Answer

We ve open-sourced what I think is a structured app  that Brian s solution above alludes too  We would love any   all feedback   https   github com tivix django-cron  It comes with one management command     manage py runcrons   That does the job  Each cron is modeled as a class  so its all OO  and each cron runs at a different frequency and we make sure the same cron type doesn t run in parallel  in case crons themselves take longer time to run than their frequency

User · Answer

Interesting new pluggable Django app  django-chronograph  You only have to add one cron entry which acts as a timer  and you have a very nice Django admin interface into the scripts to run

User · Answer

Look at Django Poor Man s Cron which is a Django app that makes use of spambots  search engine indexing robots and alike to run scheduled tasks in approximately regular intervals  See  http   code google com p django-poormanscron

User · Answer

Brian Neal s suggestion of running management commands via cron works well  but if you re looking for something a little more robust  yet not as elaborate as Celery  I d look into a library like Kronos     app cron py  import kronos   kronos register  0           def task        pass

User · Answer

I had exactly the same requirement a while ago  and ended up solving it using APScheduler  User Guide   It makes scheduling jobs super simple  and keeps it independent for from request-based execution of some code  Following is a simple example   from apscheduler schedulers background import BackgroundScheduler  scheduler   BackgroundScheduler   job   None  def tick        print  One tick      def start job        global job     job   scheduler add job tick   interval   seconds 3600      try          scheduler start       except          pass   Hope this helps somebody

User · Answer

One solution that I have employed is to do this   1  Create a custom management command  e g    python manage py my cool command   2  Use cron  on Linux  or at  on Windows  to run my command at the required times   This is a simple solution that doesn t require installing a heavy AMQP stack  However there are nice advantages to using something like Celery  mentioned in the other answers  In particular  with Celery it is nice to not have to spread your application logic out into crontab files  However the cron solution works quite nicely for a small to medium sized application and where you don t want a lot of external dependencies   EDIT   In later version of windows the at command is deprecated for Windows 8  Server 2012 and above  You can use schtasks exe for same use        UPDATE      This the new link of django doc for writing the custom management command

User · Answer

Simple way is to write a custom shell command see Django Documentation and execute it using a cronjob on linux  However i would highly recommend using a message broker like RabbitMQ coupled with celery  Maybe you can have a look at this Tutorial

User · Answer

I just thought about this rather simple solution    Define a view function do work req  param  like you would with any other view  with URL mapping  return a HttpResponse and so on  Set up a cron job with your timing preferences  or using AT or Scheduled Tasks in Windows  which runs curl http   localhost your mapped url param value    You can add parameters but just adding parameters to the URL   Tell me what you guys think    Update  I m now using runjob command from django-extensions instead of curl   My cron looks something like this    hourly python  path to project manage py runjobs hourly       and so on for daily  monthly  etc   You can also set it up to run a specific job   I find it more managable and a cleaner  Doesn t require mapping a URL to a view  Just define your job class and crontab and you re set

User · Answer

after the part of code I can write anything just like my views py                                             import os sys sys path append   home administrator development store   os environ  DJANGO SETTINGS MODULE    store settings  from django core management impor setup environ from store import settings setup environ settings                                            from http   www cotellese net 2007 09 27 running-external-scripts-against-django-models

User · Answer

If you want something more reliable than Celery  try TaskHawk which is built on top of AWS SQS SNS   Refer  http   taskhawk readthedocs io

User · Answer

Yes  the method above is so great  And I tried some of them  At last  I found a method like this       from threading import Timer      def sync             do something             sync timer   Timer self interval  sync              sync timer start     Just like Recursive   Ok  I hope this method can meet your requirement

User · Answer

Put the following at the top of your cron py file      usr bin python import os  sys sys path append   path to      the parent directory of the project sys path append   path to project     these lines only needed if not on path os environ  DJANGO SETTINGS MODULE      myproj settings     imports and code below

User · Answer

For simple dockerized projects  I could not really see any existing answer fit   So I wrote a very barebones solution without the need of external libraries or triggers  which runs on its own  No external os-cron needed  should work in every environment   It works by adding a middleware  middleware py    import threading  def should run name  seconds interval       from application models import CronJob     from django utils timezone import now      try          c   CronJob objects get name name      except CronJob DoesNotExist          CronJob name name  last ran now    save           return True      if  now   - c last ran  total seconds    gt   seconds interval          c last ran   now           c save           return True      return False   class CronTask      def   init   self  name  seconds interval  function           self name   name         self seconds interval   seconds interval         self function   function   def cron worker          if not should run  main   60           return        customize this part      from application models import Event     tasks             CronTask  events   60   30  Event clean stale objects                            for task in tasks          if should run task name  task seconds interval               task function     def cron middleware get response        def middleware request           response   get response request          threading Thread target cron worker  start           return response      return middleware   models cron py   from django db import models   class CronJob models Model       name   models CharField max length 10  primary key True      last ran   models DateTimeField     settings py   MIDDLEWARE                  application middleware cron middleware

User · Answer

Celery is a distributed task queue  built on AMQP  RabbitMQ   It also handles periodic tasks in a cron-like fashion  see periodic tasks   Depending on your app  it might be worth a gander   Celery is pretty easy to set up with django  docs   and periodic tasks will actually skip missed tasks in case of a downtime  Celery also has built-in retry mechanisms  in case a task fails

User · Answer

Although not part of Django  Airflow is a more recent project  as of 2016  that is useful for task management   Airflow is a workflow automation and scheduling system that can be used to author and manage data pipelines  A web-based UI provides the developer with a range of options for managing and viewing these pipelines   Airflow is written in Python and is built using Flask   Airflow was created by Maxime Beauchemin at Airbnb and open sourced in the spring of 2015   It joined the Apache Software Foundation   s incubation program in the winter of 2016   Here is the Git project page and some addition background information

User · Answer

A more modern solution  compared to Celery  is Django Q  https   django-q readthedocs io en latest index html  It has great documentation and is easy to grok  Windows support is lacking  because Windows does not support process forking  But it works fine if you create your dev environment using the Windows for Linux Subsystem

User · Answer

I use celery to create my periodical tasks  First you need to install it as follows   pip install django-celery   Don t forget to register django-celery in your settings and then you could do something like this   from celery import task from celery decorators import periodic task from celery task schedules import crontab from celery utils log import get task logger  periodic task run every crontab minute  0   hour  23    def do every midnight      your code

User · Answer

If you re using a standard POSIX OS  you use cron   If you re using Windows  you use at   Write a Django management command to    Figure out what platform they re on  Either execute the appropriate  AT  command for your users  or update the crontab for your users

User · Answer

I am not sure will this be useful for anyone  since I had to provide other users of the system to schedule the jobs  without giving them access to the actual server windows  Task Scheduler  I created this reusable app   Please note users have access to one shared folder on server where they can create required command task  bat file  This task then can be scheduled using this app   App name is Django Windows Scheduler  ScreenShot

User · Answer

Django APScheduler for Scheduler Jobs  Advanced Python Scheduler  APScheduler  is a Python library that lets you schedule your Python code to be executed later  either just once or periodically  You can add new jobs or remove old ones on the fly as you please   note  I m the author of this library  Install APScheduler  pip install apscheduler   View file function to call  file name  scheduler jobs py  def FirstCronTest        print         print  I am executed        Configuring the scheduler  make execute py file and add the below codes  from apscheduler schedulers background import BackgroundScheduler scheduler   BackgroundScheduler     Your written functions Here  the scheduler functions are written in scheduler jobs  import scheduler jobs   scheduler add job scheduler jobs FirstCronTest   interval   seconds 10  scheduler start     Link the File for Execution  Now  add the below line in the bottom of Url file  import execute    You can check the full code by executing  Click here   https   github com devchandansh django-apscheduler

User · Answer

You should definitely check out django-q  It requires no additional configuration and has quite possibly everything needed to handle any production issues on commercial projects   It s actively developed and integrates very well with django  django ORM  mongo  redis  Here is my configuration     django-q   -------------------------------------------------------------------------   See  http   django-q readthedocs io en latest configure html Q CLUSTER           Match recommended settings from docs       name    DjangoORM        workers   4       queue limit   50       bulk   10       orm    default      Custom Settings   ---------------   Limit the amount of successful tasks saved to Django   save limit   10000     See https   github com Koed00 django-q issues 110   catch up   False     Number of seconds a worker can spend on a task before it s terminated   timeout   60   5     Number of seconds a broker will wait for a cluster to finish a task before presenting it again  This needs to be   longer than  timeout   otherwise the same task will be processed multiple times   retry   60   6     Whether to force all async   calls to be run with sync True  making them synchronous    sync   False     Redirect worker exceptions directly to Sentry error reporter   error reporter          sentry   RAVEN CONFIG

User · Answer

RabbitMQ and Celery have more features and task handling capabilities than Cron  If task failure isn t an issue  and you think you will handle broken tasks in the next call  then Cron is sufficient   Celery  amp  AMQP will let you handle the broken task  and it will get executed again by another worker  Celery workers listen for the next task to work on   until the task s max retries attribute is reached  You can even invoke tasks on failure  like logging the failure  or sending an email to the admin once the max retries has been reached   And you can distribute Celery and AMQP servers when you need to scale your application

User · Answer

I had something similar with your problem today   I didn t wanted to have it handled by the server trhough cron  and most of the libs were just cron helpers in the end    So i ve created a scheduling module and attached it to the init    It s not the best approach  but it helps me to have all the code in a single place and with its execution related to the main app

User · Answer

I personally use cron  but the Jobs Scheduling parts of django-extensions looks interesting

[python] Set up a scheduled job?

Examples related to python

Examples related to django

Examples related to web-applications

Examples related to scheduled-tasks