How to run a cron job inside a docker container

Question

I am trying to run a cronjob inside a docker container that invokes a shell script   Yesterday I have been searching all over the web and stack overflow  but I could not really find a solution that works  How can I do this   EDIT   I ve created a  commented  github repository with a working docker cron container that invokes a shell script at given interval

User · Answer

If you re using docker for windows  remember that you have to change your line-ending format from CRLF to LF  i e  from dos to unix  if you intend on importing your crontab file from windows to your ubuntu container  If not  your cron-job won t work  Here s a working example   FROM ubuntu latest  RUN apt-get update  amp  amp  apt-get -y install cron RUN apt-get update  amp  amp  apt-get install -y dos2unix    Add crontab file  from your windows host  to the cron directory ADD cron hello-cron  etc cron d hello-cron    Change line ending format to LF RUN dos2unix  etc cron d hello-cron    Give execution rights on the cron job RUN chmod 0644  etc cron d hello-cron    Apply cron job RUN crontab  etc cron d hello-cron    Create the log file to be able to run tail RUN touch  var log hello-cron log    Run the command on container startup CMD cron  amp  amp  tail -f  var log hello-cron log   This actually took me hours to figure out  as debugging cron jobs in docker containers is a tedious task  Hope it helps anyone else out there that can t get their code to work

User · Answer

For those who wants to use a simple and lightweight image   FROM alpine 3 6    copy crontabs for root user COPY config cronjobs  etc crontabs root    start crond with log level 8 in foreground  output to stderr CMD   crond    -f    -d    8     Where cronjobs is the file that contains your cronjobs  in this form             echo  hello stackoverflow   gt  gt   test file 2 gt  amp 1   remember to end this file with an empty new line

User · Answer

When running on some trimmed down images that restrict root access  I had to add my user to the sudoers and run as sudo cron   x000D   x000D  FROM node 8 6 0 x000D  RUN apt-get update  amp  amp  apt-get install -y cron sudo x000D   x000D  COPY crontab  etc cron d my-cron x000D  RUN chmod 0644  etc cron d my-cron x000D  RUN touch  var log cron log x000D   x000D    Allow node user to start cron daemon with sudo x000D  RUN echo  node ALL NOPASSWD   usr sbin cron   gt  gt  etc sudoers x000D   x000D  ENTRYPOINT sudo cron  amp  amp  tail -f  var log cron log x000D   x000D   x000D    Maybe that helps someone

User · Answer

So  my problem was the same  The fix was to change the command section in the docker-compose yml   From  command  crontab  etc crontab  amp  amp  tail -f  etc crontab  To  command  crontab  etc crontab  command  tail -f  etc crontab  The problem was the   amp  amp   between the commands  After deleting this  it was all fine

User · Answer

There is another way to do it  is to use Tasker  a task runner that has cron  a scheduler  support   Why   Sometimes to run a cron job  you have to mix  your base image  python  java  nodejs  ruby  with the crond  That means another image to maintain  Tasker avoid that by decoupling the crond and you container  You can just focus on the image that you want to execute your commands  and configure Tasker to use it   Here an docker-compose yml file  that will run some tasks for you  version   2   services      tasker          image  strm tasker         volumes              -   var run docker sock  var run docker sock          environment              configuration                    logging                      level                          ROOT  WARN                         org springframework web  WARN                         sh strm  DEBUG                 schedule                      - every  minute                       task  hello                     - every  minute                       task  helloFromPython                     - every  minute                       task  helloFromNode                 tasks                      docker                          - name  hello                           image  debian jessie                           script                                - echo Hello world from Tasker                         - name  helloFromPython                           image  python 3-slim                           script                                - python -c  print  Hello world from python                            - name  helloFromNode                           image  node 8                           script                                - node -e  console log  Hello from node      There are 3 tasks there  all of them will run every minute  every  minute   and each of them will execute the script code  inside the image defined in image section   Just run docker-compose up  and see it working  Here is the Tasker repo with the full documentation   http   github com opsxcq tasker

User · Answer

You can copy your crontab into an image  in order for the container launched from said image to run the job  See  quot Run a cron job with Docker quot  from Julien Boulay in his Ekito docker-cron   Let   s create a new file called  quot hello-cron quot  to describe our job             echo  quot Hello world quot   gt  gt   var log cron log 2 gt  amp 1   An empty line is required at the end of this file for a valid cron file   If you are wondering what is 2 gt  amp 1  Ayman Hourieh explains   The following Dockerfile describes all the steps to build your image  FROM ubuntu latest MAINTAINER docker ekito fr  RUN apt-get update  amp  amp  apt-get -y install cron    Copy hello-cron file to the cron d directory COPY hello-cron  etc cron d hello-cron     Give execution rights on the cron job RUN chmod 0644  etc cron d hello-cron    Apply cron job RUN crontab  etc cron d hello-cron     Create the log file to be able to run tail RUN touch  var log cron log     Run the command on container startup CMD cron  amp  amp  tail -f  var log cron log   see Gaafar s comment and How do I make apt-get install less noisy   apt-get -y install -qq --force-yes cron can work too  As noted by Nathan Lloyd in the comments   Quick note about a gotcha  If you re adding a script file and telling cron to run it  remember to RUN chmod 0744  the script Cron fails silently if you forget    OR  make sure your job itself redirect directly to stdout stderr instead of a log file  as described in hugoShaka s answer             root echo hello  gt   proc 1 fd 1 2 gt  proc 1 fd 2  Replace the last Dockerfile line with CMD   quot cron quot    quot -f quot    See also  about cron -f  which is to say cron  quot foreground quot    quot docker ubuntu cron -f is not working quot   Build and run it  sudo docker build --rm -t ekito cron-example   sudo docker run -t -i ekito cron-example   Be patient  wait for 2 minutes and your commandline should display   Hello world Hello world   Eric adds in the comments   Do note that tail may not display the correct file if it is created during image build  If that is the case  you need to create or touch the file during container runtime in order for tail to pick up the correct file   See  quot Output of tail -f at the end of a docker CMD is not showing quot

User · Answer

Unfortunately  none of the above answers worked for me  although all answers lead to the solution and eventually to my solution  here is the snippet if it helps someone  Thanks This can be solved with the bash file  due to the layered architecture of the Docker  cron service doesn t get initiated with RUN CMD ENTRYPOINT commands  Simply add a bash file which will initiate the cron and other services  if required  DockerFile FROM gradle 6 5 1-jdk11 AS build   apt RUN apt-get update RUN apt-get -y install cron   Setup cron to run every minute to print  you can add update your cron here  RUN touch  var log cron-1 log RUN  crontab -l   echo  quot           echo testing cron      gt  gt   var log cron-1 log 2 gt  amp 1 quot     crontab   entrypoint sh RUN chmod  x entrypoint sh CMD   quot bash quot   quot entrypoint sh quot    entrypoint sh    bin sh service cron start  amp  tail -f  var log cron-2 log  If any other service is also required to run along with cron then add that service with  amp  in the same command  for example   opt wildfly bin standalone sh  amp  service cron start  amp  tail -f  var log cron-2 log Once you will get into the docker container there you can see that testing cron     will be getting printed every minute in file   var log cron-1 log

User · Answer

VonC s answer is pretty thorough  In addition I d like to add one thing that helped me  If you just want to run a cron job without tailing a file  you d be tempted to just remove the  amp  amp  tail -f  var log cron log from the cron command    However this will cause the Docker container to exit shortly after running because when the cron command completes  Docker thinks the last command has exited and hence kills the container  This can be avoided by running cron in the foreground via cron -f

User · Answer

Try using clockwork gem to schedule tasks  Follow the steps provided in this link   http   fuzzyblog io blog rails 2017 05 11 adding-cron-to-a-dockerized-rails-application-using-clockwork html  You can call the rake task inside lib clock rb file as below   every 1 day   Import large data from csv files    at   gt   5 00   do  job     rake  portal import data from csv   end   Create a separate container in docker-compose file  amp  run the below command inside the container   command  bundle exec clockwork lib clock rb

User · Answer

From above examples I created this combination   Alpine Image  amp  Edit Using Crontab in Nano  I hate vi   FROM alpine  RUN apk update RUN apk add curl nano  ENV EDITOR  usr bin nano     start crond with log level 8 in foreground  output to stderr CMD   crond    -f    -d    8      Shell Access   docker exec -it  lt CONTAINERID gt   bin sh    Example Cron Entry   crontab -e             echo hello  gt   proc 1 fd 1 2 gt  proc 1 fd 2   DATE TIME WILL BE IN UTC

User · Answer

The adopted solution may be dangerous in a production environment      In docker you should only execute one process per container because if you don t  the process that forked and went background is not monitored and may stop without you knowing it    When you use CMD cron  amp  amp  tail -f  var log cron log the cron process basically fork in order to execute cron in background  the main process exits and let you execute tailf in foreground  The background cron process could stop or fail you won t notice  your container will still run silently and your orchestration tool will not restart it      You can avoid such a thing by redirecting directly the cron s commands output into your docker stdout and stderr which are located respectively in  proc 1 fd 1 and  proc 1 fd 2    Using basic shell redirects you may want to do something like this               root echo hello  gt   proc 1 fd 1 2 gt  proc 1 fd 2   And your CMD will be   CMD   cron    -f

User · Answer

Define the cronjob in a dedicated container which runs the command via docker exec to your service   This is higher cohesion and the running script will have access to the environment variables you have defined for your service    docker-compose yml version   3 3  services      myservice        environment          MSG  i m being cronjobbed  every minute        image  alpine       container name  myservice       command  tail -f  dev null      cronjobber       image  docker edge      volumes        -  var run docker sock  var run docker sock      container name  cronjobber      command   gt            sh -c             echo            docker exec myservice printenv   grep MSG   gt   etc crontabs root            amp  amp  crond -f

User · Answer

Another route you can take is Ofelia  which is a highly configurable task runner image that allows 4 modes of execution  job-exec  this job is executed inside of a running container  job-run  runs a command inside of a new container  using a specific image  job-local  runs the command inside of the host running ofelia  job-service-run  runs the command inside a new  quot run-once quot  service  for running inside a swarm  The advantage here is that somebody else has done all the heavy lifting for you  Super convenient and easy  It also has a spiffy official mascot

User · Answer

What  VonC has suggested is nice but I prefer doing all cron job configuration in one line  This would avoid cross platform issues like cronjob location and you don t need a separate cron file   FROM ubuntu latest    Install cron RUN apt-get -y install cron    Create the log file to be able to run tail RUN touch  var log cron log    Setup cron job RUN  crontab -l   echo            echo  Hello world   gt  gt   var log cron log     crontab    Run the command on container startup CMD cron  amp  amp  tail -f  var log cron log   After running your docker container  you can make sure if cron service is working by     To check if the job is scheduled docker exec -ti  lt your-container-id gt  bash -c  crontab -l    To check if the cron service is running docker exec -ti  lt your-container-id gt  bash -c  pgrep cron    If you prefer to have ENTRYPOINT instead of CMD  then you can substitute the CMD above with  ENTRYPOINT cron start  amp  amp  tail -f  var log cron log

User · Answer

Though this aims to run jobs beside a running process in a container via Docker s exec interface  this may be of interest for you   I ve written a daemon that observes containers and schedules jobs  defined in their metadata  on them  Example   version   2   services    wordpress      image  wordpress   mysql      image  mariadb     volumes        -   database dumps  dumps     labels        deck-chores dump command  sh -c  mysqldump --all-databases  gt   dumps dump-   date -Idate         deck-chores dump interval  daily    Classic   cron-like configuration is also possible   Here are the docs  here s the image repository

User · Answer

When you deploy your container on another host  just note that it won t start any processes automatically  You need to make sure that  cron  service is running inside your container  In our case  I am using Supervisord with other services to start cron service    program misc  command  etc init d cron restart user root autostart true autorestart true stderr logfile  var log misc-cron err log stdout logfile  var log misc-cron out log priority 998

User · Answer

Here s my docker-compose based solution    cron      image  alpine 3 10     command  crond -f -d 8     depends on        - servicename     volumes        -    conf cron  etc crontabs root z      restart  unless-stopped  the lines with cron entries are on the   conf cron file  Note  this won t run commands that aren t on the alpine image

User · Answer

Cron jobs are stored in  var spool cron crontabs  Common place in all distros I Know   BTW  You can create a cron tab in bash using something like that   crontab -l  gt  cronexample echo  00 09     1-5 echo hello   gt  gt  cronexample crontab cronexample rm cronexample   This will create a temporary file with cron task  then program it using crontab  Last line remove temporary file

User · Answer

this line was the one that helped me run my pre-scheduled task  ADD mycron root  etc cron d root  RUN chmod 0644  etc cron d root  RUN crontab  etc cron d root  RUN touch  var log cron log  CMD   cron -f -l 8  amp     amp  amp  apache2-foreground    lt -- run cron  -- gt  My project run inside  FROM php 7 2-apache

User · Answer

The most robust way I found so far is run an independent cron container - install the docker client and bind mount the docker sock so you can talk to the docker server on the host   Then just use env vars for each cron job and an entrypoint script to generate the  etc crontab   Here is an image that I created using this principle and using it in production for the last 3-4 years   https   www vip-consult solutions post better-docker-cron content

User · Answer

I created a Docker image based on the other answers  which can be used like  docker run -v   path to cron  etc cron d crontab  gaafar cron  where  path to cron  absolute path to crontab file  or you can use it as a base in a Dockerfile   FROM gaafar cron    COPY crontab file in the cron directory COPY crontab  etc cron d crontab    Add your commands here   For reference  the image is here

User · Answer

Setup a cron in parallel to a one-time job  Create a script file  say run sh  with the job that is supposed to run periodically      bin bash timestamp  date   Y  m  d- H  M  S  echo  System path is  PATH at  timestamp    Save and exit   Use Entrypoint instead of CMD  f you have multiple jobs to kick in during docker containerization  use the entrypoint file to run them all   Entrypoint file is a script file that comes into action when a docker run command is issued  So  all the steps that we want to run can be put in this script file   For instance  we have 2 jobs to run   Run once job  echo    Docker container has been started     Run periodic job  run sh  Create entrypoint sh     bin bash    Start the run once job  echo  Docker container has been started     Setup a cron schedule echo             run sh  gt  gt   var log cron log 2 gt  amp 1   This extra line makes it a valid cron   gt  scheduler txt  crontab scheduler txt cron -f   Let   s understand the crontab that has been set up in the file             Cron schedule  the job must run every minute  You can update the schedule based on your requirement    run sh  The path to the script file which is to be run periodically   var log cron log  The filename to save the output of the scheduled cron job   2 gt  amp 1  The error logs if any  also will be redirected to the same output file used above   Note  Do not forget to add an extra new line  as it makes it a valid cron  Scheduler txt  the complete cron setup will be redirected to a file   Using System User specific environment variables in cron  My actual cron job was expecting most of the arguments as the environment variables passed to the docker run command  But  with bash  I was not able to use any of the environment variables that belongs to the system or the docker container   Then  this came up as a walkaround to this problem    Add the following line in the entrypoint sh   declare -p   grep -Ev  BASHOPTS BASH VERSINFO EUID PPID SHELLOPTS UID   gt   container env    Update the cron setup and specify-   SHELL  bin bash BASH ENV  container env   At last  your entrypoint sh should look like     bin bash    Start the run once job  echo  Docker container has been started   declare -p   grep -Ev  BASHOPTS BASH VERSINFO EUID PPID SHELLOPTS UID   gt   container env    Setup a cron schedule echo  SHELL  bin bash BASH ENV  container env            run sh  gt  gt   var log cron log 2 gt  amp 1   This extra line makes it a valid cron   gt  scheduler txt  crontab scheduler txt cron -f   Last but not the least  Create a Dockerfile  FROM ubuntu 16 04 MAINTAINER Himanshu Gupta    Install cron RUN apt-get update  amp  amp  apt-get install -y cron    Add files ADD run sh  run sh ADD entrypoint sh  entrypoint sh  RUN chmod  x  run sh  entrypoint sh  ENTRYPOINT  entrypoint sh   That   s it  Build and Run the Docker image

[docker] How to run a cron job inside a docker container?

Examples related to docker

Examples related to cron

Examples related to containers

Examples related to sh