I am trying to run a Django management command from cron. I am using virtualenv to keep my project sandboxed.
I have seen examples here and elsewhere that show running management commands from within virtualenv's like:
0 3 * * * source /home/user/project/env/bin/activate && /home/user/project/manage.py command arg
However, even though syslog shows an entry when the task should have started, this task never actually runs (the log file for the script is empty). If I run the line manually from the shell, it works as expected.
The only way I can currently get the command to run via cron, is to break the commands up and put them in a dumb bash wrapper script:
#!/bin/sh
source /home/user/project/env/bin/activate
cd /home/user/project/
./manage.py command arg
EDIT:
ars came up with a working combination of commands:
0 3 * * * cd /home/user/project && /home/user/project/env/bin/python /home/user/project/manage.py command arg
At least in my case, invoking the activate script for the virtualenv did nothing. This works, so on with the show.
This question is related to
python
django
cron
virtualenv
Rather than mucking around with virtualenv-specific shebangs, just prepend PATH
onto the crontab.
From an activated virtualenv, run these three commands and python scripts should just work:
$ echo "PATH=$PATH" > myserver.cron
$ crontab -l >> myserver.cron
$ crontab myserver.cron
The crontab's first line should now look like this:
PATH=/home/me/virtualenv/bin:/usr/bin:/bin: # [etc...]
If you're on python and using a Conda Virtual Environment where your python script contains the shebang #!/usr/bin/env python the following works:
* * * * * cd /home/user/project && /home/user/anaconda3/envs/envname/bin/python script.py 2>&1
Additionally, if you want to capture any outputs in your script (e.g. print, errors, etc) you can use the following:
* * * * * cd /home/user/project && /home/user/anaconda3/envs/envname/bin/python script.py >> /home/user/folder/script_name.log 2>&1
I'd like to add this because I spent some time solving the issue and did not find an answer here for combination of variables usage in cron and virtualenv. So maybe it'll help someone.
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DIR_SMTH="cd /smth"
VENV=". venv/bin/activate"
CMD="some_python_bin do_something"
# m h dom mon dow command
0 * * * * $DIR_SMTH && $VENV && $CMD -k2 some_target >> /tmp/crontest.log 2>&1
It did not work well when it was configured like
DIR_SMTH="cd /smth && . venv/bin/activate"
Thanks @davidwinterbottom, @reed-sandberg and @mkb for giving the right direction. The accepted answer actually works fine until your python need to run a script which have to run another python binary from venv/bin directory.
The only correct way to run python cron jobs when using a virtualenv is to activate the environment and then execute the environment's python to run your code.
One way to do this is use virtualenv's activate_this
in your python script, see: http://virtualenv.readthedocs.org/en/latest/userguide.html#using-virtualenv-without-bin-python
Another solution is echoing the complete command including activating the environment and piping it into /bin/bash
. Consider this for your /etc/crontab
:
***** root echo 'source /env/bin/activate; python /your/script' | /bin/bash
The best solution for me was to both
man python
mentions modifying the path in shell at $PYTHONPATH
or in python with sys.path
Other answers mention ideas for doing this using the shell. From python, adding the following lines to my script allows me to successfully run it directly from cron.
import sys
sys.path.insert(0,'/path/to/venv/lib/python3.3/site-packages');
Here's how it looks in an interactive session --
Python 3.3.2+ (default, Feb 28 2014, 00:52:16)
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python3.3', '/usr/lib/python3.3/plat-x86_64-linux-gnu', '/usr/lib/python3.3/lib-dynload']
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'requests'
>>> sys.path.insert(0,'/path/to/venv/modules/');
>>> import requests
>>>
Since a cron executes in its own minimal sh
environment, here's what I do to run Python scripts in a virtual environment:
* * * * * . ~/.bash_profile; . ~/path/to/venv/bin/activate; python ~/path/to/script.py
(Note: if . ~/.bash_profile
doesn't work for you, then try . ~/.bashrc
or . ~/.profile
depending on how your server is set up.)
This loads your bash
shell environment, then activates your Python virtual environment, essentially leaving you with the same setup you tested your scripts in.
No need to define environment variables in crontab and no need to modify your existing scripts.
python script
from datetime import datetime
import boto # check wheather its taking the virtualenv or not
import sys
param1=sys.argv[1] #Param
myFile = open('appendtxt.txt', 'a')
myFile.write('\nAccessed on ' + param1+str(datetime.now()))
Cron command
*/1 * * * * cd /Workspace/testcron/ && /Workspace/testcron/venvcron/bin/python3 /Workspace/testcron/testcronwithparam.py param
In above command
Running source
from a cronfile won't work as cron uses /bin/sh
as its default shell, which doesn't support source
. You need to set the SHELL environment variable to be /bin/bash
:
SHELL=/bin/bash
*/10 * * * * root source /path/to/virtualenv/bin/activate && /path/to/build/manage.py some_command > /dev/null
It's tricky to spot why this fails as /var/log/syslog
doesn't log the error details. Best to alias yourself to root so you get emailed with any cron errors. Simply add yourself to /etc/aliases
and run sendmail -bi
.
More info here: http://codeinthehole.com/archives/43-Running-django-cronjobs-within-a-virtualenv.html
the link above is changed to: https://codeinthehole.com/tips/running-django-cronjobs-within-a-virtualenv/
This is a solution that has worked well for me.
source /root/miniconda3/etc/profile.d/conda.sh && \
conda activate <your_env> && \
python <your_application> &
I am using miniconda with Conda version 4.7.12 on a Ubuntu 18.04.3 LTS.
I am able to place the above inside a script and run it via crontab as well without any trouble.
I've added the following script as manage.sh
inside my Django project, it sources the virtualenv and then runs the manage.py
script with whatever arguments you pass to it. It makes it very easy in general to run commands inside the virtualenv (cron, systemd units, basically anywhere):
#! /bin/bash
# this is a convenience script that first sources the venv (assumed to be in
# ../venv) and then executes manage.py with whatever arguments you supply the
# script with. this is useful if you need to execute the manage.py from
# somewhere where the venv isn't sourced (e.g. system scripts)
# get the script's location
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
# source venv <- UPDATE THE PATH HERE WITH YOUR VENV's PATH
source $DIR/../venv/bin/activate
# run manage.py script
$DIR/manage.py "$@"
Then in your cron entry you can just run:
0 3 * * * /home/user/project/manage.sh command arg
Just remember that you need to make the manage.sh
script executable
Don't look any further:
0 3 * * * /usr/bin/env bash -c 'cd /home/user/project && source /home/user/project/env/bin/activate && ./manage.py command arg' > /dev/null 2>&1
Generic approach:
* * * * * /usr/bin/env bash -c 'YOUR_COMMAND_HERE' > /dev/null 2>&1
The beauty about this is you DO NOT need to change the SHELL
variable for crontab from sh
to bash
Source: Stackoverflow.com