Upgrade python in a virtualenv

Question

Is there a way to upgrade the version of python used in a virtualenv  e g  if a bugfix release comes out    I could pip freeze --local  gt  requirements txt  then remove the directory and pip install -r requirements txt  but this requires a lot of reinstallation of large libraries  for instance  numpy  which I use a lot    I can see this is an advantage when upgrading from  e g   2 6 -  2 7  but what about 2 7 x -  2 7 y

User · Answer

How to upgrade the Python version for an existing virtualenvwrapper project and keep the same name

I'm adding an answer for anyone using Doug Hellmann's excellent virtualenvwrapper specifically since the existing answers didn't do it for me.

Some context:

I work on some projects that are Python 2 and some that are Python 3; while I'd love to use python3 -m venv, it doesn't support Python 2 environments
When I start a new project, I use mkproject which creates the virtual environment, creates an empty project directory, and cds into it
I want to continue using virtualenvwrapper's workon command to activate any project irrespective of Python version

Directions:

Let's say your existing project is named foo and is currently running Python 2 (mkproject -p python2 foo), though the commands are the same whether upgrading from 2.x to 3.x, 3.6.0 to 3.6.1, etc. I'm also assuming you're currently inside the activated virtual environment.

1. Deactivate and remove the old virtual environment:

$ deactivate
$ rmvirtualenv foo

Note that if you've added any custom commands to the hooks (e.g., bin/postactivate) you'd need to save those before removing the environment.

2. Stash the real project in a temp directory:

$ cd ..
$ mv foo foo-tmp

3. Create the new virtual environment (and project dir) and activate:

$ mkproject -p python3 foo

4. Replace the empty generated project dir with real project, change back into project dir:

$ cd ..
$ mv -f foo-tmp foo
$ cdproject

5. Re-install dependencies, confirm new Python version, etc:

$ pip install -r requirements.txt
$ python --version

If this is a common use case, I'll consider opening a PR to add something like $ upgradevirtualenv / $ upgradeproject to virtualenvwrapper.

User · Answer

I just want to clarify  because some of the answers refer to venv and others refer to virtualenv   Use of the -p or --python flag is supported on virtualenv  but not on venv  If you have more than one Python version and you want to specify which one to create the venv with  do it on the command line  like this   malikarumi Tetuoan2   Projects  python3 6 -m venv  path to pre-existing dir you want venv in    You can of course upgrade with venv as others have pointed out  but that assumes you have already upgraded the Python that was used to create that venv in the first place  You can t upgrade to a Python version you don t already have on your system somewhere  so make sure to get the version you want  first  then make all the venvs you want from it

User · Answer

If you re using pipenv  I don t know if it s possible to upgrade an environment in place  but at least for minor version upgrades it seems to be smart enough not to rebuild packages from scratch when it creates a new environment  E g   from 3 6 4 to 3 6 5     pipenv --python 3 6 5 install Virtualenv already exists  Removing existing virtualenv    Creating a v  pipenv --python 3 6 5 install Virtualenv already exists  Removing existing virtualenv    Creating a virtualenv for this project    Using  usr local bin python3 6m  3 6 5  to create virtualenv     Running virtualenv with interpreter  usr local bin python3 6m Using base prefix   usr local Cellar python 3 6 5 Frameworks Python framework Versions 3 6  New python executable in  Users dmoles  local share virtualenvs autoscale-aBUhewiD bin python3 6 Also creating executable in  Users dmoles  local share virtualenvs autoscale-aBUhewiD bin python Installing setuptools  pip  wheel   done   Virtualenv location   Users dmoles  local share virtualenvs autoscale-aBUhewiD Installing dependencies from Pipfile lock  84dd0e                                           47 47     00 00 24 To activate this project s virtualenv  run the following     pipenv shell   pipenv shell Spawning environment shell   bin bash   Use  exit  to leave     Users dmoles  local share virtualenvs autoscale-aBUhewiD bin activate bash-3 2     Users dmoles  local share virtualenvs autoscale-aBUhewiD bin activate  autoscale-aBUhewiD  bash-3 2  python Python 3 6 5  default  Mar 30 2018  06 41 53    GCC 4 2 1 Compatible Apple LLVM 9 0 0  clang-900 0 39 2   on darwin Type  help    copyright    credits  or  license  for more information   gt  gt  gt  import numpy as np  gt  gt  gt

User · Answer

Did you see this  If I haven t misunderstand that answer  you may try to create a new virtualenv on top of the old one  You just need to know which python is going to use your virtualenv  you will need to see your virtualenv version     If your virtualenv is installed with the same python version of the old one and upgrading your virtualenv package is not an option  you may want to read this in order to install a virtualenv with the python version you want   EDIT  I ve tested this approach  the one that create a new virtualenv on top of the old one  and it worked fine for me  I think you may have some problems if you change from python 2 6 to 2 7 or 2 7 to 3 x but if you just upgrade inside the same version  staying at 2 7 as you want  you shouldn t have any problem  as all the packages are held in the same folders for both python versions  2 7 x and 2 7 y packages are inside your env lib python2 7     If you change your virtualenv python version  you will need to install all your packages again for that version  or just link the packages you need into the new version packages folder  i e  your env lib python newversion site-packages

User · Answer

For everyone with the problem     Error  Command     Users me Sites site venv3 bin python3    -Im    ensurepip    --upgrade    --default-pip    returned non-zero exit status 1    You have to install python3 6-venv   sudo apt-get install python3 6-venv

User · Answer

I wasn t able to create a new virtualenv on top of the old one   But there are tools in pip which make it much faster to re-install requirements into a brand new venv   Pip can build each of the items in your requirements txt into a wheel package  and store that in a local cache   When you create a new venv and run pip install in it  pip will automatically use the prebuilt wheels if it finds them   Wheels install much faster than running setup py for each module   My    pip pip conf looks like this    global  download-cache    Users me  pip download-cache find-links    Users me  pip wheels    wheel  wheel-dir    Users me  pip wheels   I install wheel  pip install wheel   then run pip wheel -r requirements txt   This stores the built wheels in the wheel-dir in my pip conf   From then on  any time I pip install any of these requirements  it installs them from the wheels  which is pretty quick

User · Answer

This approach always works for me      First of all  delete all broken links  Replace  my project name  to your virtual env name find    virtualenvs my project name  -type l -delete   Then create new links to the current Python version virtualenv    virtualenvs my project name    It s it  Just repeat for each virtualenv located in    virtualenvs   Taken from    https   github com 1st python-on-osx python-virtualenv https   gist github com 1st ced02a1c64ac7b82bb27e432eea6b068

User · Answer

Let s consider that the environment that one wants to update has the name venv  1  Backup venv requirementes  optional  First of all  backup the requirements of the virtual environment  pip freeze  gt  requirements txt deactivate  Move the folder to a new one mv venv venv old  2  Install Python Assuming that one doesn t have sudo access  pyenv is a reliable and fast way to install Python  For that  one should run   curl https   pyenv run   bash  and then   exec  SHELL  As it is suggested here  If  when one tries to update pyenv pyenv update  And one gets the error bash  pyenv  command not found  It is because pyenv path wasn t exported to  bashrc  It can be solved by executing the following commands  echo  export PYENV ROOT  quot  HOME  pyenv quot    gt  gt     bashrc echo  export PATH  quot  PYENV ROOT bin  PATH quot    gt  gt     bashrc echo -e  if command -v pyenv 1 gt  dev null 2 gt  amp 1  then n eval  quot   pyenv init -  quot  nfi   gt  gt     bashrc  Then restart the shell exec  quot  SHELL quot   Now one should install the Python version that one wants  Let s say version 3 8 3 pyenv install 3 8 3  One can confirm if it was properly installed by running pyenv versions  The output should be the location and the versions  in this case 3 8 3  3  Create the new virtual environment Finally  with the new Python version installed  create a new virtual environment  let s call it venv  python3 8 -m venv venv  Activate it source venv bin activate  and install the requirements pip install -r requirements txt  Now one should be up and running with a new environment

User · Answer

Step 1  Freeze requirement  amp  take a back-up of existing env  pip freeze  gt  requirements txt deactivate mv env env old   Step 2  Install Python 3 7  amp  activate virutal environment  sudo apt-get install python3 7-venv python3 7 -m venv env source env bin activate python --version   Step 3  Install requirements  sudo apt-get install python3 7-dev pip3 install -r requirements txt

User · Answer

On OS X or macOS using Homebrew to install and upgrade Python3 I had to delete symbolic links before python -m venv --upgrade ENV DIR would work   I saved the following in upgrade python3 sh so I would remember how months from now when I need to do it again   brew upgrade python3 find    virtualenvs  -type l -delete find    virtualenvs  -type d -mindepth 1 -maxdepth 1 -exec python3 -m venv --upgrade           UPDATE  while this seemed to work well at first  when I ran py test it gave an error   In the end I just re-created the environment from a requirements file

User · Answer

Updated again  The following method might not work in newer versions of virtualenv  Before you try to make modifications to the old virtualenv  you should save the dependencies in a requirement file  pip freeze  gt  requirements txt  and make a backup of it somewhere else  If anything goes wrong  you can still create a new virtualenv and install the old dependencies in it  pip install -r requirements txt    Updated  I changed the answer 5 months after I originally answered  The following method is more convenient and robust   Side effect  it also fixes the Symbol not found   SSLv2 method exception when you do import ssl in a virtual environment after upgrading Python to v2 7 8   Notice  Currently  this is for Python 2 7 x only     If you re using Homebrew Python on OS X  first deactivate all virtualenv  then upgrade Python   brew update  amp  amp  brew upgrade python   Run the following commands   lt EXISTING ENV PATH gt  is path of your virtual environment    cd  lt EXISTING ENV PATH gt  rm  Python rm bin pip  2 2 7  rm bin python  2 2 7  rm -r include python2 7 rm lib python2 7   rm -r lib python2 7 distutils rm lib python2 7 site-packages easy install   rm -r lib python2 7 site-packages pip rm -r lib python2 7 site-packages pip-  dist-info rm -r lib python2 7 site-packages setuptools rm -r lib python2 7 site-packages setuptools-  dist-info   Finally  re-create your virtual environment   virtualenv  lt EXISTING ENV PATH gt    By doing so  old Python core files and standard libraries  plus setuptools and pip  are removed  while the custom libraries installed in site-packages are preserved and working  as soon as they are in pure Python  Binary libraries may or may not need to be reinstalled to function properly   This worked for me on 5 virtual environments with Django installed   BTW  if   manage py compilemessages is not working afterwards  try this   brew install gettext  amp  amp  brew link gettext --force

User · Answer

I moved my home directory from one mac to another  Mountain Lion to Yosemite  and didn t realize about the broken virtualenv until I lost hold of the old laptop  I had the virtualenv point to Python 2 7 installed by brew and since Yosemite came with Python 2 7  I wanted to update my virtualenv to the system python  When I ran virtualenv on top of the existing directory  I was getting OSError   Errno 17  File exists    Users hdara bin python2 7 lib python2 7 config  error  By trial and error  I worked around this issue by removing a few links and fixing up a few more manually  This is what I finally did  similar to what  Rockalite did  but simpler    cd  lt virtualenv-root gt  rm lib python2 7 config rm lib python2 7 lib-dynload rm include python2 7 rm  Python cd lib python2 7 gfind   -type l -xtype l   while read f  do ln -s -f  System Library Frameworks Python framework Versions 2 7 lib python2 7   f      f  done   After this  I was able to just run virtualenv on top of the existing directory

User · Answer

If you happen to be using the venv module that comes with Python 3 3   it supports an --upgrade option   Per the docs      Upgrade the environment directory to use this version of Python  assuming Python has been upgraded in-place   python3 -m venv --upgrade ENV DIR

[python] Upgrade python in a virtualenv

How to upgrade the Python version for an existing virtualenvwrapper project and keep the same name

Examples related to python

Examples related to version

Examples related to virtualenv

Examples related to pip