[python] What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?

Python 3.3 includes in its standard library the new package venv. What does it do, and how does it differ from all the other packages that seem to match the regex (py)?(v|virtual|pip)?env?

I would just avoid the use of virtualenv after Python3.3+ and instead use the standard shipped library venv. To create a new virtual environment you would type:

$ python3 -m venv <MYVENV>  

virtualenv tries to copy the Python binary into the virtual environment's bin directory. However it does not update library file links embedded into that binary, so if you build Python from source into a non-system directory with relative path names, the Python binary breaks. Since this is how you make a copy distributable Python, it is a big flaw. BTW to inspect embedded library file links on OS X, use otool. For example from within your virtual environment, type:

$ otool -L bin/python
python:
    @executable_path/../Python (compatibility version 3.4.0, current version 3.4.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)

Consequently I would avoid virtualenvwrapper and pipenv. pyvenv is deprecated. pyenv seems to be used often where virtualenv is used but I would stay away from it also since I think venv also does what pyenv is built for.

venv creates virtual environments in the shell that are fresh and sandboxed, with user-installable libraries, and it's multi-python safe. Fresh because virtual environments only start with the standard libraries that ship with python, you have to install any other libraries all over again with pip install while the virtual environment is active. Sandboxed because none of these new library installs are visible outside the virtual environment, so you can delete the whole environment and start again without worrying about impacting your base python install. User-installable libraries because the virtual environment's target folder is created without sudo in some directory you already own, so you won't need sudo permissions to install libraries into it. Finally it is multi-python safe, since when virtual environments activate, the shell only sees the python version (3.4, 3.5 etc.) that was used to build that virtual environment.

pyenv is similar to venv in that it lets you manage multiple python environments. However with pyenv you can't conveniently rollback library installs to some start state and you will likely need admin privileges at some point to update libraries. So I think it is also best to use venv.

In the last couple of years I have found many problems in build systems (emacs packages, python standalone application builders, installers...) that ultimately come down to issues with virtualenv. I think python will be a better platform when we eliminate this additional option and only use venv.

EDIT: Tweet of the BDFL,

I use venv (in the stdlib) and a bunch of shell aliases to quickly switch.

— Guido van Rossum (@gvanrossum) October 22, 2020

• pyenv - manages different python versions,
• all others - create virtual environment (which has isolated python version and installed "requirements"),

pipenv want combine all, in addition to previous it installs "requirements" (into the active virtual environment or create its own if none is active)

So maybe you will be happy with pipenv only.

But I use: pyenv + pyenv-virtualenvwrapper, + pipenv (pipenv for installing requirements only).

In Debian:

1. apt install libffi-dev

2. install pyenv based on https://www.tecmint.com/pyenv-install-and-manage-multiple-python-versions-in-linux/, but..

3. .. but instead of pyenv-virtualenv install pyenv-virtualenvwrapper (which can be standalone library or pyenv plugin, here the 2nd option):

   pyenv install 3.9.0

   git clone https://github.com/pyenv/pyenv-virtualenvwrapper.git $(pyenv root)/plugins/pyenv-virtualenvwrapper

   into ~/.bashrc add: export $VIRTUALENVWRAPPER_PYTHON="/usr/bin/python3"

   source ~/.bashrc

   pyenv virtualenvwrapper

Then create virtual environments for your projects (workingdir must exist):

pyenv local 3.9.0  # to prevent 'interpreter not found' in mkvirtualenv
python -m pip install --upgrade pip setuptools wheel
mkvirtualenv <venvname> -p python3.9 -a <workingdir>

and switch between projects:

workon <venvname>
python -m pip install --upgrade pip setuptools wheel pipenv

Inside a project I have the file requirements.txt, without fixing the versions inside (if some version limitation is not neccessary). You have 2 possible tools to install them into the current virtual environment: pip-tools or pipenv. Lets say you will use pipenv:

pipenv install -r requirements.txt

this will create Pipfile and Pipfile.lock files, fixed versions are in the 2nd one. If you want reinstall somewhere exactly same versions then (Pipfile.lock must be present):

pipenv install

Remember that Pipfile.lock is related to some Python version and need to be recreated if you use a different one.

As you see I write requirements.txt. This has some problems: You must remove a removed package from Pipfile too. So writing Pipfile directly is probably better.

So you can see I use pipenv very poorly. Maybe if you will use it well, it can replace everything?

EDIT 2021.01: I have changed my stack to: pyenv + pyenv-virtualenvwrapper + poetry. Ie. I use no apt or pip installation of virtualenv or virtualenvwrapper, and instead I install pyenv's plugin pyenv-virtualenvwrapper. This is easier way.

Poetry is great for me:

poetry add <package>   # install single package
poetry remove <package>
poetry install   # if you remove poetry.lock poetry will re-calculate versions

Let's start with the problems these tools want to solve:

My system package manager don't have the Python versions I wanted or I want to install multiple Python versions side by side, Python 3.9.0 and Python 3.9.1, Python 3.5.3, etc

Then use pyenv.

I want to install and run multiple applications with different, conflicting dependencies.

Then use virtualenv or venv. These are almost completely interchangeable, the difference being that virtualenv supports older python versions and has a few more minor unique features, while venv is in the standard library.

I'm developing an /application/ and need to manage my dependencies, and manage the dependency resolution of the dependencies of my project.

Then use pipenv or poetry.

I'm developing a /library/ or a /package/ and want to specify the dependencies that my library users need to install

Then use setuptools.

I used virtualenv, but I don't like virtualenv folders being scattered around various project folders. I want a centralised management of the environments and some simple project management

Then use virtualenvwrapper. Variant: pyenv-virtualenvwrapper if you also use pyenv.

---

Not recommended

• pyvenv. This is deprecated, use venv or virtualenv instead. Not to be confused with pipenv or pyenv.

Jan 2020 Update

@Flimm has explained all the differences very well. Generally, we want to know the difference between all tools because we want to decide what's best for us. So, the next question would be: which one to use? I suggest you choose one of the two official ways to manage virtual environments:

• Python Packaging now recommends Pipenv
• Python.org now recommends venv

UPDATE 20200825:

Added below "Conclusion" paragraph

I've went down the pipenv rabbit hole (it's a deep and dark hole indeed...) and since the last answer is over 2 years ago, felt it was useful to update the discussion with the latest developments on the Python virtual envelopes topic I've found.

DISCLAIMER:

This answer is NOT about continuing the raging debate about the merits of pipenv versus venv as envelope solutions- I make no endorsement of either. It's about PyPA endorsing conflicting standards and how future development of virtualenv promises to negate making an either/or choice between them at all. I focused on these two tools precisely because they are the anointed ones by PyPA.

venv

As the OP notes, venv is a tool for virtualizing environments. NOT a third party solution, but native tool. PyPA endorses venv for creating VIRTUAL ENVELOPES: "Changed in version 3.5: The use of venv is now recommended for creating virtual environments".

pipenv

pipenv- like venv - can be used to create virtual envelopes but additionally rolls-in package management and vulnerability checking functionality. Instead of using requirements.txt, pipenv delivers package management via Pipfile. As PyPA endorses pipenv for PACKAGE MANAGEMENT, that would seem to imply pipfile is to supplant requirements.txt.

HOWEVER: pipenv uses virtualenv as its tool for creating virtual envelopes, NOT venv which is endorsed by PyPA as the go-to tool for creating virtual envelopes.

Conflicting Standards:

So if settling on a virtual envelope solution wasn't difficult enough, we now have PyPA endorsing two different tools which use different virtual envelope solutions. The raging Github debate on venv vs virtualenv which highlights this conflict can be found here.

Conflict Resolution:

The Github debate referenced in above link has steered virtualenv development in the direction of accommodating venv in future releases:

prefer built-in venv: if the target python has venv we'll create the environment using that (and then perform subsequent operations on that to facilitate other guarantees we offer)

Conclusion:

So it looks like there will be some future convergence between the two rival virtual envelope solutions, but as of now pipenv- which uses virtualenv - varies materially from venv.

Given the problems pipenv solves and the fact that PyPA has given its blessing, it appears to have a bright future. And if virtualenv delivers on its proposed development objectives, choosing a virtual envelope solution should no longer be a case of either pipenv OR venv.

Update 20200825:

An oft repeated criticism of Pipenv I saw when producing this analysis was that it was not actively maintained. Indeed, what's the point of using a solution whose future could be seen questionable due to lack of continuous development? After a dry spell of about 18 months, Pipenv is once again being actively developed. Indeed, large and material updates have since been released.