[python] Pip freeze vs. pip list

A comparison of outputs reveals differences:

user@user-VirtualBox:~$ pip list
feedparser (5.1.3)
pip (1.4.1)
setuptools (1.1.5)
wsgiref (0.1.2)
user@user-VirtualBox:~$ pip freeze
feedparser==5.1.3
wsgiref==0.1.2

Pip's documentation states

freeze                      Output installed packages in requirements format.
list                        List installed packages.

but what is "requirements format," and why does pip list generate a more comprehensive list than pip freeze?

pip list

List installed packages: show ALL installed packages that even pip installed implictly

pip freeze

List installed packages: - list of packages that are installed using pip command

pip freeze has --all flag to show all the packages.

Other difference is the output it renders, that you can check by running the commands.

The main difference is that the output of pip freeze can be dumped into a requirements.txt file and used later to re-construct the "frozen" environment.

In other words you can run: pip freeze > frozen-requirements.txt on one machine and then later on a different machine or on a clean environment you can do: pip install -r frozen-requirements.txt and you'll get the an identical environment with the exact same dependencies installed as you had in the original environment where you generated the frozen-requirements.txt.

pip list shows ALL installed packages.

pip freeze shows packages YOU installed via pip (or pipenv if using that tool) command in a requirements format.

Remark below that setuptools, pip, wheel are installed when pipenv shell creates my virtual envelope. These packages were NOT installed by me using pip:

test1 % pipenv shell
Creating a virtualenv for this project…
Pipfile: /Users/terrence/Development/Python/Projects/test1/Pipfile
Using /usr/local/Cellar/pipenv/2018.11.26_3/libexec/bin/python3.8 (3.8.1) to create virtualenv…
? Creating virtual environment...
<SNIP>
Installing setuptools, pip, wheel...
done.
? Successfully created virtual environment! 
<SNIP>

Now review & compare the output of the respective commands where I've only installed cool-lib and sampleproject (of which peppercorn is a dependency):

test1 % pip freeze       <== Packages I'VE installed w/ pip

-e git+https://github.com/gdamjan/hello-world-python-package.git@10<snip>71#egg=cool_lib
peppercorn==0.6
sampleproject==1.3.1


test1 % pip list         <== All packages, incl. ones I've NOT installed w/ pip

Package       Version Location                                                                    
------------- ------- --------------------------------------------------------------------------
cool-lib      0.1  /Users/terrence/.local/share/virtualenvs/test1-y2Zgz1D2/src/cool-lib           <== Installed w/ `pip` command
peppercorn    0.6       <== Dependency of "sampleproject"
pip           20.0.2  
sampleproject 1.3.1     <== Installed w/ `pip` command
setuptools    45.1.0  
wheel         0.34.2

For those looking for a solution. If you accidentally made pip requirements with pip list instead of pip freeze, and want to convert into pip freeze format. I wrote this R script to do so.

library(tidyverse)

pip_list = read_lines("requirements.txt")

pip_freeze = pip_list %>%
  str_replace_all(" \\(", "==") %>%
  str_replace_all("\\)$", "")

pip_freeze %>% write_lines("requirements.txt")

Look at the pip documentation, which describes the functionality of both as:

pip list

List installed packages, including editables.

pip freeze

Output installed packages in requirements format.

So there are two differences:

1. Output format, freeze gives us the standard requirement format that may be used later with pip install -r to install requirements from.

2. Output content, pip list include editables which pip freeze does not.

To answer the second part of this question, the two packages shown in pip list but not pip freeze are setuptools (which is easy_install) and pip itself.

It looks like pip freeze just doesn't list packages that pip itself depends on. You may use the --all flag to show also those packages.

From the documentation:

--all

Do not skip these packages in the output: pip, setuptools, distribute, wheel