tell pip to install the dependencies of packages listed in a requirement file

Question

Developing a Django web app  I have a list of packages I need to install in a virtualenv   Say   Django  1 3 1 --extra-index-url http   dist pinaxproject com dev  Pinax  0 9b1 dev10 git git   github com pinax pinax-theme-bootstrap git cff4f5bbe9f87f0c67ee9ada9aa8ae82978f9890   and other packages   Initially I installed them manually  one by one  along the development  This installed the required dependencies and I finally used pip freeze before deploying the app   Problem is  as I upgraded some packages  some dependencies are no longer used nor required but they keep being listed by pip freeze   Now  I d like to set up a new virtualenv this way    put the needed packages  without their dependencies  in a requirement file   like manual-requirements txt install them with their dependencies  pip install -r manual-requirement txt    problem  this does not install the dependencies  freeze the full virtualenv  pip freeze -r manual-requirements txt  gt  full-requirements txt and deploy    Any way to do this without manually re-installing the packages in a new virtualenv to get their dependencies   This would be error-prone and I d like to automate the process of cleaning the virtualenv from no-longer-needed old dependencies   edit  actually  pip does install dependencies not explicitly listed in the requirement file  even if the documentation tells us that such files are flat  I was wrong about which dependencies i expected to be installed  I ll let this question for anyone in doubt about pip not installing all dependencies

User · Answer

Any way to do this without manually re-installing the packages in a new virtualenv to get their dependencies ? This would be error-prone and I'd like to automate the process of cleaning the virtualenv from no-longer-needed old dependencies.

That's what pip-tools package is for (from https://github.com/jazzband/pip-tools):

Installation

$ pip install --upgrade pip  # pip-tools needs pip==6.1 or higher (!)
$ pip install pip-tools

Example usage for pip-compile

Suppose you have a Flask project, and want to pin it for production. Write the following line to a file:

# requirements.in
Flask

Now, run pip-compile requirements.in:

$ pip-compile requirements.in
#
# This file is autogenerated by pip-compile
# Make changes in requirements.in, then run this to update:
#
#    pip-compile requirements.in
#
flask==0.10.1
itsdangerous==0.24        # via flask
jinja2==2.7.3             # via flask
markupsafe==0.23          # via jinja2
werkzeug==0.10.4          # via flask

And it will produce your requirements.txt, with all the Flask dependencies (and all underlying dependencies) pinned. Put this file under version control as well and periodically re-run pip-compile to update the packages.

Example usage for pip-sync

Now that you have a requirements.txt, you can use pip-sync to update your virtual env to reflect exactly what's in there. Note: this will install/upgrade/uninstall everything necessary to match the requirements.txt contents.

$ pip-sync
Uninstalling flake8-2.4.1:
  Successfully uninstalled flake8-2.4.1
Collecting click==4.1
  Downloading click-4.1-py2.py3-none-any.whl (62kB)
    100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 65kB 1.8MB/s
  Found existing installation: click 4.0
    Uninstalling click-4.0:
      Successfully uninstalled click-4.0
Successfully installed click-4.1

User · Answer

As  Ming mentioned  pip install -r file txt  Here s a simple line to force update all dependencies  while read -r package  do pip install --upgrade --force-reinstall  package done  lt  pipfreeze txt

User · Answer

Given your comment to the question  where you say that executing the install for a single package works as expected   I would suggest looping over your requirement file  In bash      bin sh while read p  do   pip install  p done  lt  requirements pip   HTH

User · Answer

Extending Piotr s answer  if you also need a way to figure what to put in requirements in  you can first use pip-chill to find the minimal set of required packages you have  By combining these tools  you can show the dependency reason why each package is installed  The full cycle looks like this    Create virtual environment    python3 -m venv venv Activate it      venv bin activate Install newest version of pip  pip-tools and pip-chill   venv   pip install --upgrade pip  venv   pip install pip-tools pip-chill Build your project  install more pip packages  etc  until you want to save    Extract minimal set of packages  ie  top-level without dependencies    venv   pip-chill --no-version  gt  requirements in Compile list of all required packages  showing dependency reasons    venv   pip-compile requirements in Make sure the current installation is synchronized with the list   venv   pip-sync

User · Answer

simplifily  use   pip install -r requirement txt   it can install all listed in requirement file

[python] tell pip to install the dependencies of packages listed in a requirement file

Installation

Example usage for pip-compile

Example usage for pip-sync

Examples related to python

Examples related to pip

Examples related to requirements.txt