[python] How to upgrade all Python packages with pip

Is it possible to upgrade all Python packages at one time with pip?

Note: that there is a feature request for this on the official issue tracker.

A JSON + jq answer:

pip list -o --format json | jq '.[] | .name' | xargs pip install -U

The pip_upgrade_outdated does the job. According to its documentation:

usage: pip_upgrade_outdated [-h] [-3 | -2 | --pip_cmd PIP_CMD]
                            [--serial | --parallel] [--dry_run] [--verbose]
                            [--version]

Upgrade outdated python packages with pip.

optional arguments:
  -h, --help         show this help message and exit
  -3                 use pip3
  -2                 use pip2
  --pip_cmd PIP_CMD  use PIP_CMD (default pip)
  --serial, -s       upgrade in serial (default)
  --parallel, -p     upgrade in parallel
  --dry_run, -n      get list, but don't upgrade
  --verbose, -v      may be specified multiple times
  --version          show program's version number and exit

Step 1:

pip install pip-upgrade-outdated

Step 2:

pip_upgrade_outdated

One-liner version of Ramana's answer.

python -c 'import pip, subprocess; [subprocess.call("pip install -U " + d.project_name, shell=1) for d in pip.get_installed_distributions()]'

You can try this:

for i in ` pip list | awk -F ' ' '{print $1}'`; do pip install --upgrade $i; done

I had the same problem with upgrading. Thing is, I never upgrade all packages. I upgrade only what I need, because project may break.

Because there was no easy way for upgrading package by package, and updating the requirements.txt file, I wrote this pip-upgrader which also updates the versions in your requirements.txt file for the packages chosen (or all packages).

Installation

pip install pip-upgrader

Usage

Activate your virtualenv (important, because it will also install the new versions of upgraded packages in current virtualenv).

cd into your project directory, then run:

pip-upgrade

Advanced usage

If the requirements are placed in a non-standard location, send them as arguments:

pip-upgrade path/to/requirements.txt

If you already know what package you want to upgrade, simply send them as arguments:

pip-upgrade -p django -p celery -p dateutil

If you need to upgrade to pre-release / post-release version, add --prerelease argument to your command.

Full disclosure: I wrote this package.

More Robust Solution

For pip3, use this:

pip3 freeze --local |sed -rn 's/^([^=# \t\\][^ \t=]*)=.*/echo; echo Processing \1 ...; pip3 install -U \1/p' |sh

For pip, just remove the 3s as such:

pip freeze --local |sed -rn 's/^([^=# \t\\][^ \t=]*)=.*/echo; echo Processing \1 ...; pip install -U \1/p' |sh

OS X Oddity

OS X, as of July 2017, ships with a very old version of sed (a dozen years old). To get extended regular expressions, use -E instead of -r in the solution above.

Solving Issues with Popular Solutions

This solution is well designed and tested¹, whereas there are problems with even the most popular solutions.

• Portability issues due to changing pip command line features
• Crashing of xargs because of common pip or pip3 child process failures
• Crowded logging from the raw xargs output
• Relying on a Python-to-OS bridge while potentially upgrading it³

The above command uses the simplest and most portable pip syntax in combination with sed and sh to overcome these issues completely. Details of the sed operation can be scrutinized with the commented version².

Details

[1] Tested and regularly used in a Linux 4.8.16-200.fc24.x86_64 cluster and tested on five other Linux/Unix flavors. It also runs on Cygwin64 installed on Windows 10. Testing on iOS is needed.

[2] To see the anatomy of the command more clearly, this is the exact equivalent of the above pip3 command with comments:

# Match lines from pip's local package list output
# that meet the following three criteria and pass the
# package name to the replacement string in group 1.
# (a) Do not start with invalid characters
# (b) Follow the rule of no white space in the package names
# (c) Immediately follow the package name with an equal sign
sed="s/^([^=# \t\\][^ \t=]*)=.*"

# Separate the output of package upgrades with a blank line
sed="$sed/echo"

# Indicate what package is being processed
sed="$sed; echo Processing \1 ..."

# Perform the upgrade using just the valid package name
sed="$sed; pip3 install -U \1"

# Output the commands
sed="$sed/p"

# Stream edit the list as above
# and pass the commands to a shell
pip3 freeze --local | sed -rn "$sed" | sh

[3] Upgrading a Python or PIP component that is also used in the upgrading of a Python or PIP component can be a potential cause of a deadlock or package database corruption.

Windows PowerShell solution

pip freeze | %{$_.split('==')[0]} | %{pip install --upgrade $_}

Windows version after consulting the excellent documentation for FOR by Rob van der Woude:

for /F "delims===" %i in ('pip freeze -l') do pip install -U %i

This seemed to work for me...

pip install -U $(pip list --outdated | awk '{printf $1" "}')

I used printf with a space afterwards to properly separate the package names.

The following one-liner might prove of help:

(pip > 20.0)

pip list --format freeze --outdated | sed 's/=.*//g' | xargs -n1 pip install -U

Older Versions:

pip list --format freeze --outdated | sed 's/(.*//g' | xargs -n1 pip install -U

xargs -n1 keeps going if an error occurs.

If you need more "fine grained" control over what is omitted and what raises an error you should not add the -n1 flag and explicitly define the errors to ignore, by "piping" the following line for each separate error:

| sed 's/^<First characters of the error>.*//'

Here is a working example:

pip list --format freeze --outdated | sed 's/=.*//g' | sed 's/^<First characters of the first error>.*//' | sed 's/^<First characters of the second error>.*//' | xargs pip install -U

The simplest and fastest solution that I found in the pip issue discussion is:

pip install pipdate
pipdate

Source: https://github.com/pypa/pip/issues/3819

Here is another way of doing with a script in Python:

import pip, tempfile, contextlib

with tempfile.TemporaryFile('w+') as temp:
    with contextlib.redirect_stdout(temp):
        pip.main(['list', '-o'])
    temp.seek(0)
    for line in temp:
        pk = line.split()[0]
        print('--> updating', pk, '<--')
        pip.main(['install', '-U', pk])

When using a virtualenv and if you just want to upgrade packages added to your virtualenv, you may want to do:

pip install `pip freeze -l | cut --fields=1 -d = -` --upgrade

Use AWK update packages:

pip install -U $(pip freeze | awk -F'[=]' '{print $1}')

Windows PowerShell update

foreach($p in $(pip freeze)){ pip install -U $p.Split("=")[0]}

Updating Python Packages On Windows Or Linux

1-Output a list of installed packages into a requirements file (requirements.txt):

pip freeze > requirements.txt

2- Edit requirements.txt, and replace all ‘==’ with ‘>=’. Use the ‘Replace All’ command in the editor.

3 - Upgrade all outdated packages

pip install -r requirements.txt --upgrade

Source:https://www.activestate.com/resources/quick-reads/how-to-update-all-python-packages/

You can just print the packages that are outdated:

pip freeze | cut -d = -f 1 | xargs -n 1 pip search | grep -B2 'LATEST:'

python -c 'import pip; [pip.main(["install", "--upgrade", d.project_name]) for d in pip.get_installed_distributions()]'

One liner!

Here's the code for updating all Python 3 packages (in the activated virtualenv) via pip:

import pkg_resources
from subprocess import call

for dist in pkg_resources.working_set:
    call("python3 -m pip install --upgrade " + dist.project_name, shell=True)

If you want upgrade only packaged installed by pip, and to avoid upgrading packages that are installed by other tools (like apt, yum etc.), then you can use this script that I use on my Ubuntu (maybe works also on other distros) - based on this post:

printf "To update with pip: pip install -U"
pip list --outdated 2>/dev/null | gawk '{print $1;}' | while read; do pip show "${REPLY}" 2>/dev/null | grep 'Location: /usr/local/lib/python2.7/dist-packages' >/dev/null; if (( $? == 0 )); then printf " ${REPLY}"; fi; done; echo

The below Windows cmd snippet does the following:

• Upgrades pip to latest version.
• Upgrades all outdated packages.
• For each packages being upgraded checks requirements.txt for any version specifiers.

@echo off
Setlocal EnableDelayedExpansion
rem https://stackoverflow.com/questions/2720014/

echo Upgrading pip...
python -m pip install --upgrade pip
echo.

echo Upgrading packages...
set upgrade_count=0
pip list --outdated > pip-upgrade-outdated.txt
for /F "skip=2 tokens=1,3 delims= " %%i in (pip-upgrade-outdated.txt) do (
    echo ^>%%i
    set package=%%i
    set latest=%%j
    set requirements=!package!

    rem for each outdated package check for any version requirements:
    set dotest=1
    for /F %%r in (.\python\requirements.txt) do (
        if !dotest!==1 (
            call :substr "%%r" !package! _substr
            rem check if a given line refers to a package we are about to upgrade:
            if "%%r" NEQ !_substr! (
                rem check if the line contains more than just a package name:
                if "%%r" NEQ "!package!" (
                    rem set requirements to the contents of the line:
                    echo requirements: %%r, latest: !latest!
                    set requirements=%%r
                )
                rem stop testing after the first instance found,
                rem prevents from mistakenly matching "py" with "pylint", "numpy" etc.
                rem requirements.txt must be structured with shorter names going first
                set dotest=0
            )
        )
    )
    rem pip install !requirements!
    pip install --upgrade !requirements!
    set /a "upgrade_count+=1"
    echo.
)

if !upgrade_count!==0 (
    echo All packages are up to date.
) else (
    type pip-upgrade-outdated.txt
)

if "%1" neq "-silent" (
    echo.
    set /p temp="> Press Enter to exit..."
)
exit /b


:substr
rem string substition done in a separate subroutine -
rem allows expand both variables in the substring syntax.
rem replaces str_search with an empty string.
rem returns the result in the 3rd parameter, passed by reference from the caller.
set str_source=%1
set str_search=%2
set str_result=!str_source:%str_search%=!
set "%~3=!str_result!"
rem echo !str_source!, !str_search!, !str_result!
exit /b

If you are on macOS,

1. make sure you have Homebrew installed
2. install jq in order to read the JSON you’re about to generate

brew install jq

3. update each item on the list of outdated packages generated by pip3 list --outdated

pip3 install --upgrade  `pip3 list --outdated --format json | jq '.[] | .name' | awk -F'"' '{print $2}'`

This seems more concise.

pip list --outdated | cut -d ' ' -f1 | xargs -n1 pip install -U

Explanation:

pip list --outdated gets lines like these

urllib3 (1.7.1) - Latest: 1.15.1 [wheel]
wheel (0.24.0) - Latest: 0.29.0 [wheel]

In cut -d ' ' -f1, -d ' ' sets "space" as the delimiter, -f1 means to get the first column.

So the above lines becomes:

urllib3
wheel

then pass them to xargs to run the command, pip install -U, with each line as appending arguments

-n1 limits the number of arguments passed to each command pip install -U to be 1

Use:

import pip
pkgs = [p.key for p in pip.get_installed_distributions()]
for pkg in pkgs:
    pip.main(['install', '--upgrade', pkg])

Or even:

import pip
commands = ['install', '--upgrade']
pkgs = commands.extend([p.key for p in pip.get_installed_distributions()])
pip.main(commands)

It works fast as it is not constantly launching a shell.

Use:

pip install -r <(pip freeze) --upgrade

See all outdated packages

 pip list --outdated --format=columns

Install

 sudo pip install pipdate

then type

 sudo -H pipdate

This option seems to me more straightforward and readable:

pip install -U `pip list --outdated | awk 'NR>2 {print $1}'`

The explanation is that pip list --outdated outputs a list of all the outdated packages in this format:

Package   Version Latest Type 
--------- ------- ------ -----
fonttools 3.31.0  3.32.0 wheel
urllib3   1.24    1.24.1 wheel
requests  2.20.0  2.20.1 wheel

In the awk command, NR>2 skips the first two records (lines) and {print $1} selects the first word of each line (as suggested by SergioAraujo, I removed tail -n +3 since awk can indeed handle skipping records).

As another answer here stated:

pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U

Is a possible solution: Some comments here, myself included, had issues with permissions while using this command. A little change to the following solved those for me.

pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 sudo -H pip install -U

Note the added sudo -H which allowed the command to run with root permissions.

One line in PowerShell 5.1 with administrator rights, Python 3.6.5, and pip version 10.0.1:

pip list -o --format json | ConvertFrom-Json | foreach {pip install $_.name -U --no-warn-script-location}

It works smoothly if there are no broken packages or special wheels in the list...

One line in cmd:

for /F "delims= " %i in ('pip list --outdated --format=legacy') do pip install -U %i

So a

pip check

afterwards should make sure no dependencies are broken.

I have tried the code of Ramana and I found out on Ubuntu you have to write sudo for each command. Here is my script which works fine on Ubuntu 13.10 (Saucy Salamander):

#!/usr/bin/env python
import pip
from subprocess import call

for dist in pip.get_installed_distributions():
    call("sudo pip install --upgrade " + dist.project_name, shell=True)

Here is my variation on rbp's answer, which bypasses "editable" and development distributions. It shares two flaws of the original: it re-downloads and reinstalls unnecessarily; and an error on one package will prevent the upgrade of every package after that.

pip freeze |sed -ne 's/==.*//p' |xargs pip install -U --

Related bug reports, a bit disjointed after the migration from Bitbucket:

• https://github.com/pypa/pip/issues/49
• https://github.com/pypa/pip/issues/59

The following works on Windows and should be good for others too ($ is whatever directory you're in, in the command prompt. For example, C:/Users/Username).

Do

$ pip freeze > requirements.txt

Open the text file, replace the == with >=, and execute:

$ pip install -r requirements.txt --upgrade

If you have a problem with a certain package stalling the upgrade (NumPy sometimes), just go to the directory ($), comment out the name (add a # before it) and run the upgrade again. You can later uncomment that section back. This is also great for copying Python global environments.

Another way:

I also like the pip-review method:

py2
$ pip install pip-review

$ pip-review --local --interactive

py3
$ pip3 install pip-review

$ py -3 -m pip_review --local --interactive

You can select 'a' to upgrade all packages; if one upgrade fails, run it again and it continues at the next one.

The shortest and easiest on Windows.

pip freeze > requirements.txt && pip install --upgrade -r requirements.txt && rm requirements.txt

You can use the following Python code. Unlike pip freeze, this will not print warnings and FIXME errors. For pip < 10.0.1

import pip
from subprocess import call

packages = [dist.project_name for dist in pip.get_installed_distributions()]
call("pip install --upgrade " + ' '.join(packages), shell=True)

For pip >= 10.0.1

import pkg_resources
from subprocess import call

packages = [dist.project_name for dist in pkg_resources.working_set]
call("pip install --upgrade " + ' '.join(packages), shell=True)

A pure Bash/Z shell one-liner for achieving that:

for p in $(pip list -o --format freeze); do pip install -U ${p%%=*}; done

Or, in a nicely-formatted way:

for p in $(pip list -o --format freeze)
do
    pip install -U ${p%%=*}
done

The rather amazing yolk makes this easy.

pip install yolk3k # Don't install `yolk`, see https://github.com/cakebread/yolk/issues/35
yolk --upgrade

For more information on yolk: https://pypi.python.org/pypi/yolk/0.4.3

It can do lots of things you'll probably find useful.

Use pipupgrade!

$ pip install pipupgrade
$ pipupgrade --verbose --latest --yes

pipupgrade helps you upgrade your system, local or packages from a requirements.txt file! It also selectively upgrades packages that don't break change.

pipupgrade also ensures to upgrade packages present within multiple Python environments. It is compatible with Python 2.7+, Python 3.4+ and pip 9+, pip 10+, pip 18+, pip 19+.

Note: I'm the author of the tool.

This ought to be more effective:

pip3 list -o | grep -v -i warning | cut -f1 -d' ' | tr " " "\n" | awk '{if(NR>=3)print}' | cut -d' ' -f1 | xargs -n1 pip3 install -U

1. pip list -o lists outdated packages;
2. grep -v -i warning inverted match on warning to avoid errors when updating
3. cut -f1 -d1' ' returns the first word - the name of the outdated package;
4. tr "\n|\r" " " converts the multiline result from cut into a single-line, space-separated list;
5. awk '{if(NR>=3)print}' skips header lines
6. cut -d' ' -f1 fetches the first column
7. xargs -n1 pip install -U takes 1 argument from the pipe left of it, and passes it to the command to upgrade the list of packages.

Sent through a pull-request to the pip folks; in the meantime use this pip library solution I wrote:

from pip import get_installed_distributions
from pip.commands import install

install_cmd = install.InstallCommand()

options, args = install_cmd.parse_args([package.project_name
                                        for package in
                                        get_installed_distributions()])

options.upgrade = True
install_cmd.run(options, args)  # Chuck this in a try/except and print as wanted

From https://github.com/cakebread/yolk:

$ pip install -U `yolk -U | awk '{print $1}' | uniq`

However, you need to get yolk first:

$ sudo pip install -U yolk

Ramana's answer worked the best for me, of those here, but I had to add a few catches:

import pip
for dist in pip.get_installed_distributions():
    if 'site-packages' in dist.location:
        try:
            pip.call_subprocess(['pip', 'install', '-U', dist.key])
        except Exception, exc:
            print exc

The site-packages check excludes my development packages, because they are not located in the system site-packages directory. The try-except simply skips packages that have been removed from PyPI.

To endolith: I was hoping for an easy pip.install(dist.key, upgrade=True), too, but it doesn't look like pip was meant to be used by anything but the command line (the docs don't mention the internal API, and the pip developers didn't use docstrings).

The shortest and easiest I can find:

pip install -U $(pip freeze | cut -d"=" -f1)

The $(cmd) key allows you to wrap any shell command line (it returns its output).

to upgrade all of your pip default packages in your default python version just run the bottom python code in your terminal or command prompt:

import subprocess
import re


pkg_list = subprocess.getoutput('pip freeze')

pkg_list = pkg_list.split('\n')


new_pkg = []
for i in pkg_list:
    re.findall(r"^(.*)==.*", str(i))
    new = re.findall(r"^(.*)==.*", str(i))[0]
    new_pkg.append(new)

for i in new_pkg:
    print(subprocess.getoutput('pip install '+str(i)+' --upgrade'))

I've been using pur lately. It's simple and to the point. It updates your requirements.txt file to reflect the upgrades and you can then upgrade with your requirements.txt file as usual.

$ pip install pur
...
Successfully installed pur-4.0.1

$ pur
Updated boto3: 1.4.2 -> 1.4.4
Updated Django: 1.10.4 -> 1.10.5
Updated django-bootstrap3: 7.1.0 -> 8.1.0
All requirements up-to-date.

$ pip install --upgrade -r requirements.txt
Successfully installed Django-1.10.5 ...

Here is a script that only updates the outdated packages.

import os, sys
from subprocess import check_output, call

file = check_output(["pip.exe",  "list", "--outdated", "--format=legacy"])
line = str(file).split()

for distro in line[::6]:
    call("pip install --upgrade " + distro, shell=True)

For a new version of pip that does not output as a legacy format (version 18+):

import os, sys
from subprocess import check_output, call

file = check_output(["pip.exe", "list", "-o", "--format=json"])
line = str(file).split()

for distro in line[1::8]:
    distro = str(distro).strip('"\",')
    call("pip install --upgrade " + distro, shell=True)

import os
import pip
from subprocess import call, check_call

pip_check_list = ['pip', 'pip3']
pip_list = []
FNULL = open(os.devnull, 'w')


for s_pip in pip_check_list:
    try:
        check_call([s_pip, '-h'], stdout=FNULL)
        pip_list.append(s_pip)
    except FileNotFoundError:
        pass


for dist in pip.get_installed_distributions():
    for pip in pip_list:
        call("{0} install --upgrade ".format(pip) + dist.project_name, shell=True)

I took Ramana's answer and made it pip3 friendly.

This is a PowerShell solution for Python 3:

pip3 list --outdated --format=legacy | ForEach { pip3 install -U $_.split(" ")[0] }

And for Python 2:

pip2 list --outdated --format=legacy | ForEach { pip2 install -U $_.split(" ")[0] }

This upgrades the packages one by one. So a

pip3 check
pip2 check

afterwards should make sure no dependencies are broken.

To upgrade all local packages; you could use pip-review:

$ pip install pip-review
$ pip-review --local --interactive

pip-review is a fork of pip-tools. See pip-tools issue mentioned by @knedlsepp. pip-review package works but pip-tools package no longer works.

pip-review works on Windows since version 0.5.

for in a bat script

call pip freeze > requirements.txt
call powershell "(Get-Content requirements.txt) | ForEach-Object { $_ -replace '==', '>=' } | Set-Content requirements.txt"
call pip install -r requirements.txt --upgrade