How do I keep track of pip-installed packages in an Anaconda Conda environment

Question

I ve installed and have been using the Anaconda Python distribution  and I have started using the Anaconda  Conda  environment  I can use the standard conda install    command to put packages from the distribution into my environments  but to use anything outside  i e  Flask-WTF  flask-sqlalchemy  and alembic  I need to use pip install in the active environment  However  when I look at the contents of the environment  either in the directory  or using conda list these pip installed packages don t show up    Using pip freeze and pip list just lists every package I ve ever installed    Is there a way to keep track of what is in each of my Anaconda envs  both pip and conda installed

User · Answer

There is a branch of conda  new-pypi-install  that adds better integration with pip and PyPI    In particular conda list will also show pip installed packages and conda install will first try to find a conda package and failing that will use pip to install the package    This branch is scheduled to be merged later this week so that version 2 1 of conda will have better pip-integration with conda

User · Answer

My which pip shows the following path     which pip  home kmario23 anaconda3 bin pip   So  whatever package I install using pip install  lt package-name gt  will have to be reflected in the list of packages when the list is exported using     conda list --export  gt  conda list txt   But  I don t  So  instead I used the following command as suggested by several others     get environment name by   conda-env list    get list of all installed packages by  conda  pip  etc      conda-env export -n  lt my-environment-name gt   gt  all packages yml   if you haven t created any specific env  then just use  root    Now  I can see all the packages in my all-packages yml file

User · Answer

You can start by installing the below given command in the conda environment   conda install pip  Followed by installing all pip packages you need in the environment   After installing all the conda and pip packages to export the environment use   conda env export -n  lt env-name gt   gt  environment yml  This will create the required file in the folder

User · Answer

conda-env now does this automatically  if pip was installed with conda    You can see how this works by using the export tool used for migrating an environment   conda env export -n  lt env-name gt   gt  environment yml   The file will list both conda packages and pip packages   name  stats channels    - javascript dependencies    - python 3 4   - bokeh 0 9 2   - numpy 1 9     - nodejs 0 10     - flask   - pip      - Flask-Testing   If you re looking to follow through with exporting the environment  move environment yml to the new host machine and run   conda env create -f path to environment yml

User · Answer

I followed  Viktor Kerkez s answer and have had mixed success  I found that sometimes this recipe of      conda skeleton pypi PACKAGE      conda build PACKAGE   would look like everything worked but I could not successfully import PACKAGE  Recently I asked about this on the Anaconda user group and heard from  Travis Oliphant himself on the best way to use conda to build and manage packages that do not ship with Anaconda  You can read this thread here  but I ll describe the approach below to hopefully make the answers to the OP s question more complete     Example  I am going to install the excellent prettyplotlib package on Windows using conda 2 2 5   1a  conda build --build-recipe prettyplotlib  You ll see the build messages all look good until the final TEST section of the build  I saw this error     File  C  Anaconda conda-bld test-tmp dir run test py   line 23       import None SyntaxError  cannot assign to None TESTS FAILED  prettyplotlib-0 1 3-py27 0   1b  Go into  conda-recipes prettyplotlib and edit the meta yaml file  Presently  the packages being set up like in step 1a result in yaml files that have an error in the test section  For example  here is how mine looked for prettyplotlib  test      Python imports   imports      -      - prettyplotlib     - prettyplotlib   Edit this section to remove the blank line preceded by the - and also remove the redundant prettyplotlib line  At the time of this writing I have found that I need to edit most meta yaml files like this for external packages I am installing with conda  meaning that there is a blank import line causing the error along with a redundant import of the given package   1c  Rerun the command from 1a  which should complete with out error this time  At the end of the build you ll be asked if you want to upload the build to binstar  I entered No and then saw this message      If you want to upload this package to binstar org later  type         binstar upload C  Anaconda conda-bld win-64 prettyplotlib-0 1 3-py27 0 tar bz2   That tar bz2 file is the build that you now need to actually install   2  conda install C  Anaconda conda-bld win-64 prettyplotlib-0 1 3-py27 0 tar bz2  Following these steps I have successfully used conda to install a number of packages that do not come with Anaconda  Previously  I had installed some of these using pip  so I did pip uninstall PACKAGE prior to installing PACKAGE with conda  Using conda  I can now manage  almost  all of my packages with a single approach rather than having a mix of stuff installed with conda  pip  easy install  and python setup py install   For context  I think this recent blog post by  Travis Oliphant will be helpful for people like me who do not appreciate everything that goes into robust Python packaging but certainly appreciate when stuff  just works   conda seems like a great way forward

User · Answer

This is why I wrote Picky  http   picky readthedocs io   It s a python package that tracks packages installed with either pip or conda in either virtualenvs and conda envs

User · Answer

conda env export lists all conda and pip packages in an environment  conda-env must be installed in the conda root  conda install -c conda conda-env    To write an environment yml file describing the current environment   conda env export  gt  environment yml   References    https   github com conda conda-env  https   github com conda conda-env issues 12

User · Answer

I usually prefix the  bin pip  folder for the specific environment you want to install the package before the  pip  command  For instance  if you would like to install pymc3 in the environment py34  you should use this command     anaconda envs py34 bin pip install git https   github com pymc-devs pymc3    You basically just need to find the right path to your environment  bin pip  folder and put it before the install command

User · Answer

conda will only keep track of the packages it installed  And pip will give you the packages that were either installed using the pip installer itself or they used setuptools in their setup py so conda build generated the egg information  So you have basically three options    You can take the union of the conda list and pip freeze and manage packages that were installed using conda  that show in the conda list  with the conda package manager and the ones that are installed with pip  that show in pip freeze but not in conda list  with pip  Install in your environment only the python  pip and distribute packages and manage everything with pip   This is not that trivial if you re on Windows     Build your own conda packages  and manage everything with conda    I would personally recommend the third option since it s very easy to build conda packages  There is a git repository of example recipes on the continuum s github account  But it usually boils down to    conda skeleton pypi PACKAGE  conda build PACKAGE   or just   conda pipbuild PACKAGE   Also when you have built them once  you can upload them to https   binstar org  and just install from there   Then you ll have everything managed using conda

User · Answer

I think what s missing here is that when you do    gt pip install     to install a local package with a setup py  it installs a package that is visible to all the conda envs that use the same version of python  Note I am using the conda version of pip   e g   if I m using python2 7 it puts the local package here    usr local anaconda lib python2 7 site-packages  If I then later create a new conda env with python 2 7    the default     gt conda create --name new   gt source activate new   And then do    new  gt conda list       empty - conda is not aware of any packages yet   However  if I do    new  gt pip list         the local package installed above is present   So in this case  conda does not know about the pip package  but the package is available to python   However  If I instead install the local package  again using pip  after I ve created  and activated  the new conda env  now conda sees it    new  gt conda list      sees that the package is there and was installed by pip   So I think the interaction between conda and pip has some issues - ie  using pip to install a local package from within one conda env makes that package available  but not seen via conda list  to all other conda envs of the same python version

[python] How do I keep track of pip-installed packages in an Anaconda (Conda) environment?

Examples related to python

Examples related to pip

Examples related to anaconda

Examples related to env

Examples related to conda