Expand Python Search Path to Other Source

Question

I have just joined a project with a rather large existing code base  We develop in linux and do not use and IDE  We run through the command line  I m trying to figure out how to get python to search for the right path when I run project modules  For instance  when I run something like   python someprojectfile py   I get  ImportError  no module named core  somemodule    I get this for all of my imports to I assume it s an issue with the path   TLDR   How do I get Python to search   codez project  and all the files and folders for   py files during import statements

User · Answer

You should also read about python packages here: http://docs.python.org/tutorial/modules.html.

From your example, I would guess that you really have a package at ~/codez/project. The file __init__.py in a python directory maps a directory into a namespace. If your subdirectories all have an __init__.py file, then you only need to add the base directory to your PYTHONPATH. For example:

PYTHONPATH=$PYTHONPATH:$HOME/adaifotis/project

In addition to testing your PYTHONPATH environment variable, as David explains, you can test it in python like this:

$ python
>>> import project                      # should work if PYTHONPATH set
>>> import sys
>>> for line in sys.path: print line    # print current python path

...

User · Answer

There are a few possible ways to do this    Set the environment variable PYTHONPATH to a colon-separated list of directories to search for imported modules  In your program  use sys path append   path to search   to add the names of directories you want Python to search for imported modules  sys path is just the list of directories Python searches every time it gets asked to import a module  and you can alter it as needed  although I wouldn t recommend removing any of the standard directories    Any directories you put in the environment variable PYTHONPATH will be inserted into sys path when Python starts up  Use site addsitedir to add a directory to sys path  The difference between this and just plain appending is that when you use addsitedir  it also looks for  pth files within that directory and uses them to possibly add additional directories to sys path based on the contents of the files  See the documentation for more detail    Which one of these you want to use depends on your situation  Remember that when you distribute your project to other users  they typically install it in such a manner that the Python code files will be automatically detected by Python s importer  i e  packages are usually installed in the site-packages directory   so if you mess with sys path in your code  that may be unnecessary and might even have adverse effects when that code runs on another computer  For development  I would venture a guess that setting PYTHONPATH is usually the best way to go   However  when you re using something that just runs on your own computer  or when you have nonstandard setups  e g  sometimes in web app frameworks   it s not entirely uncommon to do something like  import sys from os path import dirname sys path append dirname   file

User · Answer

The easiest way I find is to create a file  any name pth  and put it in your folder   Lib site-packages   You should find that folder wherever python is installed   In that file  put a list of directories where you want to keep modules for importing  For instance  make a line in that file like this   C  Users example    example  You will be able to tell it works by running this in python   import sys for line in sys  print line   You will see your directory printed out  amongst others from where you can also import  Now you can import a  mymodule py  file that sits in that directory as easily as   import mymodule   This will not import subfolders  For that you could imagine creating a python script to create a  pth file containing all sub folders of a folder you define  Have it run at startup perhaps

User · Answer

New option for old question   Installing fail2ban package on Debian  looks like it s hardcoded to install on  usr lib python3 dist-packages fail2ban a path not on python3 sys path     gt  python3 Python 3 7 3  v3 7 3 ef4ec6ed12  Jun 25 2019  18 51 50   GCC 6 3 0 20170516  on linux Type  help    copyright    credits  or  license  for more information   gt  gt  gt  import sys  gt  gt  gt  sys path        usr lib python37 zip     usr lib python3 7     usr lib python3 7 lib-dynload     usr lib python3 7 site-packages    gt  gt  gt    so  instead of just copying  I  bash  linked the library to newer versions  Future updates to the original app  will also be automatically applied to the linked versions    if   -d  usr lib python3 dist-packages fail2ban      then       for d in  usr lib python3         do            -d   d  fail2ban                    ln -vs  usr lib python3 dist-packages fail2ban   d         done    fi

User · Answer

I know this thread is a bit old  but it took me some time to get to the heart of this  so I wanted to share   In my project  I had the main script in a parent directory  and  to differentiate the modules  I put all the supporting modules in a sub-folder called  modules    In my main script  I import these modules like this  for a module called report py    from modules report import report  reportError   If I call my main script  this works   HOWEVER  I wanted to test each module by including a main   in each  and calling each directly  as   python modules report py   Now Python complains that it can t find  a module called modules    The key here is that  by default  Python includes the folder of the script in its search path  BUT NOT THE CWD   So what this error says  really  is  I can t find a modules subfolder    The is because there is no  modules  subdirectory from the directory where the report py module resides   I find that the neatest solution to this is to append the CWD in Python search path by including this at the top   import sys  sys path append        Now Python searches the CWD  current directory   finds the  modules  sub-folder  and all is well

User · Answer

I read this question looking for an answer  and didn t like any of them   So I wrote a quick and dirty solution  Just put this somewhere on your sys path  and it ll add any directory under folder  from the current working directory   or under abspath    using py  import sys  os path  def all from folder     abspath None          add all dirs under  folder  to sys path if any  py files are found      Use an abspath if you d rather do it that way       Uses the current working directory as the location of using py       Keep in mind that os walk goes  all the way  down the directory tree      With that  try not to use this on something too close to                  add   set sys path      if abspath is None          cwd   os path abspath os path curdir          abspath   os path join cwd  folder      for root  dirs  files in os walk abspath           for f in files              if f -3   in   py                   add add root                  break     for i in add  sys path append i    gt  gt  gt  import using  sys  pprint  gt  gt  gt  using all from  py    if in     home user py   gt  gt  gt  pprint pprint sys path     that was easy     And I like it because I can have a folder for some random tools and not have them be a part of packages or anything  and still get access to some  or all  of them in a couple lines of code

[python] Expand Python Search Path to Other Source

Examples related to python

Examples related to search

Examples related to import

Examples related to path