Python - Get path of root project structure

Question

I ve got a python project with a configuration file in the project root   The configuration file needs to be accessed in a few different files throughout the project    So it looks something like   lt ROOT gt  configuration conf   lt ROOT gt  A a py   lt ROOT gt  A B b py  when b a py access the configuration file     What s the best   easiest way to get the path to the project root and the configuration file without depending on which file inside the project I m in  i e without using         It s okay to assume that we know the project root s name

User · Answer

Here s my take on this issue  I have a simple use-case that bugged me for a while  Tried a few solutions  but I didn t like either of them flexible enough  So here s what I figured out   create a blank python file in the root dir - gt  I call this beacon py  assuming that the project root is in the PYTHONPATH so it can be imported  add a few lines to my module class which I call here not in root py  This will import the beacon py module and get the path to that module  Here s an example project structure this project  -- beacon py  -- lv1         --   init   py         -- lv2             --   init   py             -- not in root py       The content of the not in root py import os from pathlib import Path   class Config      try          import beacon         print f quot  import beacon  - gt   os path dirname os path abspath beacon   file      quot      only for demo purposes         print f quot  import beacon  - gt   Path beacon   file    parent resolve    quot      only for demo purposes     except ModuleNotFoundError as e          print f quot ModuleNotFoundError  import beacon failed with  e    quot                f quot Please  create a file called beacon py and place it to the project root directory  quot        project root   Path beacon   file    parent resolve       input dir   project root    input      output dir   project root    output    if   name         main         c   Config       print f quot Config project root   c project root  quot       print f quot Config input dir   c input dir  quot       print f quot Config output dir   c output dir  quot    The output would be  home xyz projects this project venv bin python  home xyz projects this project lv1 lv2 not in root py  import beacon  - gt   home xyz projects this project  import beacon  - gt   home xyz projects this project Config project root   home xyz projects this project Config input dir   home xyz projects this project input Config output dir   home xyz projects this project output  Of course  it doesn t need to be called beacon py nor need to be empty  essentially any python file  importable  file would do as long as it s in the root directory  Using an empty  py file sort of guarantees that it will not be moved elsewhere due to some future refactoring  Cheers

User · Answer

At the time of writing  none of the other solutions are very self-contained  They depend either on an environment variable or the position of the module in the package structure  The top answer with the    Django    solution falls victim to the latter by requiring a relative import  It also has the disadvantage of having to modify a module at the top level   This should be the correct approach for finding the directory path of the top-level package   import sys import os  root name           name   partition      root module   sys modules root name  root dir   os path dirname root module   file     config path   os path join root dir   configuration conf     It works by taking the first component in the dotted string contained in   name   and using it as a key in sys modules which returns the module object of the top-level package  Its   file   attribute contains the path we want after trimming off    init   py using os path dirname     This solution is self-contained  It works anywhere in any module of the package  including in the top-level   init   py file

User · Answer

To get the path of the  root  module  you can use   import os import sys os path dirname sys modules    main       file      But more interestingly if you have an config  object  in your top-most module you could -read- from it like so   app   sys modules    main     stuff   app config somefunc

User · Answer

I struggled with this problem too until I came to this solution  This is the cleanest solution in my opinion   In your setup py add  packages   setup  name  package name  version  0 0 1        packages   package name             In your python script py  import pkg resources import os  resource package   pkg resources get distribution       package name   location config path   os path join resource package  configuration conf

User · Answer

If you are working with anaconda-project  you can query the PROJECT ROOT from the environment variable --  os getenv  PROJECT ROOT    This works only if the script is executed via anaconda-project run    If you do not want your script run by anaconda-project  you can query the absolute path of the executable binary of the Python interpreter you are using and extract the path string up to the envs directory exclusiv  For example  The python interpreter of my conda env is located at        home user project root envs default bin python     You can first retrieve the env variable PROJECT DIR    If not set  get the python interpreter location and strip off the string till envs inclusiv     if os getenv  PROJECT DIR        PROJECT DIR   os getenv  PROJECT DIR   else      PYTHON PATH   sys executable     path rem   os path join  envs    default    bin    python       PROJECT DIR   py path split path rem  0    This works only with conda-project with fixed project structure of a anaconda-project

User · Answer

Try   ROOT DIR   os path dirname os path dirname os path abspath   file

User · Answer

I ve recently been trying to do something similar and I have found these answers inadequate for my use cases  a distributed library that needs to detect project root   Mainly I ve been battling different environments and platforms  and still haven t found something perfectly universal   Code local to project  I ve seen this example mentioned and used in a few places  Django  etc   import os print os path dirname os path abspath   file        Simple as this is  it only works when the file that the snippet is in is actually part of the project  We do not retrieve the project directory  but instead the snippet s directory   Similarly  the sys modules approach breaks down when called from outside the entrypoint of the application  specifically I ve observed a child thread cannot determine this without relation back to the  main  module  I ve explicitly put the import inside a function to demonstrate an import from a child thread  moving it to top level of app py would fix it   app   -- config      --   init   py      -- settings py  -- app py   app py     usr bin env python import threading   def background setup          Explicitly importing this from the context of the child thread     from config import settings     print settings ROOT DIR      Spawn a thread to background preparation tasks t   threading Thread target background setup  t start      Do other things during initialization  t join      Ready to take traffic   settings py  import os import sys   ROOT DIR   None   def setup        global ROOT DIR     ROOT DIR   os path dirname sys modules    main       file          Do something slow   Running this program produces an attribute error    gt  gt  gt  import main  gt  gt  gt  Exception in thread Thread-1  Traceback  most recent call last     File  C  Python2714 lib threading py   line 801  in   bootstrap inner     self run     File  C  Python2714 lib threading py   line 754  in run     self   target  self   args    self   kwargs    File  main py   line 6  in background setup     from config import settings   File  config settings py   line 34  in  lt module gt      ROOT DIR   get root     File  config settings py   line 31  in get root     return os path dirname sys modules    main       file    AttributeError   module  object has no attribute    file         hence a threading-based solution  Location independent  Using the same application structure as before but modifying settings py  import os import sys import inspect import platform import threading   ROOT DIR   None   def setup        main id   None     for t in threading enumerate            if t name     MainThread               main id   t ident             break      if not main id          raise RuntimeError  Main thread exited before execution        current main frame   sys  current frames   main id      base frame   inspect getouterframes current main frame  -1       if platform system       Windows           filename   base frame filename     else          filename   base frame 0  f code co filename      global ROOT DIR     ROOT DIR   os path dirname os path abspath filename     Breaking this down  First we want to accurately find the thread ID of the main thread  In Python3 4  the threading library has threading main thread   however  everybody doesn t use 3 4  so we search through all threads looking for the main thread save it s ID  If the main thread has already exited  it won t be listed in the threading enumerate    We raise a RuntimeError   in this case until I find a better solution   main id   None for t in threading enumerate        if t name     MainThread           main id   t ident         break  if not main id      raise RuntimeError  Main thread exited before execution     Next we find the very first stack frame of the main thread  Using the cPython specific function sys  current frames   we get a dictionary of every thread s current stack frame  Then utilizing inspect getouterframes   we can retrieve the entire stack for the main thread and the very first frame      current main frame   sys  current frames   main id      base frame   inspect getouterframes current main frame  -1  Finally  the differences between Windows and Linux implementations of inspect getouterframes   need to be handled  Using the cleaned up filename  os path abspath   and os path dirname   clean things up   if platform system       Windows       filename   base frame filename else      filename   base frame 0  f code co filename  global ROOT DIR ROOT DIR   os path dirname os path abspath filename     So far I ve tested this on Python2 7 and 3 6 on Windows as well as Python3 4 on WSL

User · Answer

I had to implement a custom solution because it s not as simple as you might think  My solution is based on stack trace inspection  inspect stack      sys path and is working fine no matter the location of the python module in which the function is invoked nor the interpreter  I tried by running it in PyCharm  in a poetry shell and other      This is the full implementation with comments  def get project root dir   - gt  str       quot  quot  quot      Returns the name of the project root directory        return  Project root directory name      quot  quot  quot         stack trace history related to the call of this function     frame stack   FrameInfo    inspect stack          get info about the module that has invoked this function        index 0 is always this very module  index 1 is fine as long this function is not called by some other       function in this module      frame info  FrameInfo   frame stack 1         if there are multiple calls in the stacktrace of this very module  we have to skip those and take the first       one which comes from another module     if frame info filename      file            for frame in frame stack              if frame filename      file                    frame info   frame                 break        path of the module that has invoked this function     caller path  str   frame info filename        absolute path of the of the module that has invoked this function     caller absolute path  str   os path abspath caller path         get the top most directory path which contains the invoker module     paths   str     p for p in sys path if p in caller absolute path      paths sort key lambda p  len p       caller root path  str   paths 0       if not os path isabs caller path             file name of the invoker module  eg   quot mymodule py quot           caller module name  str   Path caller path  name            this piece represents a subpath in the project directory            eg  if the root folder is  quot myproject quot  and this function has ben called from myproject foo bar mymodule py           this will be  quot foo bar quot           project related folders  str   caller path replace os sep   caller module name                 fix root path by removing the undesired subpath         caller root path   caller root path replace project related folders           dir name  str   Path caller root path  name      return dir name

User · Answer

Below Code Returns the path until your project root  import sys print sys path 1

User · Answer

I decided for myself as follows  Need to get the path to  MyProject drivers  from the main file  MyProject   --- RootPackge         --   init   py        -- main py        -- definitions py     --- drivers         -- geckodriver exe     -- requirements txt  -- setup py  definitions py Put not in the root of the project  but in the root of the main package from pathlib import Path  ROOT DIR   Path   file    parent parent  Use ROOT DIR  main py   imports must be relative    not from the root of the project    but from the root of the main package    Not this way    from RootPackge definitions import ROOT DIR   But like this  from definitions import ROOT DIR    Here we use ROOT DIR   get path to MyProject drivers drivers dir   ROOT DIR    drivers    Thus  you can get the path to any directory   or file from the project root  driver   webdriver Firefox drivers dir  driver get  http   www google com    Then PYTHON PATH will not be used to access the  definitions py  file  Works in PyCharm  run file  main py   ctrl   shift   F10 in Windows  Works in CLI from project root    py RootPackge main py  Works in CLI from RootPackge    cd RootPackge   py main py  Works from directories above project    cd                py MyWork PythoProjects MyProject RootPackge main py  Works from anywhere if you give an absolute path to the main file  Doesn t depend on venv

User · Answer

I used the     method to fetch the current project path   Example  Project1 -- D  projects  src  ConfigurationFiles  Configuration cfg  Path     src ConfigurationFiles Configuration cfg

User · Answer

Other answers advice to use a file in the top-level of the project  This is not necessary if you use pathlib Path and parent  Python 3 4 and up   Consider the following directory structure where all files except README md and utils py have been omitted  project      README md    ---src           utils py                      In utils py we define the following function  from pathlib import Path  def get project root   - gt  Path      return Path   file    parent parent  In any module in the project we can now get the project root as follows  from src utils import get project root  root   get project root    Benefits  Any module which calls get project root can be moved without changing program behavior  Only when the module utils py is moved we have to update get project root and the imports  refactoring tools can be used to automate this

User · Answer

This worked for me using a standard PyCharm project with my virtual environment  venv  under the project root directory   Code below isnt the prettiest  but consistently gets the project root  It returns the full directory path to venv from the VIRTUAL ENV environment variable e g   Users NAME documents PROJECT venv  It then splits the path at the last    giving an array with two elements  The first element will be the project path e g   Users NAME documents PROJECT   import os  print os path split os environ  VIRTUAL ENV    0

User · Answer

A standard way to achieve this would be to use the pkg resources module which is part of the setuptools package   setuptools is used to create an install-able python package   You can use pkg resources to return the contents of your desired file as a string and you can use pkg resources to get the actual path of the desired file on your system   Let s say that you have a package called stackoverflow   stackoverflow   -- app      --   init   py  -- resources      -- bands          -- Dream  Theater          --   init   py          -- King s  X          -- Megadeth          -- Rush      --   init   py  3 directories  7 files   Now let s say that you want to access the file Rush from a module app run  Use pkg resources resouces filename to get the path to Rush and pkg resources resource string to get the contents of Rush  thusly   import pkg resources  if   name         main         print pkg resources resource filename  resources bands    Rush       print pkg resources resource string  resources bands    Rush     The output    home sri workspace stackoverflow resources bands Rush Base  Geddy Lee Vocals  Geddy Lee Guitar  Alex Lifeson Drums  Neil Peart   This works for all packages in your python path  So if you want to know where lxml etree exists on your system   import pkg resources  if   name         main         print pkg resources resource filename  lxml    etree     output    usr lib64 python2 7 site-packages lxml etree   The point is that you can use this standard method to access files that are installed on your system  e g pip install xxx or yum -y install python-xxx  and files that are within the module that you re currently working on

User · Answer

All the previous solutions seem to be overly complicated for what I think you need  and often didn t work for me  The following one-line command does what you want   import os ROOT DIR   os path abspath os curdir

User · Answer

Just an example  I want to run runio py from within helper1 py   Project tree example   myproject root - modules dir helpers dir helper1 py - tools dir runio py   Get project root   import os rootdir   os path dirname os path realpath   file     rsplit os sep  2  0    Build path to script   runme   os path join rootdir   tools dir    runio py   execfile runme

User · Answer

There are many answers here but I couldn t find something simple that covers all cases so allow me to suggest my solution too    x000D   x000D  import pathlib x000D  import os x000D   x000D  def get project root    x000D          x000D      There is no way in python to get project root  This function uses a trick  x000D      We know that the function that is currently running is in the project  x000D      We know that the root project path is in the list of PYTHONPATH x000D      look for any path in PYTHONPATH list that is contained in this function s path x000D      Lastly we filter and take the shortest path because we are looking for the root  x000D       return  path to project root x000D          x000D      apth   str pathlib Path   absolute    x000D      ppth   os environ  PYTHONPATH   split      x000D      matches    x for x in ppth if x in apth  x000D      project root   min matches  key len  x000D      return project root x000D   x000D   x000D

User · Answer

You can do this how Django does it  define a variable to the Project Root from a file that is in the top-level of the project  For example  if this is what your project structure looks like   project      configuration conf     definitions py     main py     utils py   In definitions py you can define  this requires import os    ROOT DIR   os path dirname os path abspath   file       This is your Project Root   Thus  with the Project Root known  you can create a variable that points to the location of the configuration  this can be defined anywhere  but a logical place would be to put it in a location where constants are defined - e g  definitions py    CONFIG PATH   os path join ROOT DIR   configuration conf      requires  import os    Then  you can easily access the constant  in any of the other files  with the import statement  e g  in utils py   from definitions import CONFIG PATH

[python] Python - Get path of root project structure

Examples related to python