Why can t Python find shared objects that are in directories in sys path

Question

I m trying to import pycurl     python -c  import pycurl  Traceback  most recent call last   File   lt string gt    line 1  in  lt module gt  ImportError  libcurl so 4  cannot open shared object file  No such file or directory   Now  libcurl so 4 is in  usr local lib  As you can see  this is in sys path     python -c  import sys  print sys path          usr local lib python2 5 site-packages setuptools-0 6c9-py2 5 egg      usr local lib python25 zip     usr local lib python2 5      usr local lib python2 5 plat-linux2     usr local lib python2 5 lib-tk      usr local lib python2 5 lib-dynload      usr local lib python2 5 sitepackages     usr local lib      usr local lib python2 5 site-packages     Any help will be greatly appreciated

User · Answer

Ensure your libcurl so module is in the system library path  which is distinct and separate from the python library path   A  quick fix  is to add this path to a LD LIBRARY PATH variable  However  setting that system wide  or even account wide  is a BAD IDEA  as it is possible to set it in such a way that some programs will find a library it shouldn t  or even worse  open up security holes   If your  locally installed libraries  are installed in  for example   usr local lib  add this directory to  etc ld so conf  it s a text file  and run  ldconfig   The command will run a caching utility  but will also create all the necessary  symbolic links  required for the loader system to function  It is surprising that the  make install  for libcurl did not do this already  but it s possible it could not if  usr local lib is not in  etc ld so conf already   PS  it s possible that your  etc ld so conf contains nothing but  include ld so conf d   conf   You can still add a directory path after it  or just create a new file inside the directory it s being included from  Dont forget to run  ldconfig  after it   Be careful  Getting this wrong can screw up your system    Additionally  make sure your python module is compiled against THAT version of libcurl  If you just copied some files over from another system  this wont always work  If in doubt  compile your modules on the system you intend to run them on

User · Answer

As a supplement to above answers - I m just bumping into a similar problem  and working completely of the default installed python    When I call the example of the shared object library I m looking for with LD LIBRARY PATH  I get something like this      LD LIBRARY PATH  path to mysodir  LD LIBRARY PATH python example-so-user py python  can t open file  example-so-user py    Errno 2  No such file or directory   Notably  it doesn t even complain about the import - it complains about the source file   But if I force loading of the object using LD PRELOAD     LD PRELOAD  path to mysodir mypyobj so python example-so-user py python  error while loading shared libraries  libtiff so 5  cannot open shared object file  No such file or directory       I immediately get a more meaningful error message - about a missing dependency   Just thought I d jot this down here - cheers

User · Answer

Had the exact same issue  I installed curl 7 19 to  opt curl  to make sure that I would not affect current curl on our production servers   Once I linked libcurl so 4 to  usr lib   sudo ln -s  opt curl lib libcurl so  usr lib libcurl so 4  I still got the same error  Durf   But running ldconfig make the linkage for me and that worked  No need to set the LD RUN PATH or LD LIBRARY PATH at all  Just needed to run ldconfig

User · Answer

sys path is only searched for Python modules  For dynamic linked libraries  the paths searched must be in LD LIBRARY PATH  Check if your LD LIBRARY PATH includes  usr local lib  and if it doesn t  add it and try again   Some more information  source       In Linux  the environment variable   LD LIBRARY PATH is a colon-separated   set of directories where libraries   should be searched for first  before   the standard set of directories  this   is useful when debugging a new library   or using a nonstandard library for   special purposes  The environment   variable LD PRELOAD lists shared   libraries with functions that override   the standard set  just as    etc ld so preload does  These are   implemented by the loader    lib ld-linux so  I should note that    while LD LIBRARY PATH works on many   Unix-like systems  it doesn t work on   all  for example  this functionality   is available on HP-UX but as the   environment variable SHLIB PATH  and   on AIX this functionality is through   the variable LIBPATH  with the same   syntax  a colon-separated list     Update  to set LD LIBRARY PATH  use one of the following  ideally in your    bashrc  or equivalent file   export LD LIBRARY PATH  usr local lib   or  export LD LIBRARY PATH  usr local lib  LD LIBRARY PATH   Use the first form if it s empty  equivalent to the empty string  or not present at all   and the second form if it isn t  Note the use of export

User · Answer

You can also set LD RUN PATH to  usr local lib in your user environment when you compile pycurl in the first place  This will embed  usr local lib in the RPATH attribute of the C extension module  so so that it automatically knows where to find the library at run time without having to have LD LIBRARY PATH set at run time

User · Answer

I use python setup py build ext -R usr local lib -I usr local include libcalg-1 0 and the compiled  so file is under the build folder  you can type python setup py --help build ext to see the explanations of -R and -I

User · Answer

For me what works here is to using a version manager such as pyenv  which I strongly recommend to get your project environments and package versions well managed and separate from that of the operative system   I had this same error after an OS update  but was easily fixed with pyenv install 3 7-dev  the version I use

[python] Why can't Python find shared objects that are in directories in sys.path?

Examples related to python

Examples related to shared-libraries

Examples related to libcurl

Examples related to pycurl