Multiple glibc libraries on a single host

Question

My linux  SLES-8  server currently has glibc-2 2 5-235  but I have a program which won t work on this version and requires glibc-2 3 3  Is it possible to have multiple glibcs installed on the same host  This is the error I get when I run my program on the old glibc    myapp   lib i686 libc so 6  version  GLIBC 2 3  not found  required by   myapp    myapp   lib i686 libpthread so 0  version  GLIBC 2 3 2  not found  required by   myapp    myapp   lib i686 libc so 6  version  GLIBC 2 3  not found  required by   libxerces-c so 27    myapp   lib ld-linux so 2  version  GLIBC 2 3  not found  required by   libstdc   so 6    myapp   lib i686 libc so 6  version  GLIBC 2 3  not found  required by   libstdc   so 6   So I created a new directory called newglibc and copied the following files in  libpthread so 0 libm so 6 libc so 6 ld-2 3 3 so ld-linux so 2 - gt  ld-2 3 3 so  and export LD LIBRARY PATH newglibc  LD LIBRARY PATH  But I get an error    myapp   lib ld-linux so 2  version  GLIBC PRIVATE  not found  required by   newglibc libpthread so 0    myapp   lib ld-linux so 2  version  GLIBC 2 3  not found  required by libstdc   so 6    myapp   lib ld-linux so 2  version  GLIBC PRIVATE  not found  required by   newglibc libm so 6    myapp   lib ld-linux so 2  version  GLIBC 2 3  not found  required by   newglibc libc so 6    myapp   lib ld-linux so 2  version  GLIBC PRIVATE  not found  required by   newglibc libc so 6   So it appears that they are still linking to  lib and not picking up from where I put them

User · Answer

First of all  the most important dependency of each dynamically linked program is the linker  All so libraries must match the version of the linker   Let s take simple exaple  I have the newset ubuntu system where I run some program  in my case it is D compiler - ldc2   I d like to run it on the old CentOS  but because of the older glibc library it is impossible  I got   ldc2-1 5 0-linux-x86 64 bin ldc2   lib64 libc so 6  version  GLIBC 2 15  not found  required by ldc2-1 5 0-linux-x86 64 bin ldc2  ldc2-1 5 0-linux-x86 64 bin ldc2   lib64 libc so 6  version  GLIBC 2 14  not found  required by ldc2-1 5 0-linux-x86 64 bin ldc2    I have to copy all dependencies from ubuntu to centos  The proper method is following   First  let s check all dependencies   ldd ldc2-1 5 0-linux-x86 64 bin ldc2      linux-vdso so 1   gt    0x00007ffebad3f000      librt so 1   gt   lib x86 64-linux-gnu librt so 1  0x00007f965f597000      libpthread so 0   gt   lib x86 64-linux-gnu libpthread so 0  0x00007f965f378000      libz so 1   gt   lib x86 64-linux-gnu libz so 1  0x00007f965f15b000      libdl so 2   gt   lib x86 64-linux-gnu libdl so 2  0x00007f965ef57000      libm so 6   gt   lib x86 64-linux-gnu libm so 6  0x00007f965ec01000      libgcc s so 1   gt   lib x86 64-linux-gnu libgcc s so 1  0x00007f965e9ea000      libc so 6   gt   lib x86 64-linux-gnu libc so 6  0x00007f965e60a000       lib64 ld-linux-x86-64 so 2  0x00007f965f79f000    linux-vdso so 1 is not a real library and we don t have to care about it    lib64 ld-linux-x86-64 so 2 is the linker  which is used by the linux do link the executable with all dynamic libraries   Rest of the files are real libraries and all of them together with the linker must be copied somewhere in the centos   Let s assume all the libraries and linker are in   mylibs  directory   ld-linux-x86-64 so 2 - as I ve already said - is the linker  It s not dynamic library but static executable  You can run it and see that it even have some parameters  eg --library-path  I ll return to it    On the linux  dynamically linked program may be lunched just by its name  eg   bin ldc2   Linux loads such program into RAM  and checks which linker is set for it  Usually  on 64-bit system  it is  lib64 ld-linux-x86-64 so 2  in your filesystem it is symbolic link to the real executable   Then linux runs the linker and it loads dynamic libraries   You can also change this a little and do such trick    mylibs ld-linux-x86-64 so 2  bin ldc2   It is the method for forcing the linux to use specific linker   And now we can return to the mentioned earlier parameter --library-path   mylibs ld-linux-x86-64 so 2 --library-path  mylibs  bin ldc2   It will run ldc2 and load dynamic libraries from  mylibs   This is the method to call the executable with choosen  not system default  libraries

User · Answer

This question is old  the other answers are old   quot Employed Russian quot s answer is very good and informative  but it only works if you have the source code  If you don t  the alternatives back then were very tricky  Fortunately nowadays we have a simple solution to this problem  as commented in one of his replies   using patchelf  All you have to do is      patchelf --set-interpreter  path to newglibc ld-linux so 2 --set-rpath  path to newglibc  myapp  And after that  you can just execute your file      myapp  No need to chroot or manually edit binaries  thankfully  But remember to backup your binary before patching it  if you re not sure what you re doing  because it modifies your binary file  After you patch it  you can t restore the old path to interpreter rpath  If it doesn t work  you ll have to keep patching it until you find the path that will actually work    Well  it doesn t have to be a trial-and-error process  For example  in OP s example  he needed GLIBC 2 3  so you can easily find which lib provides that version using strings    strings  lib i686 libc so 6   grep GLIBC 2 3   strings  path to newglib libc so 6   grep GLIBC 2 3  In theory  the first grep would come empty because the system libc doesn t have the version he wants  and the 2nd one should output GLIBC 2 3 because it has the version myapp is using  so we know we can patchelf our binary using that path  If you get a segmentation fault  read the note at the end  When you try to run a binary in linux  the binary tries to load the linker  then the libraries  and they should all be in the path and or in the right place  If your problem is with the linker and you want to find out which path your binary is looking for  you can find out with this command    readelf -l myapp   grep interpreter    Requesting program interpreter   lib ld-linux so 2                                                                                                                                                                                      If your problem is with the libs  commands that will give you the libs being used are    readelf -d myapp   grep Shared   ldd myapp   This will list the libs that your binary needs  but you probably already know the problematic ones  since they are already yielding errors as in OP s case   quot patchelf quot  works for many different problems that you may encounter while trying to run a program  related to these 2 problems  For example  if you get  ELF file OS ABI invalid  it may be fixed by setting a new loader  the --set-interpreter part of the command  as I explain here  Another example is for the problem of getting No such file or directory when you run a file that is there and executable  as exemplified here  In that particular case  OP was missing a link to the loader  but maybe in your case you don t have root access and can t create the link  Setting a new interpreter would solve your problem  Thanks Employed Russian and Michael Pankov for the insight and solution   Note for segmentation fault  you might be in the case where myapp uses several libs  and most of them are ok but some are not  then you patchelf it to a new dir  and you get segmentation fault  When you patchelf your binary  you change the path of several libs  even if some were originally in a different path  Take a look at my example below    ldd myapp   myapp   usr lib x86 64-linux-gnu libstdc   so 6  version  GLIBCXX 3 4 20  not found  required by   myapp    myapp   usr lib x86 64-linux-gnu libstdc   so 6  version  GLIBCXX 3 4 21  not found  required by   myapp          linux-vdso so 1   gt    0x00007fffb167c000          libm so 6   gt   lib x86 64-linux-gnu libm so 6  0x00007f9a9aad2000          libdl so 2   gt   lib x86 64-linux-gnu libdl so 2  0x00007f9a9a8ce000          libpthread so 0   gt   lib x86 64-linux-gnu libpthread so 0  0x00007f9a9a6af000          libstdc   so 6   gt   usr lib x86 64-linux-gnu libstdc   so 6  0x00007f9a9a3ab000          libc so 6   gt   lib x86 64-linux-gnu libc so 6  0x00007f9a99fe6000           lib64 ld-linux-x86-64 so 2  0x00007f9a9adeb000          libgcc s so 1   gt   lib x86 64-linux-gnu libgcc s so 1  0x00007f9a99dcf000   Note that most libs are in  lib x86 64-linux-gnu  but the problematic one  libstdc   so 6  is on  usr lib x86 64-linux-gnu  After I patchelf ed myapp to point to  path to mylibs  I got segmentation fault  For some reason  the libs are not totally compatible with the binary  Since myapp didn t complain about the original libs  I copied them from  lib x86 64-linux-gnu  to  path to mylibs2  and I also copied libstdc   so 6 from  path to mylibs there  Then I patchelf ed it to  path to mylibs2  and myapp works now  If your binary uses different libs  and you have different versions  it might happen that you can t fix your situation     But if it s possible  mixing libs might be the way  It s not ideal  but maybe it will work  Good luck

User · Answer

Use LD PRELOAD  put your library somewhere out of the man lib directories and run   LD PRELOAD  mylibc so anotherlib so  program   See  the Wikipedia article

User · Answer

Can you consider using Nix http   nixos org nix        Nix supports multi-user package management  multiple users can share a   common Nix store securely  don   t need to have root privileges to   install software  and can install and use different versions of a   package

User · Answer

I am not sure that the question is still relevant  but there is another way of fixing the problem  Docker  One can install an almost empty container of the Source Distribution  The Distribution used for development  and copy the files into the Container  That way You do not need to create the filesystem needed for chroot

User · Answer

If you look closely at the second output you can see that the new location for the libraries is used  Maybe there are still missing libraries that are part of the glibc   I also think that all the libraries used by your program should be compiled against that version of glibc  If you have access to the source code of the program  a fresh compilation appears to be the best solution

User · Answer

Setup 1  compile your own glibc without dedicated GCC and use it  This setup might work and is quick as it does not recompile the whole GCC toolchain  just glibc   But it is not reliable as it uses host C runtime objects such as crt1 o  crti o  and crtn o provided by glibc  This is mentioned at  https   sourceware org glibc wiki Testing Builds action recall amp rev 21 Compile against glibc in an installed location Those objects do early setup that glibc relies on  so I wouldn t be surprised if things crashed in wonderful and awesomely subtle ways   For a more reliable setup  see Setup 2 below   Build glibc and install locally   export glibc install    pwd  glibc build install   git clone git   sourceware org git glibc git cd glibc git checkout glibc-2 28 mkdir build cd build    configure --prefix   glibc install  make -j  nproc  make install -j  nproc    Setup 1  verify the build  test glibc c   define  GNU SOURCE  include  lt assert h gt   include  lt gnu libc-version h gt   include  lt stdatomic h gt   include  lt stdio h gt   include  lt threads h gt   atomic int acnt  int cnt   int f void  thr data        for int n   0  n  lt  1000    n              cnt            acnt            return 0     int main int argc  char   argv           Basic library version check         printf  gnu get libc version      s n   gnu get libc version             Exercise thrd create from -pthread         which is not present in glibc 2 27 in Ubuntu 18 04         https   stackoverflow com questions 56810 how-do-i-start-threads-in-plain-c 52453291 52453291        thrd t thr 10       for int n   0  n  lt  10    n          thrd create  amp thr n   f  NULL       for int n   0  n  lt  10    n          thrd join thr n   NULL       printf  The atomic counter is  u n   acnt       printf  The non-atomic counter is  u n   cnt       Compile and run with test glibc sh      usr bin env bash set -eux gcc     -L    glibc install  lib      -I    glibc install  include      -Wl --rpath    glibc install  lib      -Wl --dynamic-linker    glibc install  lib ld-linux-x86-64 so 2      -std c11     -o test glibc out     -v     test glibc c     -pthread     ldd   test glibc out   test glibc out   The program outputs the expected   gnu get libc version     2 28 The atomic counter is 10000 The non-atomic counter is 8674   Command adapted from https   sourceware org glibc wiki Testing Builds action recall amp rev 21 Compile against glibc in an installed location but --sysroot made it fail with   cannot find  home ciro glibc build install lib libc so 6 inside  home ciro glibc build install   so I removed it   ldd output confirms that the ldd and libraries that we ve just built are actually being used as expected     ldd test glibc out         linux-vdso so 1  0x00007ffe4bfd3000          libpthread so 0   gt   home ciro glibc build install lib libpthread so 0  0x00007fc12ed92000          libc so 6   gt   home ciro glibc build install lib libc so 6  0x00007fc12e9dc000           home ciro glibc build install lib ld-linux-x86-64 so 2   gt   lib64 ld-linux-x86-64 so 2  0x00007fc12f1b3000    The gcc compilation debug output shows that my host runtime objects were used  which is bad as mentioned previously  but I don t know how to work around it  e g  it contains   COLLECT GCC OPTIONS  usr lib gcc x86 64-linux-gnu 7          x86 64-linux-gnu crt1 o   Setup 1  modify glibc  Now let s modify glibc with   diff --git a nptl thrd create c b nptl thrd create c index 113ba0d93e  b00f088abb 100644 --- a nptl thrd create c     b nptl thrd create c    -16 11  16 14        License along with the GNU C Library  if not  see      lt http   www gnu org licenses  gt          include  lt stdio h gt      include  thrd priv h    int  thrd create  thrd t  thr  thrd start t func  void  arg        puts  hacked        Static assert  sizeof  thr     sizeof  pthread t                       sizeof  thr     sizeof  pthread t       Then recompile and re-install glibc  and recompile and re-run our program   cd glibc build make -j  nproc  make -j  nproc  install   test glibc sh   and we see hacked printed a few times as expected   This further confirms that we actually used the glibc that we compiled and not the host one   Tested on Ubuntu 18 04   Setup 2  crosstool-NG pristine setup  This is an alternative to setup 1  and it is the most correct setup I ve achieved far  everything is correct as far as I can observe  including the C runtime objects such as crt1 o  crti o  and crtn o   In this setup  we will compile a full dedicated GCC toolchain that uses the glibc that we want   The only downside to this method is that the build will take longer  But I wouldn t risk a production setup with anything less   crosstool-NG is a set of scripts that downloads and compiles everything from source for us  including GCC  glibc and binutils   Yes the GCC build system is so bad that we need a separate project for that   This setup is only not perfect because crosstool-NG does not support building the executables without extra -Wl flags  which feels weird since we ve built GCC itself  But everything seems to work  so this is only an inconvenience   Get crosstool-NG  configure and build it   git clone https   github com crosstool-ng crosstool-ng cd crosstool-ng git checkout a6580b8e8b55345a5a342b5bd96e42c83e640ac5 export CT PREFIX    pwd   build install  export PATH   usr lib ccache   PATH     bootstrap   configure --enable-local make -j  nproc    ct-ng x86 64-unknown-linux-gnu   ct-ng menuconfig env -u LD LIBRARY PATH time   ct-ng build CT JOBS  nproc    The build takes about thirty minutes to two hours   The only mandatory configuration option that I can see  is making it match your host kernel version to use the correct kernel headers  Find your host kernel version with   uname -a   which shows me   4 15 0-34-generic   so in menuconfig I do    Operating System   Version of linux    so I select   4 14 71   which is the first equal or older version  It has to be older since the kernel is backwards compatible   Setup 2  optional configurations  The  config that we generated with   ct-ng x86 64-unknown-linux-gnu has   CT GLIBC V 2 27 y   To change that  in menuconfig do    C-library Version of glibc   save the  config  and continue with the build   Or  if you want to use your own glibc source  e g  to use glibc from the latest git  proceed like this    Paths and misc options   Try features marked as EXPERIMENTAL  set to true  C-library   Source of glibc   Custom location  say yes Custom location   Custom source location  point to a directory containing your glibc source      where glibc was cloned as   git clone git   sourceware org git glibc git cd glibc git checkout glibc-2 28   Setup 2  test it out  Once you have built he toolchain that you want  test it out with      usr bin env bash set -eux install dir    CT PREFIX  x86 64-unknown-linux-gnu  PATH    PATH    install dir  bin      x86 64-unknown-linux-gnu-gcc     -Wl --dynamic-linker    install dir  x86 64-unknown-linux-gnu sysroot lib ld-linux-x86-64 so 2      -Wl --rpath    install dir  x86 64-unknown-linux-gnu sysroot lib      -v     -o test glibc out     test glibc c     -pthread     ldd test glibc out   test glibc out   Everything seems to work as in Setup 1  except that now the correct runtime objects were used   COLLECT GCC OPTIONS  home ciro crosstool-ng  build install x86 64-unknown-linux-gnu bin    x86 64-unknown-linux-gnu sysroot usr lib    lib64 crt1 o   Setup 2  failed efficient glibc recompilation attempt  It does not seem possible with crosstool-NG  as explained below   If you just re-build   env -u LD LIBRARY PATH time   ct-ng build CT JOBS  nproc    then your changes to the custom glibc source location are taken into account  but it builds everything from scratch  making it unusable for iterative development   If we do     ct-ng list-steps   it gives a nice overview of the build steps   Available build steps  in order    - companion tools for build   - companion libs for build   - binutils for build   - companion tools for host   - companion libs for host   - binutils for host   - cc core pass 1   - kernel headers   - libc start files   - cc core pass 2   - libc   - cc for build   - cc for host   - libc post cc   - companion libs for target   - binutils for target   - debug   - test suite   - finish Use   lt step gt   as action to execute only that step  Use    lt step gt   as action to execute up to that step  Use   lt step gt    as action to execute from that step onward    therefore  we see that there are glibc steps intertwined with several GCC steps  most notably libc start files comes before cc core pass 2  which is likely the most expensive step together with cc core pass 1   In order to build just one step  you must first set the  Save intermediate steps  in  config option for the intial build    Paths and misc options   Debug crosstool-NG   Save intermediate steps     and then you can try   env -u LD LIBRARY PATH time   ct-ng libc  -j nproc    but unfortunately  the   required as mentioned at  https   github com crosstool-ng crosstool-ng issues 1033 issuecomment-424877536     Note however that restarting at an intermediate step resets the installation directory to the state it had during that step  I e   you will have a rebuilt libc - but no final compiler built with this libc  and hence  no compiler libraries like libstdc   either     and basically still makes the rebuild too slow to be feasible for development  and I don t see how to overcome this without patching crosstool-NG   Furthermore  starting from the libc step didn t seem to copy over the source again from Custom source location  further making this method unusable   Bonus  stdlibc    A bonus if you re also interested in the C   standard library  How to edit and re-build the GCC libstdc   C   standard library source

User · Answer

Employed Russian  is among the best answer  and I think all other suggested answer may not work    The reason is simply because when an application is first created  all its the APIs it needs are resolved at compile time    Using  ldd  u can see all the statically linked dependencies   ldd  usr lib firefox firefox     linux-vdso so 1   gt    0x00007ffd5c5f0000      libpthread so 0   gt   lib x86 64-linux-gnu libpthread so 0  0x00007f727e708000      libdl so 2   gt   lib x86 64-linux-gnu libdl so 2  0x00007f727e500000      libstdc   so 6   gt   usr lib x86 64-linux-gnu libstdc   so 6  0x00007f727e1f8000      libm so 6   gt   lib x86 64-linux-gnu libm so 6  0x00007f727def0000      libc so 6   gt   lib x86 64-linux-gnu libc so 6  0x00007f727db28000       lib64 ld-linux-x86-64 so 2  0x00007f727eb78000      libgcc s so 1   gt   lib x86 64-linux-gnu libgcc s so 1  0x00007f727d910000    But at runtime  firefox will also load many other dynamic libraries  eg  for firefox  there are many  glib -labelled libraries loaded  even though statically linked there are none      usr lib x86 64-linux-gnu libdbus-glib-1 so 2 2 2   lib x86 64-linux-gnu libglib-2 0 so 0 4002 0   usr lib x86 64-linux-gnu libavahi-glib so 1 0 2   Manytimes  you can see names of one version being soft-linked into another version   Eg   lrwxrwxrwx 1 root root     23 Dec 21  2014 libdbus-glib-1 so 2 - gt  libdbus-glib-1 so 2 2 2 -rw-r--r-- 1 root root 160832 Mar  1  2013 libdbus-glib-1 so 2 2 2   This therefore means different version of  libraries  exists in one system - which is not a problem as it is the same file  and it will provide compatibilities when applications have multiple versions dependencies   Therefore  at the system level  all the libraries are almost interdependent on one another  and just changing the libraries loading priority via manipulating LD PRELOAD or LD LIBRARY PATH will not help - even it can load  runtime it may still crash   http   lightofdawn org wiki wiki cgi -wiki NewAppsOnOldGlibc  Best alternative is chroot  mentioned by ER briefly     but for this you will need to recreate the entire environment in which is the original binary execute - usually starting from  lib   usr lib    usr lib x86 etc    You can either use  Buildroot   or YoctoProject  or just tar from an existing Distro environment     like Fedora Suse etc

User · Answer

It is very possible to have multiple versions of glibc on the same system  we do that every day   However  you need to know that glibc consists of many pieces  200  shared libraries  which all must match  One of the pieces is ld-linux so 2  and it must match libc so 6  or you ll see the errors you are seeing  The absolute path to ld-linux so 2 is hard-coded into the executable at link time  and can not be easily changed after the link is done  Update  can be done with  patchelf  see this answer below   To build an executable that will work with the new glibc  do this  g   main o -o myapp          -Wl --rpath  path to newglibc      -Wl --dynamic-linker  path to newglibc ld-linux so 2  The -rpath linker option will make the runtime loader search for libraries in  path to newglibc  so you wouldn t have to set LD LIBRARY PATH before running it   and the -dynamic-linker option will  quot bake quot  path to correct ld-linux so 2 into the application  If you can t relink the myapp application  e g  because it is a third-party binary   not all is lost  but it gets trickier  One solution is to set a proper chroot environment for it  Another possibility is to use rtldi and a binary editor  Update  or you can use patchelf

User · Answer

msb gives a safe solution   I met this problem when I did import tensorflow as tf in conda environment in CentOS 6 5 which only has glibc-2 12    ImportError   lib64 libc so 6  version  GLIBC 2 16  not found  required by  home    I want to supply some details   First install glibc to your home directory   mkdir   glibc-install  cd   glibc-install wget http   ftp gnu org gnu glibc glibc-2 17 tar gz tar -zxvf glibc-2 17 tar gz cd glibc-2 17 mkdir build cd build    configure --prefix  home myself opt glibc-2 17     lt -- where you install new glibc make -j lt number of CPU Cores gt     You can find your  lt number of CPU Cores gt  by using   nproc   command make install   Second  follow the same way to install patchelf   Third  patch your Python    myself nfkd     patchelf --set-interpreter  home myself opt glibc-2 17 lib ld-linux-x86-64 so 2 --set-rpath  home myself opt glibc-2 17 lib   home myself miniconda3 envs tensorflow bin python   as mentioned by  msb  Now I can use tensorflow-2 0 alpha in CentOS 6 5   ref  https   serverkurma com linux how-to-update-glibc-newer-version-on-centos-6-x

User · Answer

When I wanted to run a chromium-browser on Ubuntu precise  glibc-2 15   I got the  typical  message     libc so 6  version  GLIBC 2 19  not found      I considered the fact  that files are not needed permamently  but only for start  So I collected the files needed for the browser and sudo and created a mini-glibc-2 19- environment  started the browser and then copied the original files back  again  The needed files are in RAM and the original glibc is the same   as root the files   -2 15 so  already exist    mkdir -p  glibc-2 19 i386-linux-gnu    glibc-2 19 ld-linux so 2 - gt   glibc-2 19 i386-linux-gnu ld-2 19 so  glibc-2 19 i386-linux-gnu libc so 6 - gt  libc-2 19 so  glibc-2 19 i386-linux-gnu libdl so 2 - gt  libdl-2 19 so  glibc-2 19 i386-linux-gnu libpthread so 0 - gt  libpthread-2 19 so   mkdir -p  glibc-2 15 i386-linux-gnu   glibc-2 15 ld-linux so 2 - gt    glibc-2 15 i386-linux-gnu ld-2 15 so   glibc-2 15 i386-linux-gnu libc so 6 - gt   libc-2 15 so   glibc-2 15 i386-linux-gnu libdl so 2 - gt   libdl-2 15 so   glibc-2 15 i386-linux-gnu libpthread so 0 - gt   libpthread-2 15 so    the script to run the browser      bin sh sudo cp -r  glibc-2 19    lib  path to the browser  amp  sleep 1 sudo cp -r  glibc-2 15    lib sudo rm -r  lib i386-linux-gnu  -2 19 so

[linux] Multiple glibc libraries on a single host

Examples related to linux

Examples related to gcc

Examples related to glibc