[git] How can I get Git to follow symlinks?

Is my best be going to be a shell script which replaces symlinks with copies, or is there another way of telling Git to follow symlinks?

PS: I know it's not very secure, but I only want to do it in a few specific cases.

I'm using Git 1.5.4.3 and it's following the passed symlink if it has a trailing slash. E.g.

# Adds the symlink itself
$ git add symlink

# Follows symlink and adds the denoted directory's contents
$ git add symlink/

What I did to add to get the files within a symlink into Git (I didn't use a symlink but):

sudo mount --bind SOURCEDIRECTORY TARGETDIRECTORY

Do this command in the Git-managed directory. TARGETDIRECTORY has to be created before the SOURCEDIRECTORY is mounted into it.

It works fine on Linux, but not on OS X! That trick helped me with Subversion too. I use it to include files from an Dropbox account, where a webdesigner does his/her stuff.

On MacOS (I have Mojave/ 10.14, git version 2.7.1), use bindfs.

brew install bindfs

cd /path/to/git_controlled_dir

mkdir local_copy_dir

bindfs </full/path/to/source_dir> </full/path/to/local_copy_dir>

It's been hinted by other comments, but not clearly provided in other answers. Hopefully this saves someone some time.

Hmmm, mount --bind doesn't seem to work on Darwin.

Does anyone have a trick that does?

[edited]

OK, I found the answer on Mac OS X is to make a hardlink. Except that that API is not exposed via ln, so you have to use your own tiny program to do this. Here is a link to that program:

Creating directory hard links in Mac OS X

Enjoy!

Why not create symlinks the other way around? Meaning instead of linking from the Git repository to the application directory, just link the other way around.

For example, let’s say I am setting up an application installed in ~/application that needs a configuration file config.conf:

• I add config.conf to my Git repository, for example, at ~/repos/application/config.conf.
• Then I create a symlink from ~/application by running ln -s ~/repos/application/config.conf.

This approach might not always work, but it worked well for me so far.

This is a pre-commit hook which replaces the symlink blobs in the index, with the content of those symlinks.

Put this in .git/hooks/pre-commit, and make it executable:

#!/bin/sh
# (replace "find ." with "find ./<path>" below, to work with only specific paths)

# (these lines are really all one line, on multiple lines for clarity)
# ...find symlinks which do not dereference to directories...
find . -type l -exec test '!' -d {} ';' -print -exec sh -c \
# ...remove the symlink blob, and add the content diff, to the index/cache
    'git rm --cached "$1"; diff -au /dev/null "$1" | git apply --cached -p1 -' \
# ...and call out to "sh".
    "process_links_to_nondir" {} ';'

# the end

Notes

We use POSIX compliant functionality as much as possible; however, diff -a is not POSIX compliant, possibly among other things.

There may be some mistakes/errors in this code, even though it was tested somewhat.

Use hard links instead. This differs from a soft (symbolic) link. All programs, including git will treat the file as a regular file. Note that the contents can be modified by changing either the source or the destination.

On macOS (before 10.13 High Sierra)

If you already have git and Xcode installed, install hardlink. It's a microscopic tool to create hard links.

To create the hard link, simply:

hln source destination

macOS High Sierra update

Does Apple File System support directory hard links?

Directory hard links are not supported by Apple File System. All directory hard links are converted to symbolic links or aliases when you convert from HFS+ to APFS volume formats on macOS.

From APFS FAQ on developer.apple.com

Follow https://github.com/selkhateeb/hardlink/issues/31 for future alternatives.

On Linux and other Unix flavors

The ln command can make hard links:

ln source destination

On Windows (Vista, 7, 8, …)

Use mklink to create a junction on Windows:

mklink /j "source" "destination"

I used to add files beyond symlinks for quite some time now. This used to work just fine, without making any special arrangements. Since I updated to Git 1.6.1, this does not work any more.

You may be able to switch to Git 1.6.0 to make this work. I hope that a future version of Git will have a flag to git-add allowing it to follow symlinks again.

An alternative implementation of what @user252400 proposes is to use bwrap, a small setuid sandbox that can be found in all major distributions - often installed by default. bwrap allows you to bind mount directories without sudo and automatically unbinds them when git or your shell exits.

Assuming your development process isn't too crazy (see bellow), start bash in a private namespace and bind the external directory under the git directory:

bwrap --ro-bind / / \
      --bind {EXTERNAL-DIR} {MOUNTPOINT-IN-GIT-DIR} \
      --dev /dev \
      bash

Then do everything you'd do normally like git add, git commit, and so on. When you're done, just exit bash. Clean and simple.

Caveats: To prevent sandbox escapes, bwrap is not allowed to execute other setuid binaries. See man bwrap for more details.

This is a pre-commit hook which replaces the symlink blobs in the index, with the content of those symlinks.

Put this in .git/hooks/pre-commit, and make it executable:

#!/bin/sh
# (replace "find ." with "find ./<path>" below, to work with only specific paths)

# (these lines are really all one line, on multiple lines for clarity)
# ...find symlinks which do not dereference to directories...
find . -type l -exec test '!' -d {} ';' -print -exec sh -c \
# ...remove the symlink blob, and add the content diff, to the index/cache
    'git rm --cached "$1"; diff -au /dev/null "$1" | git apply --cached -p1 -' \
# ...and call out to "sh".
    "process_links_to_nondir" {} ';'

# the end

Notes

We use POSIX compliant functionality as much as possible; however, diff -a is not POSIX compliant, possibly among other things.

There may be some mistakes/errors in this code, even though it was tested somewhat.

Hmmm, mount --bind doesn't seem to work on Darwin.

Does anyone have a trick that does?

[edited]

OK, I found the answer on Mac OS X is to make a hardlink. Except that that API is not exposed via ln, so you have to use your own tiny program to do this. Here is a link to that program:

Creating directory hard links in Mac OS X

Enjoy!

An alternative implementation of what @user252400 proposes is to use bwrap, a small setuid sandbox that can be found in all major distributions - often installed by default. bwrap allows you to bind mount directories without sudo and automatically unbinds them when git or your shell exits.

Assuming your development process isn't too crazy (see bellow), start bash in a private namespace and bind the external directory under the git directory:

bwrap --ro-bind / / \
      --bind {EXTERNAL-DIR} {MOUNTPOINT-IN-GIT-DIR} \
      --dev /dev \
      bash

Then do everything you'd do normally like git add, git commit, and so on. When you're done, just exit bash. Clean and simple.

Caveats: To prevent sandbox escapes, bwrap is not allowed to execute other setuid binaries. See man bwrap for more details.

I got tired of every solution in here either being outdated or requiring root, so I made an LD_PRELOAD-based solution (Linux only).

It hooks into Git's internals, overriding the 'is this a symlink?' function, allowing symlinks to be treated as their contents. By default, all links to outside the repo are inlined; see the link for details.

I used to add files beyond symlinks for quite some time now. This used to work just fine, without making any special arrangements. Since I updated to Git 1.6.1, this does not work any more.

You may be able to switch to Git 1.6.0 to make this work. I hope that a future version of Git will have a flag to git-add allowing it to follow symlinks again.

What I did to add to get the files within a symlink into Git (I didn't use a symlink but):

sudo mount --bind SOURCEDIRECTORY TARGETDIRECTORY

Do this command in the Git-managed directory. TARGETDIRECTORY has to be created before the SOURCEDIRECTORY is mounted into it.

It works fine on Linux, but not on OS X! That trick helped me with Subversion too. I use it to include files from an Dropbox account, where a webdesigner does his/her stuff.

With Git 2.3.2+ (Q1 2015), there is one other case where Git will not follow symlink anymore: see commit e0d201b by Junio C Hamano (gitster) (main Git maintainer)

apply: do not touch a file beyond a symbolic link

Because Git tracks symbolic links as symbolic links, a path that has a symbolic link in its leading part (e.g. path/to/dir/file, where path/to/dir is a symbolic link to somewhere else, be it inside or outside the working tree) can never appear in a patch that validly applies, unless the same patch first removes the symbolic link to allow a directory to be created there.

Detect and reject such a patch.

Similarly, when an input creates a symbolic link path/to/dir and then creates a file path/to/dir/file, we need to flag it as an error without actually creating path/to/dir symbolic link in the filesystem.

Instead, for any patch in the input that leaves a path (i.e. a non deletion) in the result, we check all leading paths against the resulting tree that the patch would create by inspecting all the patches in the input and then the target of patch application (either the index or the working tree).

This way, we:

• catch a mischief or a mistake to add a symbolic link path/to/dir and a file path/to/dir/file at the same time,
• while allowing a valid patch that removes a symbolic link path/to/dir and then adds a file path/to/dir/file.

That means, in that case, the error message won't be a generic one like "%s: patch does not apply", but a more specific one:

affected file '%s' is beyond a symbolic link

I used to add files beyond symlinks for quite some time now. This used to work just fine, without making any special arrangements. Since I updated to Git 1.6.1, this does not work any more.

You may be able to switch to Git 1.6.0 to make this work. I hope that a future version of Git will have a flag to git-add allowing it to follow symlinks again.

Conversion from symlinks could be useful. Link in a Git folder instead of a symlink by a script.

I used to add files beyond symlinks for quite some time now. This used to work just fine, without making any special arrangements. Since I updated to Git 1.6.1, this does not work any more.

You may be able to switch to Git 1.6.0 to make this work. I hope that a future version of Git will have a flag to git-add allowing it to follow symlinks again.

I got tired of every solution in here either being outdated or requiring root, so I made an LD_PRELOAD-based solution (Linux only).

It hooks into Git's internals, overriding the 'is this a symlink?' function, allowing symlinks to be treated as their contents. By default, all links to outside the repo are inlined; see the link for details.

On MacOS (I have Mojave/ 10.14, git version 2.7.1), use bindfs.

brew install bindfs

cd /path/to/git_controlled_dir

mkdir local_copy_dir

bindfs </full/path/to/source_dir> </full/path/to/local_copy_dir>

It's been hinted by other comments, but not clearly provided in other answers. Hopefully this saves someone some time.

I'm using Git 1.5.4.3 and it's following the passed symlink if it has a trailing slash. E.g.

# Adds the symlink itself
$ git add symlink

# Follows symlink and adds the denoted directory's contents
$ git add symlink/

Use hard links instead. This differs from a soft (symbolic) link. All programs, including git will treat the file as a regular file. Note that the contents can be modified by changing either the source or the destination.

On macOS (before 10.13 High Sierra)

If you already have git and Xcode installed, install hardlink. It's a microscopic tool to create hard links.

To create the hard link, simply:

hln source destination

macOS High Sierra update

Does Apple File System support directory hard links?

Directory hard links are not supported by Apple File System. All directory hard links are converted to symbolic links or aliases when you convert from HFS+ to APFS volume formats on macOS.

From APFS FAQ on developer.apple.com

Follow https://github.com/selkhateeb/hardlink/issues/31 for future alternatives.

On Linux and other Unix flavors

The ln command can make hard links:

ln source destination

On Windows (Vista, 7, 8, …)

Use mklink to create a junction on Windows:

mklink /j "source" "destination"

Why not create symlinks the other way around? Meaning instead of linking from the Git repository to the application directory, just link the other way around.

For example, let’s say I am setting up an application installed in ~/application that needs a configuration file config.conf:

• I add config.conf to my Git repository, for example, at ~/repos/application/config.conf.
• Then I create a symlink from ~/application by running ln -s ~/repos/application/config.conf.

This approach might not always work, but it worked well for me so far.

With Git 2.3.2+ (Q1 2015), there is one other case where Git will not follow symlink anymore: see commit e0d201b by Junio C Hamano (gitster) (main Git maintainer)

apply: do not touch a file beyond a symbolic link

Because Git tracks symbolic links as symbolic links, a path that has a symbolic link in its leading part (e.g. path/to/dir/file, where path/to/dir is a symbolic link to somewhere else, be it inside or outside the working tree) can never appear in a patch that validly applies, unless the same patch first removes the symbolic link to allow a directory to be created there.

Detect and reject such a patch.

Similarly, when an input creates a symbolic link path/to/dir and then creates a file path/to/dir/file, we need to flag it as an error without actually creating path/to/dir symbolic link in the filesystem.

Instead, for any patch in the input that leaves a path (i.e. a non deletion) in the result, we check all leading paths against the resulting tree that the patch would create by inspecting all the patches in the input and then the target of patch application (either the index or the working tree).

This way, we:

• catch a mischief or a mistake to add a symbolic link path/to/dir and a file path/to/dir/file at the same time,
• while allowing a valid patch that removes a symbolic link path/to/dir and then adds a file path/to/dir/file.

That means, in that case, the error message won't be a generic one like "%s: patch does not apply", but a more specific one:

affected file '%s' is beyond a symbolic link

Conversion from symlinks could be useful. Link in a Git folder instead of a symlink by a script.