How do I clone a subdirectory only of a Git repository

Question

I have my Git repository which  at the root  has two sub directories    finisht  static   When this was in SVN   finisht was checked out in one place  while  static was checked out elsewhere  like so   svn co svn ssh   admin domain com home admin repos finisht static static   Is there a way to do this with Git

User · Answer

If you re actually ony interested in the latest revision files of a directory  Github lets you download a repository as Zip file  which does not contain history  So downloading is very much faster

User · Answer

Git 1 7 0 has    sparse checkouts     See     core sparseCheckout    in the git config manpage      Sparse checkout    in the git read-tree manpage  and    Skip-worktree bit    in the git update-index manpage   The interface is not as convenient as SVN   s  e g  there is no way to make a sparse checkout at the time of an initial clone   but the base functionality upon which simpler interfaces could be built is now available

User · Answer

While I hate actually having to use svn when dealing with git repos    I use this all the time   function git-scp       URL   1   amp  amp  shift 1   svn export   URL blob  master trunk      This allows you to copy out from the github url without modification  Usage   ---  tmp    git-scp https   github com dgraph-io dgraph blob master contrib config kubernetes helm                                                                                                                  1   A    helm A    helm Chart yaml A    helm README md A    helm values yaml Exported revision 6367   ---  tmp    ls   grep helm Permissions Size User    Date Modified    Name drwxr-xr-x     - anthony 2020-01-07 15 53 helm

User · Answer

You can combine the sparse checkout and the shallow clone features  The shallow clone cuts off the history and the sparse checkout only pulls the files matching your patterns   git init  lt repo gt  cd  lt repo gt  git remote add origin  lt url gt  git config core sparsecheckout true echo  finisht     gt  gt   git info sparse-checkout git pull --depth 1 origin master   You ll need minimum git 1 9 for this to work  Tested it myself only with 2 2 0 and 2 2 2   This way you ll be still able to push  which is not possible with git archive

User · Answer

For other users who just want to download a file folder from github  simply use   svn export  lt repo gt  trunk  lt folder gt    e g   svn export https   github com lodash lodash com trunk docs    yes  that s svn here  apparently in 2016 you still need svn to simply download some github files   Courtesy  Download a single folder or directory from a GitHub repo   Important - Make sure you update the github URL and replace  tree master  with   trunk     As bash script   git-download        folder     tree  master trunk      folder   folder blob  master trunk      svn export  folder     Note This method downloads a folder  does not clone checkout it  You can t push changes back to the repository  On the other hand - this results in smaller download compared to sparse checkout or shallow checkout

User · Answer

EDIT  As of Git 2 19  this is finally possible  as can be seen in this answer   Consider upvoting that answer   Note  in Git 2 19  only client-side support is implemented  server-side support is still missing  so it only works when cloning local repositories  Also note that large Git hosters  e g  GitHub  don t actually use the Git server  they use their own implementation  so even if support shows up in the Git server  it does not automatically mean that it works on Git hosters   OTOH  since they don t use the Git server  they could implement it faster in their own implementations before it shows up in Git server      No  that s not possible in Git   Implementing something like this in Git would be a substantial effort and it would mean that the integrity of the clientside repository could no longer be guaranteed  If you are interested  search for discussions on  sparse clone  and  sparse fetch  on the git mailinglist   In general  the consensus in the Git community is that if you have several directories that are always checked out independently  then these are really two different projects and should live in two different repositories  You can glue them back together using Git Submodules

User · Answer

git clone --filter from git 2 19 now works on GitHub  tested 2021-01-14  git 2 30 0  This option was added together with an update to the remote protocol  and it truly prevents objects from being downloaded from the server  E g   to clone only objects required for d1 of this minimal test repository  https   github com cirosantilli test-git-partial-clone I can do  git clone     --depth 1      --filter blob none      --sparse     https   github com cirosantilli test-git-partial-clone     cd test-git-partial-clone git sparse-checkout init --cone git sparse-checkout set d1  Here s a less minimal and more realistic version at https   github com cirosantilli test-git-partial-clone-big-small git clone     --depth 1      --filter blob none      --sparse     https   github com cirosantilli test-git-partial-clone-big-small     cd test-git-partial-clone git sparse-checkout init --cone git sparse-checkout set small  That repository contains   a big directory with 10 10MB files a small directory with 1000 files of size one byte  All contents are pseudo-random and therefore incompressible  Clone times on my 36 4 Mbps internet   full  24s partial   quot instantaneous quot   The sparse-checkout part is also needed unfortunately  You can also only download certain files with the much more understandable  git clone     --depth 1      --filter blob none      --no-checkout     https   github com cirosantilli test-git-partial-clone     cd test-git-partial-clone git checkout master -- di  but that method for some reason downloads files one by one very slowly  making it unusable unless you have very few files in the directory  Analysis of the objects in the minimal repository The clone command obtains only   a single commit object with the tip of the master branch all 4 tree objects of the repository   toplevel directory of commit the the three directories d1  d2  master    Then  the git sparse-checkout set command fetches only the missing blobs  files  from the server   d1 a d1 b  Even better  later on GitHub will likely start supporting    --filter blob none     --filter tree 0    where --filter tree 0 from Git 2 20 will prevent the unnecessary clone fetch of all tree objects  and allow it to be deferred to checkout  But on my 2020-09-18 test that fails with  fatal  invalid filter-spec  combine blob none tree 0   presumably because the --filter combine  composite filter  added in Git 2 24  implied by multiple --filter  is not yet implemented  I observed which objects were fetched with  git verify-pack -v  git objects pack   pack  as mentioned at  How to list ALL git objects in the database  It does not give me a super clear indication of what each object is exactly  but it does say the type of each object  commit  tree  blob   and since there are so few objects in that minimal repo  I can unambiguously deduce what each object is  git rev-list --objects --all did produce clearer output with paths for tree blobs  but it unfortunately fetches some objects when I run it  which makes it hard to determine what was fetched when  let me know if anyone has a better command  TODO find GitHub announcement that saying when they started supporting it  https   github blog 2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout  from 2020-01-17 already mentions --filter blob none  git sparse-checkout I think this command is meant to manage a settings file that says  quot I only care about these subtrees quot  so that future commands will only affect those subtrees  But it is a bit hard to be sure because the current documentation is a bit    sparse  -  It does not  by itself  prevent the fetching of blobs  If this understanding is correct  then this would be a good complement to git clone --filter described above  as it would prevent unintentional fetching of more objects if you intend to do git operations in the partial cloned repo  When I tried on Git 2 25 1  git clone     --depth 1     --filter blob none     --no-checkout     https   github com cirosantilli test-git-partial-clone     cd test-git-partial-clone git sparse-checkout init  it didn t work because the init actually fetched all objects  However  in Git 2 28 it didn t fetch the objects as desired  But then if I do  git sparse-checkout set d1  d1 is not fetched and checked out  even though this explicitly says it should  https   github blog 2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout  sparse-checkout-and-partial-clones With disclaimer   Keep an eye out for the partial clone feature to become generally available 1    1   GitHub is still evaluating this feature internally while it   s enabled on a select few repositories  including the example used in this post   As the feature stabilizes and matures  we   ll keep you updated with its progress   So yeah  it s just too hard to be certain at the moment  thanks in part to the joys of GitHub being closed source  But let s keep an eye on it  Command breakdown The server should be configured with  git config --local uploadpack allowfilter 1 git config --local uploadpack allowanysha1inwant 1  Command breakdown   --filter blob none skips all blobs  but still fetches all tree objects  --filter tree 0 skips the unneeded trees  https   www spinics net lists git msg342006 html  --depth 1 already implies --single-branch  see also  How do I clone a single branch in Git   file     path  is required to overcome git clone protocol shenanigans  How to shallow clone a local git repository with a relative path   --filter combine FILTER1 FILTER2 is the syntax to use multiple filters at once  trying to pass --filter for some reason fails with   quot multiple filter-specs cannot be combined quot   This was added in Git 2 24 at e987df5fe62b8b29be4cdcdeb3704681ada2b29e  quot list-objects-filter  implement composite filters quot  Edit  on Git 2 28  I experimentally see that --filter FILTER1 --filter FILTER2 also has the same effect  since GitHub does not implement combine  yet as of 2020-09-18 and complains fatal  invalid filter-spec  combine blob none tree 0   TODO introduced in which version    The format of --filter is documented on man git-rev-list  Docs on Git tree   https   github com git git blob v2 19 0 Documentation technical partial-clone txt https   github com git git blob v2 19 0 Documentation rev-list-options txt L720 https   github com git git blob v2 19 0 t t5616-partial-clone sh  Test it out locally The following script reproducibly generates the https   github com cirosantilli test-git-partial-clone repository locally  does a local clone  and observes what was cloned     usr bin env bash set -eu  list-objects       git rev-list --all --objects   echo  quot master commit SHA    git log -1 --format  quot  H quot   quot    echo  quot mybranch commit SHA    git log -1 --format  quot  H quot   quot    git ls-tree master   git ls-tree mybranch   grep mybranch   git ls-tree master    grep root      Reproducibility  export GIT COMMITTER NAME  a  export GIT COMMITTER EMAIL  a  export GIT AUTHOR NAME  a  export GIT AUTHOR EMAIL  a  export GIT COMMITTER DATE  2000-01-01T00 00 00 0000  export GIT AUTHOR DATE  2000-01-01T00 00 00 0000   rm -rf server repo local repo mkdir server repo cd server repo    Create repo  git init --quiet git config --local uploadpack allowfilter 1 git config --local uploadpack allowanysha1inwant 1    First commit    Directories present in all branches  mkdir d1 d2 printf  d1 a   gt    d1 a printf  d1 b   gt    d1 b printf  d2 a   gt    d2 a printf  d2 b   gt    d2 b   Present only in root  mkdir  root  printf  root   gt    root root git add   git commit -m  root  --quiet    Second commit only on master  git rm --quiet -r   root mkdir  master  printf  master   gt    master master git add   git commit -m  master commit  --quiet    Second commit only on mybranch  git checkout -b mybranch --quiet master  git rm --quiet -r   root mkdir  mybranch  printf  mybranch   gt    mybranch mybranch git add   git commit -m  mybranch commit  --quiet  echo  quot   List and identify all objects quot  list-objects echo    Restore master  git checkout --quiet master cd       Clone  Don t checkout for now  only  git  dir  git clone --depth 1 --quiet --no-checkout --filter blob none  quot file     pwd  server repo quot  local repo cd local repo    List missing objects from master  echo  quot   Missing objects after --no-checkout quot  git rev-list --all --quiet --objects --missing print echo  echo  quot   Git checkout fails without internet quot  mv    server repo    server repo off   git checkout master echo  echo  quot   Git checkout fetches the missing directory from internet quot  mv    server repo off    server repo git checkout master -- d1  echo  echo  quot   Missing objects after checking out d1 quot  git rev-list --all --quiet --objects --missing print  GitHub upstream  Output in Git v2 19 0    List and identify all objects c6fcdfaf2b1462f809aecdad83a186eeec00f9c1 fc5e97944480982cfc180a6d6634699921ee63ec 7251a83be9a03161acde7b71a8fda9be19f47128 62d67bce3c672fe2b9065f372726a11e57bade7e b64bf435a3e54c5208a1b70b7bcb0fc627463a75 d1 308150e8fddde043f3dbbb8573abb6af1df96e63 d1 a f70a17f51b7b30fec48a32e4f19ac15e261fd1a4 d1 b 84de03c312dc741d0f2a66df7b2f168d823e122a d2 0975df9b39e23c15f63db194df7f45c76528bccb d2 a 41484c13520fcbb6e7243a26fdb1fc9405c08520 d2 b 7d5230379e4652f1b1da7ed1e78e0b8253e03ba3 master 8b25206ff90e9432f6f1a8600f87a7bd695a24af master master ef29f15c9a7c5417944cc09711b6a9ee51b01d89 19f7a4ca4a038aff89d803f017f76d2b66063043 mybranch 1b671b190e293aa091239b8b5e8c149411d00523 mybranch mybranch c3760bb1a0ece87cdbaf9a563c77a45e30a4e30e a0234da53ec608b54813b4271fbf00ba5318b99f root 93ca1422a8da0a9effc465eccbcb17e23015542d root root master commit SHA  fc5e97944480982cfc180a6d6634699921ee63ec mybranch commit SHA  fc5e97944480982cfc180a6d6634699921ee63ec 040000 tree b64bf435a3e54c5208a1b70b7bcb0fc627463a75    d1 040000 tree 84de03c312dc741d0f2a66df7b2f168d823e122a    d2 040000 tree 7d5230379e4652f1b1da7ed1e78e0b8253e03ba3    master 040000 tree 19f7a4ca4a038aff89d803f017f76d2b66063043    mybranch 040000 tree a0234da53ec608b54813b4271fbf00ba5318b99f    root    Missing objects after --no-checkout  f70a17f51b7b30fec48a32e4f19ac15e261fd1a4  8b25206ff90e9432f6f1a8600f87a7bd695a24af  41484c13520fcbb6e7243a26fdb1fc9405c08520  0975df9b39e23c15f63db194df7f45c76528bccb  308150e8fddde043f3dbbb8573abb6af1df96e63    Git checkout fails without internet fatal    home ciro bak git test-git-web-interface other-test-repos partial-clone tmp server repo  does not appear to be a git repository fatal  Could not read from remote repository   Please make sure you have the correct access rights and the repository exists     Git checkout fetches the missing directory from internet remote  Enumerating objects  1  done  remote  Counting objects  100   1 1   done  remote  Total 1  delta 0   reused 0  delta 0  Receiving objects  100   1 1   45 bytes   45 00 KiB s  done  remote  Enumerating objects  1  done  remote  Counting objects  100   1 1   done  remote  Total 1  delta 0   reused 0  delta 0  Receiving objects  100   1 1   45 bytes   45 00 KiB s  done     Missing objects after checking out d1  8b25206ff90e9432f6f1a8600f87a7bd695a24af  41484c13520fcbb6e7243a26fdb1fc9405c08520  0975df9b39e23c15f63db194df7f45c76528bccb  Conclusions  all blobs from outside of d1  are missing  E g  0975df9b39e23c15f63db194df7f45c76528bccb  which is d2 b is not there after checking out d1 a  Note that root root and mybranch mybranch are also missing  but --depth 1 hides that from the list of missing files  If you remove --depth 1  then they show on the list of missing files  I have a dream This feature could revolutionize Git  Imagine having all the code base of your enterprise in a single repo without ugly third-party tools like repo  Imagine storing huge blobs directly in the repo without any ugly third party extensions  Imagine if GitHub would allow per file   directory metadata like stars and permissions  so you can store all your personal stuff under a single repo  Imagine if submodules were treated exactly like regular directories  just request a tree SHA  and a DNS-like mechanism resolves your request  first looking on your local    git  then first to closer servers  your enterprise s mirror   cache  and ending up on GitHub

User · Answer

If you never plan to interact with the repository from which you cloned  you can do a full git clone and rewrite your repository using git filter-branch --subdirectory-filter  This way  at least the history will be preserved

User · Answer

Using Linux  And only want easy to access and clean working tree   without bothering rest of code on your machine  try symlinks   git clone https   github com  user   repo  git   my-project ln -s   my-project my-subfolder   Desktop my-subfolder   Test   cd   Desktop my-subfolder git status

User · Answer

I wrote a  gitconfig  alias  for performing a  sparse checkout   Check it out  no pun intended    On Windows run in cmd exe  git config --global alias sparse-checkout   f         -eq 2    amp  amp  L   1      L   L  git     L  2  mkdir -p    L  git info    amp  amp  cd    L    amp  amp  git init --template   amp  amp  git remote add origin    1    amp  amp  git config core sparseCheckout 1       -eq 2    amp  amp  echo    2    gt  gt   git info sparse-checkout      shift 2  for i  do echo  i  gt  gt   git info sparse-checkout  done    git pull --depth 1 origin master   f    Otherwise   git config --global alias sparse-checkout   f         -eq 2    amp  amp  L   1      L   L  git     L  2  mkdir -p   L  git info   amp  amp  cd   L   amp  amp  git init --template   amp  amp  git remote add origin   1   amp  amp  git config core sparseCheckout 1       -eq 2    amp  amp  echo   2   gt  gt   git info sparse-checkout      shift 2  for i  do echo  i  gt  gt   git info sparse-checkout  done    git pull --depth 1 origin master   f    Usage     Makes a directory ForStackExchange with Plug checked out git sparse-checkout https   github com YenForYang ForStackExchange Plug    To do more than 1 directory  you have to specify the local directory  git sparse-checkout https   github com YenForYang ForStackExchange ForStackExchange Plug Folder   The git config commands are  minified  for convenience and storage  but here is the alias expanded     Note the --template  is for disabling templates    Feel free to remove it if you don t have issues with them  like I did     mkdir  makes the  git info directory ahead of time  as I ve found it missing sometimes for some reason f               -eq 2    amp  amp  L    1       L   L  git     L  2      mkdir -p   L  git info           amp  amp  cd   L           amp  amp  git init --template           amp  amp  git remote add origin   1           amp  amp  git config core sparseCheckout 1             -eq 2            amp  amp  echo   2   gt  gt   git info sparse-checkout                          shift 2              for i  do                 echo  i  gt  gt   git info sparse-checkout              done                git pull --depth 1 origin master     f

User · Answer

I just wrote a script for GitHub   Usage    python get git sub dir py path to sub dir  lt RECURSIVE gt

User · Answer

This will clone a specific folder and remove all history not related to it   git clone --single-branch -b  branch  git github com  user   repo  git git filter-branch --subdirectory-filter  path to folder  HEAD git remote remove origin git remote add origin git github com  user   new-repo  git git push -u origin master

User · Answer

What you are trying to do is called a sparse checkout  and that feature was added in git 1 7 0  Feb  2012   The steps to do a sparse clone are as follows  mkdir  lt repo gt  cd  lt repo gt  git init git remote add -f origin  lt url gt   This creates an empty repository with your remote  and fetches all objects but doesn t check them out  Then do  git config core sparseCheckout true  Now you need to define which files folders you want to actually check out  This is done by listing them in  git info sparse-checkout  eg  echo  quot some dir  quot   gt  gt   git info sparse-checkout echo  quot another sub tree quot   gt  gt   git info sparse-checkout  Last but not least  update your empty repo with the state from the remote  git pull origin master  You will now have files  quot checked out quot  for some dir and another sub tree on your file system  with those paths still   and no other paths present  You might want to have a look at the extended tutorial and you should probably read the official documentation for sparse checkout  As a function  function git sparse clone       rurl  quot  1 quot  localdir  quot  2 quot   amp  amp  shift 2    mkdir -p  quot  localdir quot    cd  quot  localdir quot     git init   git remote add -f origin  quot  rurl quot     git config core sparseCheckout true      Loops over remaining args   for i  do     echo  quot  i quot   gt  gt   git info sparse-checkout   done    git pull origin master    Usage  git sparse clone  quot http   github com tj n quot   quot   local location quot   quot  bin quot    Note that this will still download the whole repository from the server     only the checkout is reduced in size  At the moment it is not possible to clone only a single directory  But if you don t need the history of the repository  you can at least save on bandwidth by creating a shallow clone  See udondan s answer below for information on how to combine shallow clone and sparse checkout   As of git 2 25 0  Jan 2020  an experimental sparse-checkout command is added in git  git sparse-checkout init   same as     git config core sparseCheckout true  git sparse-checkout set  quot A B quot    same as    echo  quot A B quot   gt  gt   git info sparse-checkout  git sparse-checkout list   same as    cat  git info sparse-checkout

User · Answer

It s not possible to clone subdirectory only with Git  but below are few workarounds   Filter branch  You may want to rewrite the repository to look as if trunk public html  had been its project root  and discard all other history  using filter-branch   try on already checkout branch   git filter-branch --subdirectory-filter trunk public html -- --all   Notes  The -- that separates filter-branch options from revision options  and the --all to rewrite all branches and tags  All information including original commit times or merge information will be preserved  This command honors  git info grafts file and refs in the refs replace  namespace  so if you have any grafts or replacement refs defined  running this command will make them permanent      Warning  The rewritten history will have different object names for all the objects and will not converge with the original branch  You will not be able to easily push and distribute the rewritten branch on top of the original branch  Please do not use this command if you do not know the full implications  and avoid using it anyway  if a simple single commit would suffice to fix your problem       Sparse checkout  Here are simple steps with sparse checkout approach which will populate the working directory sparsely  so you can tell Git which folder s  or file s  in the working directory are worth checking out    Clone repository as usual  --no-checkout is optional    git clone --no-checkout git foo bar git cd bar   You may skip this step  if you ve your repository already cloned   Hint  For large repos  consider shallow clone  --depth 1  to checkout only latest revision or and --single-branch only  Enable sparseCheckout option   git config core sparseCheckout true  Specify folder s  for sparse checkout  without space at the end    echo  trunk public html    gt   git info sparse-checkout   or edit  git info sparse-checkout  Checkout the branch  e g  master    git checkout master    Now you should have selected folders in your current directory   You may consider symbolic links if you ve too many levels of directories or filtering branch instead

User · Answer

So i tried everything in this tread and nothing worked for me     Turns out that on version 2 24 of Git  the one that comes with cpanel at the time of this answer   you don t need to do this  echo  wpm     gt  gt   git info sparse-checkout   all you need is the folder name  wpm     So in short you do this  git config core sparsecheckout true   you then edit the  git info sparse-checkout and add the folder names  one per line  with    at the end to get subfolders and files  wpm     Save and run the checkout command  git checkout master   The result was the expected folder from my repo and nothing else Upvote if this worked for you

User · Answer

This looks far simpler   git archive --remote  lt repo url gt   lt branch gt   lt path gt    tar xvf -

User · Answer

Just to clarify some of the great answers here  the steps outlined in many of the answers assume that you already have a remote repository somewhere  Given  an existing git repository  e g  git github com some-user full-repo git  with one or more directories that you wish to pull independently of the rest of the repo  e g  directories named app1 and app2 Assuming you have a git repository as the above    Then  you can run steps like the following to pull only specific directories from that larger repo  mkdir app1 cd app1 git init git remote add origin git github com some-user full-repo git git config core sparsecheckout true echo  quot app1  quot   gt  gt   git info sparse-checkout git pull origin master  I had mistakenly thought that the sparse-checkout options had to be set on the original repository  but this is not the case  you define which directories you want locally  prior to pulling from the remote  The remote repo doesn t know or care about your only wanting to track a part of the repo  Hope this clarification helps someone else

User · Answer

Here s a shell script I wrote for the use case of a single subdirectory sparse checkout  coSubDir sh  localRepo  1 remoteRepo  2 subDir  3     Create local repository for subdirectory checkout  make it hidden to avoid having to drill down to the subfolder mkdir     localRepo cd     localRepo git init git remote add -f origin  remoteRepo git config core sparseCheckout true    Add the subdirectory of interest to the sparse checkout  echo  subDir  gt  gt   git info sparse-checkout  git pull origin master    Create convenience symlink to the subdirectory of interest cd    ln -s     localRepo  subDir  localRepo

[git] How do I clone a subdirectory only of a Git repository?

Examples related to git

Examples related to repository

Examples related to subdirectory

Examples related to git-clone

Examples related to sparse-checkout