[linux] tar: add all files and directories in current directory INCLUDING .svn and so on

I try to tar.gz a directory and use

tar -czf workspace.tar.gz *

The resulting tar includes .svn directories in subdirs but NOT in the current directory (as * gets expanded to only 'visible' files before it is passed to tar

I tried to

tar -czf workspace.tar.gz . instead but then I am getting an error because '.' has changed while reading:

tar: ./workspace.tar.gz: file changed as we read it

Is there a trick so that * matches all files (including dot-prefixed) in a directory?

(using bash on Linux SLES-11 (2.6.27.19)

You can include the hidden directories by going back a directory and doing:

cd ..
tar czf workspace.tar.gz workspace

Assuming the directory you wanted to gzip was called workspace.

The problem with the most solutions provided here is that tar contains ./ at the begging of every entry. So this results in having . directory when opening it through GUI compressor. So what I ended up doing is:

ls -1A | xargs -d "\n" tar cfz my.tar.gz

If you already have my.tar.gz in current directory you may want to grep this out:

ls -1A | grep -v my.tar.gz | xargs -d "\n" tar cfz my.tar.gz

Be aware of that xargs has certain limit (see xargs --show-limits). So this solution would not work if you are trying to create a package which has lots of entries (directories and files) on a directory which you are trying to tar.

Update: I added a fix for the OP's comment.

tar -czf workspace.tar.gz .

will indeed change the current directory, but why not place the file somewhere else?

tar -czf somewhereelse/workspace.tar.gz .
mv somewhereelse/workspace.tar.gz . # Update

10 years later, you have an alternative to tar, illustrated with Git 2.30 (Q1 2021), which uses "git archive"^(man) to produce the release tarball instead of tar.
(You don't need Git 2.30 to apply that alternative)

See commit 4813277 (11 Oct 2020), and commit 93e7031 (10 Oct 2020) by René Scharfe (rscharfe).
^{(Merged by Junio C Hamano -- gitster -- in commit 63e5273, 27 Oct 2020)}

Makefile: use git init/add/commit/archive for dist-doc

^{Signed-off-by: René Scharfe}

Reduce the dependency on external tools by generating the distribution archives for HTML documentation and manpages using git^(man) commands instead of tar. This gives the archive entries the same meta data as those in the dist archive for binaries.

So instead of:

tar cf ../archive.tar .

You can do using Git only:

git -C workspace init
git -C workspace add .
git -C workspace commit -m workspace
git -C workspace archive --format=tar --prefix=./ HEAD^{tree} > workspace.tar
rm -Rf workspace/.git

That was initially proposed because, as explained here, some exotic platform might have an old tar distribution with lacking options.

A good question. In ZSH you can use the globbing modifier (D), which stands for "dotfiles". Compare:

ls $HOME/*

and

ls $HOME/*(D)

This correctly excludes the special directory entries . and ... In Bash you can use .* to include the dotfiles explicitly:

ls $HOME/* $HOME/.*

But that includes . and .. as well, so it's not what you were looking for. I'm sure there's some way to make * match dotfiles in bash, too.

Using find is probably the easiest way:

find . -maxdepth 1 -exec tar zcvf workspace.tar.gz {} \+

find . -maxdepth 1 will find all files/directories/symlinks/etc in the current directory and run the command specified by -exec. The {} in the command means file list goes here and \+ means that the command will be run as:

tar zcvf workspace.tar.gz .file1 .file2 .dir3

instead of

tar zcvf workspace.tar.gz .file1
tar zcvf workspace.tar.gz .file2
tar zcvf workspace.tar.gz .dir3

Yet another solution, assuming the number of items in the folder is not huge:

tar -czf workspace.tar.gz `ls -A`

(ls -A prints normal and hidden files but not "." and ".." as ls -a does.)

If disk space space is not an issue, this could also be a very easy thing to do:

mkdir backup
cp -r ./* backup
tar -zcvf backup.tar.gz ./backup

tar -czf workspace.tar.gz .??* *

Specifying .??* will include "dot" files and directories that have at least 2 characters after the dot. The down side is it will not include files/directories with a single character after the dot, such as .a, if there are any.

There are a couple of steps to take:

1. Replace * by . to include hidden files as well.
2. To create the archive in the same directory a --exclude=workspace.tar.gz can be used to exclude the archive itself.
3. To prevent the tar: .: file changed as we read it error when the archive is not yet created, make sure it exists (e.g. using touch), so the --exclude matches with the archive filename. (It does not match it the file does not exists)

Combined this results in the following script:

touch workspace.tar.gz
tar -czf workspace.tar.gz --exclude=workspace.tar.gz .

You can fix the . form by using --exclude:

tar -czf workspace.tar.gz --exclude=workspace.tar.gz .

Actually the problem is with the compression options. The trick is the pipe the tar result to a compressor instead of using the built-in options. Incidentally that can also give you better compression, since you can set extra compresion options.

Minimal tar:

tar --exclude=*.tar* -cf workspace.tar .

Pipe to a compressor of your choice. This example is verbose and uses xz with maximum compression:

tar --exclude=*.tar* -cv . | xz -9v >workspace.tar.xz

Solution was tested on Ubuntu 14.04 and Cygwin on Windows 7. It's a community wiki answer, so feel free to edit if you spot a mistake.

Don't create the tar file in the directory you are packing up:

tar -czf /tmp/workspace.tar.gz .

does the trick, except it will extract the files all over the current directory when you unpack. Better to do:

cd ..
tar -czf workspace.tar.gz workspace

or, if you don't know the name of the directory you were in:

base=$(basename $PWD)
cd ..
tar -czf $base.tar.gz $base

(This assumes that you didn't follow symlinks to get to where you are and that the shell doesn't try to second guess you by jumping backwards through a symlink - bash is not trustworthy in this respect. If you have to worry about that, use cd -P .. to do a physical change directory. Stupid that it is not the default behaviour in my view - confusing, at least, for those for whom cd .. never had any alternative meaning.)

One comment in the discussion says:

I [...] need to exclude the top directory and I [...] need to place the tar in the base directory.

The first part of the comment does not make much sense - if the tar file contains the current directory, it won't be created when you extract file from that archive because, by definition, the current directory already exists (except in very weird circumstances).

The second part of the comment can be dealt with in one of two ways:

1. Either: create the file somewhere else - /tmp is one possible location - and then move it back to the original location after it is complete.
2. Or: if you are using GNU Tar, use the --exclude=workspace.tar.gz option. The string after the = is a pattern - the example is the simplest pattern - an exact match. You might need to specify --exclude=./workspace.tar.gz if you are working in the current directory contrary to recommendations; you might need to specify --exclude=workspace/workspace.tar.gz if you are working up one level as suggested. If you have multiple tar files to exclude, use '*', as in --exclude=./*.gz.

Had a similar situation myself. I think it is best to create the tar elsewhere and then use -C to tell tar the base directory for the compressed files. Example:

tar -cjf workspace.tar.gz -C <path_to_workspace> $(ls -A <path_to_workspace>)

This way there is no need to exclude your own tarfile. As noted in other comments, -A will list hidden files.

If you really don't want to include top directory in the tarball (and that's generally bad idea):

tar czf workspace.tar.gz -C /path/to/workspace .

in directory want to compress (current directory) try this :

tar -czf workspace.tar.gz . --exclude=./*.gz