[gzip] How to check if a Unix .tar.gz file is a valid file without uncompressing?

I have found the question How to determine if data is valid tar file without a file?, but I was wondering: is there a ready made command line solution?

This question is related to gzip validation tar gunzip

The answer is


you could probably use the gzip -t option to test the files integrity

http://linux.about.com/od/commands/l/blcmdl1_gzip.htm

from: http://unix.ittoolbox.com/groups/technical-functional/shellscript-l/how-to-test-file-integrity-of-targz-1138880

To test the gzip file is not corrupt:

gunzip -t file.tar.gz

To test the tar file inside is not corrupt:

gunzip -c file.tar.gz | tar -t > /dev/null

As part of the backup you could probably just run the latter command and check the value of $? afterwards for a 0 (success) value. If either the tar or the gzip has an issue, $? will have a non zero value.


A nice option is to use tar -tvvf <filePath> which adds a line that reports the kind of file.

Example in a valid .tar file:

> tar -tvvf filename.tar 
drwxr-xr-x  0 diegoreymendez staff       0 Jul 31 12:46 ./testfolder2/
-rw-r--r--  0 diegoreymendez staff      82 Jul 31 12:46 ./testfolder2/._.DS_Store
-rw-r--r--  0 diegoreymendez staff    6148 Jul 31 12:46 ./testfolder2/.DS_Store
drwxr-xr-x  0 diegoreymendez staff       0 Jul 31 12:42 ./testfolder2/testfolder/
-rw-r--r--  0 diegoreymendez staff      82 Jul 31 12:42 ./testfolder2/testfolder/._.DS_Store
-rw-r--r--  0 diegoreymendez staff    6148 Jul 31 12:42 ./testfolder2/testfolder/.DS_Store
-rw-r--r--  0 diegoreymendez staff  325377 Jul  5 09:50 ./testfolder2/testfolder/Scala.pages
Archive Format: POSIX ustar format,  Compression: none

Corrupted .tar file:

> tar -tvvf corrupted.tar 
tar: Unrecognized archive format
Archive Format: (null),  Compression: none
tar: Error exit delayed from previous errors.

These are all very sub-optimal solutions. From the GZIP spec

ID2 (IDentification 2)
These have the fixed values ID1 = 31 (0x1f, \037), ID2 = 139 (0x8b, \213), to identify the file as being in gzip format.

Has to be coded into whatever language you're using.


> use the -O option. [...] If the tar file is corrupt, the process will abort with an error.

Sometimes yes, but sometimes not. Let's see an example of a corrupted file:

echo Pete > my_name
tar -cf my_data.tar my_name 

# // Simulate a corruption
sed < my_data.tar 's/Pete/Fool/' > my_data_now.tar
# // "my_data_now.tar" is the corrupted file

tar -xvf my_data_now.tar -O

It shows:

my_name
Fool  

Even if you execute

echo $?

tar said that there was no error:

0

but the file was corrupted, it has now "Fool" instead of "Pete".


I have tried the following command and they work well.

bzip2 -t file.bz2
gunzip -t file.gz

However, we can found these two command are time-consuming. Maybe we need some more quick way to determine the intact of the compress files.


You can also check contents of *.tag.gz file using pigz (parallel gzip) to speedup the archive check:

pigz -cvdp number_of_threads /[...]path[...]/archive_name.tar.gz | tar -tv > /dev/null

If you want to do a real test extract of a tar file without extracting to disk, use the -O option. This spews the extract to standard output instead of the filesystem. If the tar file is corrupt, the process will abort with an error.

Example of failed tar ball test...

$ echo "this will not pass the test" > hello.tgz
$ tar -xvzf hello.tgz -O > /dev/null
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors
$ rm hello.*

Working Example...

$ ls hello*
ls: hello*: No such file or directory
$ echo "hello1" > hello1.txt
$ echo "hello2" > hello2.txt
$ tar -cvzf hello.tgz hello[12].txt
hello1.txt
hello2.txt
$ rm hello[12].txt
$ ls hello*
hello.tgz
$ tar -xvzf hello.tgz -O
hello1.txt
hello1
hello2.txt
hello2
$ ls hello*
hello.tgz
$ tar -xvzf hello.tgz
hello1.txt
hello2.txt
$ ls hello*
hello1.txt  hello2.txt  hello.tgz
$ rm hello*

Examples related to gzip

gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now How to unzip gz file using Python How to enable GZIP compression in IIS 7.5 Using GZIP compression with Spring Boot/MVC/JavaConfig with RESTful How are zlib, gzip and zip related? What do they have in common and how are they different? How to uncompress a tar.gz in another directory compression and decompression of string data in java Extract and delete all .gz in a directory- Linux How to extract filename.tar.gz file Read from a gzip file in python

Examples related to validation

Rails 2.3.4 Persisting Model on Validation Failure Input type number "only numeric value" validation How can I manually set an Angular form field as invalid? Laravel Password & Password_Confirmation Validation Reactjs - Form input validation Get all validation errors from Angular 2 FormGroup Min / Max Validator in Angular 2 Final How to validate white spaces/empty spaces? [Angular 2] How to Validate on Max File Size in Laravel? WebForms UnobtrusiveValidationMode requires a ScriptResourceMapping for jquery

Examples related to tar

gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now Create a .tar.bz2 file Linux what does -zxvf mean in tar -zxvf <filename>? tar: file changed as we read it Create a tar.xz in one command How to tar certain file types in all subdirectories? Tar a directory, but don't store full absolute paths in the archive How to uncompress a tar.gz in another directory How to extract filename.tar.gz file Utilizing multi core for tar+gzip/bzip compression/decompression

Examples related to gunzip

Extract and delete all .gz in a directory- Linux How to check if a Unix .tar.gz file is a valid file without uncompressing?