SHA-256 or MD5 for file integrity

Question

I know that SHA-256 is favored over MD5 for security  etc   but  if I am to use a method to only check file integrity  that is  nothing to do with password encryption  etc    is there any advantage of using SHA-256   Since MD5 is 128-bit and SHA-256 is 256-bit  therefore twice as big       Would it take up to twice as long to encrypt  Where time is not of essence  like in a backup program  and file integrity is all that is needed  would anyone argue against MD5 for a different algorithm  or even suggest a different technique  Does using MD5 produce a checksum

User · Answer

Every answer seems to suggest that you need to use secure hashes to do the job but all of these are tuned to be slow to force a bruteforce attacker to have lots of computing power and depending on your needs this may not be the best solution.

There are algorithms specifically designed to hash files as fast as possible to check integrity and comparison (murmur, XXhash...). Obviously these are not designed for security as they don't meet the requirements of a secure hash algorithm (i.e. randomness) but have low collision rates for large messages. This features make them ideal if you are not looking for security but speed.

Examples of this algorithms and comparison can be found in this excellent answer: Which hashing algorithm is best for uniqueness and speed?.

As an example, we at our Q&A site use murmur3 to hash the images uploaded by the users so we only store them once even if users upload the same image in several answers.

User · Answer

To 1   Yes  on most CPUs  SHA-256 is about only 40  as fast as MD5  To 2   I would argue for a different algorithm than MD5 in such a case  I would definitely prefer an algorithm that is considered safe  However  this is more a feeling  Cases where this matters would be rather constructed than realistic  e g  if your backup system encounters an example case of an attack on an MD5-based certificate  you are likely to have two files in such an example with different data  but identical MD5 checksums  For the rest of the cases  it doesn t matter  because MD5 checksums have a collision    same checksums for different data  virtually only when provoked intentionally  I m not an expert on the various hashing  checksum generating  algorithms  so I can not suggest another algorithm  Hence this part of the question is still open  Suggested further reading is Cryptographic Hash Function - File or Data Identifier on Wikipedia  Also further down on that page there is a list of cryptographic hash algorithms  To 3   MD5 is an algorithm to calculate checksums  A checksum calculated using this algorithm is then called an MD5 checksum

User · Answer

Both SHA256 and MDA5 are hashing algorithms  They take your input data  in this case your file  and output a 256 128-bit number  This number is a checksum  There is no encryption taking place because an infinite number of inputs can result in the same hash value  although in reality collisions are rare   SHA256 takes somewhat more time to calculate than MD5  according to this answer    Offhand  I d say that MD5 would be probably be suitable for what you need

User · Answer

The underlying MD5 algorithm is no longer deemed secure  thus while md5sum is well-suited for identifying known files in situations that are not security related  it should not be relied on if there is a chance that files have been purposefully and maliciously tampered  In the latter case  the use of a newer hashing tool such as sha256sum is highly recommended   So  if you are simply looking to check for file corruption or file differences  when the source of the file is trusted  MD5 should be sufficient   If you are looking to verify the integrity of a file coming from an untrusted source  or over from a trusted source over an unencrypted connection  MD5 is not sufficient   Another commenter noted that Ubuntu and others use MD5 checksums   Ubuntu has moved to PGP and SHA256  in addition to MD5  but the documentation of the stronger verification strategies are more difficult to find   See the HowToSHA256SUM page for more details

User · Answer

No  it s less fast but not that slow For a backup program it s maybe necessary to have something even faster than MD5   All in all  I d say that MD5 in addition to the file name is absolutely safe  SHA-256 would just be slower and harder to handle because of its size   You could also use something less secure than MD5 without any problem  If nobody tries to hack your file integrity this is safe  too

User · Answer

It is technically approved that MD5 is faster than SHA256 so in just verifying file integrity it will be sufficient and better for performance   You are able to checkout the following resources    Speed Comparison of Popular Crypto Algorithms Comparison of cryptographic hash functions

User · Answer

Yes  on most CPUs  SHA-256 is two to three times slower than MD5  though not primarily because of its longer hash  See other answers here and the answers to this Stack Overflow questions  Here s a backup scenario where MD5 would not be appropriate    Your backup program hashes each file being backed up  It then stores each file s data by its hash  so if you re backing up the same file twice you only end up with one copy of it  An attacker can cause the system to backup files they control  The attacker knows the MD5 hash of a file they want to remove from the backup  The attacker can then use the known weaknesses of MD5 to craft a new file that has the same hash as the file to remove  When that file is backed up  it will replace the file to remove  and that file s backed up data will be lost  This backup system could be strengthened a bit  and made more efficient  by not replacing files whose hash it has previously encountered  but then an attacker could prevent a target file with a known hash from being backed up by preemptively backing up a specially constructed bogus file with the same hash  Obviously most systems  backup and otherwise  do not satisfy the conditions necessary for this attack to be practical  but I just wanted to give an example of a situation where SHA-256 would be preferable to MD5  Whether this would be the case for the system you re creating depends on more than just the characteristics of MD5 and SHA-256   Yes  cryptographic hashes like the ones generated by MD5 and SHA-256 are a type of checksum    Happy hashing

[file] SHA-256 or MD5 for file integrity

Examples related to file

Examples related to hash

Examples related to md5

Examples related to sha256