Is calculating an MD5 hash less CPU intensive than SHA family functions

Question

Is calculating an MD5 hash less CPU intensive than SHA-1 or SHA-2 on  standard  laptop x86 hardware  I m interested in general information  not specific to a certain chip   UPDATE  In my case  I m interested in calculating the hash of a file  If file-size matters  let s assume its 300K

User · Answer

sha1sum is quite a bit faster on Power9 than md5sum    uname -mov  1 SMP Mon May 13 12 16 08 EDT 2019 ppc64le GNU Linux    cat  proc cpuinfo processor         0 cpu               POWER9  altivec supported clock             2166 000000MHz revision          2 2  pvr 004e 1202     ls -l linux-master tar -rw-rw-r-- 1 x x 829685760 Jan 29 14 30 linux-master tar    time sha1sum linux-master tar 10fbf911e254c4fe8e5eb2e605c6c02d29a88563  linux-master tar  real    0m1 685s user    0m1 528s sys     0m0 156s    time md5sum linux-master tar d476375abacda064ae437a683c537ec4  linux-master tar  real    0m2 942s user    0m2 806s sys     0m0 136s    time sum linux-master tar 36928 810240  real    0m2 186s user    0m1 917s sys     0m0 268s

User · Answer

Yes  MD5 is somewhat less CPU-intensive  On my Intel x86  Core2 Quad Q6600  2 4 GHz  using one core   I get this in 32-bit mode   MD5       411 SHA-1     218 SHA-256   118 SHA-512    46   and this in 64-bit mode   MD5       407 SHA-1     312 SHA-256   148 SHA-512   189   Figures are in megabytes per second  for a  long  message  this is what you get for messages longer than 8 kB   This is with sphlib  a library of hash function implementations in C  and Java   All implementations are from the same author  me  and were made with comparable efforts at optimizations  thus the speed differences can be considered as really intrinsic to the functions   As a point of comparison  consider that a recent hard disk will run at about 100 MB s  and anything over USB will top below 60 MB s  Even though SHA-256 appears  slow  here  it is fast enough for most purposes   Note that OpenSSL includes a 32-bit implementation of SHA-512 which is quite faster than my code  but not as fast as the 64-bit SHA-512   because the OpenSSL implementation is in assembly and uses SSE2 registers  something which cannot be done in plain C  SHA-512 is the only function among those four which benefits from a SSE2 implementation   Edit  on this page  archive   one can find a report on the speed of many hash functions  click on the  Telechargez maintenant  link   The report is in French  but it is mostly full of tables and numbers  and numbers are international  The implemented hash functions do not include the SHA-3 candidates  except SHABAL  but I am working on it

User · Answer

As someone who s spent a bit of time optimizing MD5 performance  I thought I d supply more of a technical explanation than the benchmarks provided here  to anyone who happens to find this in the future  MD5 does less  quot work quot  than SHA1  e g  fewer compression rounds   so one may think it should be faster   However  the MD5 algorithm is mostly one big dependency chain  which means that it doesn t exploit modern superscalar processors particularly well  i e  exhibits low instructions-per-clock   SHA1 has more parallelism available  so despite needing more  quot computational work quot  done  it often ends up being faster than MD5 on modern superscalar processors  If you do the MD5 vs SHA1 comparison on older processors or ones with less superscalar  quot width quot   such as a Silvermont based Atom CPU   you ll generally find MD5 is faster than SHA1  SHA2 and SHA3 are even more compute intensive than SHA1  and generally much slower  One thing to note  however  is that some new x86 and ARM CPUs have instructions to accelerate SHA1 and SHA256  which obviously helps these algorithms greatly if the instructions are being used  As an aside  SHA256 and SHA512 performance may exhibit similarly curious behaviour  SHA512 does more  quot work quot  than SHA256  however a key difference between the two is that SHA256 operates using 32-bit words  whilst SHA512 operates using 64-bit words  As such  SHA512 will generally be faster than SHA256 on a platform with a 64-bit word size  as it s processing twice the amount of data at once  Conversely  SHA256 should outperform SHA512 on a platform with a 32-bit word size  Note that all of the above only applies to single buffer hashing  by far the most common use case   If you re fancy and computing multiple hashes in parallel  i e  a multi-buffer SIMD approach  the behaviour changes somewhat

User · Answer

On my 2012 MacBook Air  Intel Core i5-3427U  2x 1 8 GHz  2 8 GHz Turbo   SHA-1 is slightly faster than MD5  using OpenSSL in 64-bit mode      openssl speed md5 sha1 OpenSSL 0 9 8r 8 Feb 2011 The  numbers  are in 1000s of bytes per second processed  type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes md5              30055 02k    94158 96k   219602 97k   329008 21k   384150 47k sha1             31261 12k    95676 48k   224357 36k   332756 21k   396864 62k   Update  10 months later with OS X 10 9  SHA-1 got slower on the same machine     openssl speed md5 sha1 OpenSSL 0 9 8y 5 Feb 2013 The  numbers  are in 1000s of bytes per second processed  type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes md5              36277 35k   106558 04k   234680 17k   334469 33k   381756 70k sha1             35453 52k    99530 85k   206635 24k   281695 48k   313881 86k   Second update  On OS X 10 10  SHA-1 speed is back to the 10 8 level     openssl speed md5 sha1 OpenSSL 0 9 8zc 15 Oct 2014 The  numbers  are in 1000s of bytes per second processed  type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes md5              35391 50k   104905 27k   229872 93k   330506 91k   382791 75k sha1             38054 09k   110332 44k   238198 72k   340007 12k   387137 77k   Third update  OS X 10 14 with LibreSSL is a lot faster  still on the same machine   SHA-1 still comes out on top     openssl speed md5 sha1 LibreSSL 2 6 5 The  numbers  are in 1000s of bytes per second processed  type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes md5              43128 00k   131797 91k   304661 16k   453120 00k   526789 29k sha1             55598 35k   157916 03k   343214 08k   489092 34k   570668 37k

User · Answer

The real answer is   It depends  There are a couple factors to consider  the most obvious are   the cpu you are running these algorithms on and the implementation of the algorithms   For instance  me and my friend both run the exact same openssl version and get slightly different results with different Intel Core i7 cpus   My test at work with an Intel R  Core TM  i7-2600 CPU   3 40GHz  The  numbers  are in 1000s of bytes per second processed  type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes md5              64257 97k   187370 26k   406435 07k   576544 43k   649827 67k sha1             73225 75k   202701 20k   432679 68k   601140 57k   679900 50k   And his with an Intel R  Core TM  i7 CPU 920    2 67GHz  The  numbers  are in 1000s of bytes per second processed  type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes md5              51859 12k   156255 78k   350252 00k   513141 73k   590701 52k sha1             56492 56k   156300 76k   328688 76k   452450 92k   508625 68k   We both are running the exact same binaries of OpenSSL 1 0 1j 15 Oct 2014 from the ArchLinux official package   My opinion on this is that with the added security of sha1  cpu designers are more likely to improve the speed of sha1 and more programmers will be working on the algorithm s optimization than md5sum    I guess that md5 will no longer be used some day since it seems that it has no advantage over sha1  I also tested some cases on real files and the results were always the same in both cases  likely limited by disk I O    md5sum of a large 4 6GB file took the exact same time than sha1sum of the same file  same goes with many small files  488 in the same directory   I ran the tests a dozen times and they were consitently getting the same results   --  It would be very interesting to investigate this further  I guess there are some experts around that could provide a solid answer to why sha1 is getting faster than md5 on newer processors

User · Answer

MD5 also benefits from SSE2 usage  check out BarsWF and then tell me that it doesn t  All it takes is a little assembler knowledge and you can craft your own MD5 SSE2 routine s   For large amounts of throughput however  there is a tradeoff of the speed during hashing as opposed to the time spent rearranging the input data to be compatible with the SIMD instructions used

[hash] Is calculating an MD5 hash less CPU intensive than SHA family functions?

Examples related to hash

Examples related to cryptography

Examples related to md5

Examples related to sha1

Examples related to sha2