Comparing two byte arrays in NET

Question

How can I do this fast   Sure I can do this   static bool ByteArrayCompare byte   a1  byte   a2        if  a1 Length    a2 Length          return false       for  int i 0  i lt a1 Length  i            if  a1 i   a2 i               return false       return true      But I m looking for either a BCL function or some highly optimized proven way to do this   java util Arrays equals  sbyte    Array a1   sbyte    Array a2     works nicely  but it doesn t look like that would work for x64   Note my super-fast answer here

User · Answer

If you look at how  NET does string Equals  you see that it uses a private method called EqualsHelper which has an  unsafe  pointer implementation   NET Reflector is your friend to see how things are done internally   This can be used as a template for byte array comparison which I did an implementation on in blog post Fast byte array comparison in C   I also did some rudimentary benchmarks to see when a safe implementation is faster than the unsafe   That said  unless you really need killer performance  I d go for a simple fr loop comparison

User · Answer

There s a new built-in solution for this in  NET 4 - IStructuralEquatable  static bool ByteArrayCompare byte   a1  byte   a2         return StructuralComparisons StructuralEqualityComparer Equals a1  a2

User · Answer

This is almost certainly much slower than any other version given here  but it was fun to write   static bool ByteArrayEquals byte   a1  byte   a2         return a1 Zip a2   l  r    gt  l    r  All x   gt  x

User · Answer

Sorry  if you re looking for a managed way you re already doing it correctly and to my knowledge there s no built in method in the BCL for doing this   You should add some initial null checks and then just reuse it as if it where in BCL

User · Answer

P Invoke powers activate    DllImport  msvcrt dll   CallingConvention CallingConvention Cdecl   static extern int memcmp byte   b1  byte   b2  long count    static bool ByteArrayCompare byte   b1  byte   b2           Validate buffers are the same length         This also ensures that the count does not exceed the length of either buffer        return b1 Length    b2 Length  amp  amp  memcmp b1  b2  b1 Length     0

User · Answer

using System Linq    SequenceEqual   byte   ByteArray1   null   byte   ByteArray2   null    ByteArray1   MyFunct1     ByteArray2   MyFunct2      if  ByteArray1 SequenceEqual lt byte gt  ByteArray2     true         MessageBox Show  Match        else       MessageBox Show  Don t match

User · Answer

You can use Enumerable SequenceEqual method   using System  using System Linq      var a1   new int     1  2  3   var a2   new int     1  2  3   var a3   new int     1  2  4   var x   a1 SequenceEqual a2      true var y   a1 SequenceEqual a3      false   If you can t use  NET 3 5 for some reason  your method is OK  Compiler run-time environment will optimize your loop so you don t need to worry about performance

User · Answer

If you are looking for a very fast byte array equality comparer  I suggest you take a look at this STSdb Labs article  Byte array equality comparer  It features some of the fastest implementations for byte   array equality comparing  which are presented  performance tested and summarized   You can also focus on these implementations   BigEndianByteArrayComparer - fast byte   array comparer from left to right  BigEndian  BigEndianByteArrayEqualityComparer - - fast byte   equality comparer from left to right  BigEndian  LittleEndianByteArrayComparer - fast byte   array comparer from right to left  LittleEndian  LittleEndianByteArrayEqualityComparer - fast byte   equality comparer from right to left  LittleEndian

User · Answer

I settled on a solution inspired by the EqualBytesLongUnrolled method posted by ArekBulski with an additional optimization  In my instance  array differences in arrays tend to be near the tail of the arrays  In testing  I found that when this is the case for large arrays  being able to compare array elements in reverse order gives this solution a huge performance gain over the memcmp based solution  Here is that solution   public enum CompareDirection   Forward  Backward    private static unsafe bool UnsafeEquals byte   a  byte   b  CompareDirection direction   CompareDirection Forward           returns when a and b are same array or both null     if  a    b  return true          if either is null or different lengths  can t be equal     if  a    null    b    null    a Length    b Length          return false       const int UNROLLED   16                    count of longs  unrolled  in optimization     int size   sizeof long    UNROLLED         128 bytes  min size for  unrolled  optimization      int len   a Length      int n   len   size             count of full 128 byte segments     int r   len   size             count of remaining  unoptimized  bytes         pin the arrays and access them via pointers     fixed  byte  pb a   a  pb b   b                if  r  gt  0  amp  amp  direction    CompareDirection Backward                        byte  pa   pb a   len - 1              byte  pb   pb b   len - 1              byte  phead   pb a   len - r              while pa  gt   phead                                if   pa     pb  return false                  pa--                  pb--                                   if  n  gt  0                        int nOffset   n   size              if  direction    CompareDirection Forward                                long  pa    long  pb a                  long  pb    long  pb b                  long  ptail    long   pb a   nOffset                   while  pa  lt  ptail                                        if    pa   0       pb   0       pa   1       pb   1                               pa   2       pb   2       pa   3       pb   3                               pa   4       pb   4       pa   5       pb   5                               pa   6       pb   6       pa   7       pb   7                               pa   8       pb   8       pa   9       pb   9                               pa   10       pb   10       pa   11       pb   11                               pa   12       pb   12       pa   13       pb   13                               pa   14       pb   14       pa   15       pb   15                                                                      return false                                            pa    UNROLLED                      pb    UNROLLED                                              else                               long  pa    long   pb a   nOffset                   long  pb    long   pb b   nOffset                   long  phead    long  pb a                  while  phead  lt  pa                                        if    pa - 1       pb - 1       pa - 2       pb - 2                               pa - 3       pb - 3       pa - 4       pb - 4                               pa - 5       pb - 5       pa - 6       pb - 6                               pa - 7       pb - 7       pa - 8       pb - 8                               pa - 9       pb - 9       pa - 10       pb - 10                               pa - 11       pb - 11       pa - 12       pb - 12                               pa - 13       pb - 13       pa - 14       pb - 14                               pa - 15       pb - 15       pa - 16       pb - 16                                                                      return false                                            pa -  UNROLLED                      pb -  UNROLLED                                                     if  r  gt  0  amp  amp  direction    CompareDirection Forward                        byte  pa   pb a   len - r              byte  pb   pb b   len - r              byte  ptail   pb a   len              while pa  lt  ptail                                if   pa     pb  return false                  pa                    pb                                       return true

User · Answer

Since many of the fancy solutions above don t work with UWP and because I love Linq and functional approaches I pressent you my version to this problem  To escape the comparison when the first difference occures  I chose  FirstOrDefault    public static bool CompareByteArrays byte   ba0  byte   ba1    gt        ba0 Length    ba1 Length    Enumerable Range 1 ba0 Length           FirstOrDefault n   gt  ba0 n     ba1 n    gt  0

User · Answer

I posted a similar question about checking if byte   is full of zeroes   SIMD code was beaten so I removed it from this answer   Here is fastest code from my comparisons   static unsafe bool EqualBytesLongUnrolled  byte   data1  byte   data2        if  data1    data2          return true      if  data1 Length    data2 Length          return false       fixed  byte  bytes1   data1  bytes2   data2            int len   data1 Length          int rem   len    sizeof long    16           long  b1    long  bytes1          long  b2    long  bytes2          long  e1    long   bytes1   len - rem            while  b1  lt  e1                if    b1       b2       b1   1       b2   1                        b1   2       b2   2       b1   3       b2   3                       b1   4       b2   4       b1   5       b2   5                        b1   6       b2   6       b1   7       b2   7                       b1   8       b2   8       b1   9       b2   9                        b1   10       b2   10       b1   11       b2   11                       b1   12       b2   12       b1   13       b2   13                        b1   14       b2   14       b1   15       b2   15                   return false              b1    16              b2    16                     for  int i   0  i  lt  rem  i                if  data1  len - 1 - i     data2  len - 1 - i                   return false           return true            Measured on two 256MB byte arrays   UnsafeCompare                             86 8784 ms EqualBytesSimd                            71 5125 ms EqualBytesSimdUnrolled                    73 1917 ms EqualBytesLongUnrolled                    39 8623 ms

User · Answer

I would use unsafe code and run the for loop comparing Int32 pointers   Maybe you should also consider checking the arrays to be non-null

User · Answer

If you are not opposed to doing it  you can import the J  assembly  vjslib dll  and use its Arrays equals byte    byte    method     Don t blame me if someone laughs at you though       EDIT  For what little it is worth  I used Reflector to disassemble the code for that  and here is what it looks like   public static bool equals sbyte   a1  sbyte   a2      if  a1    a2          return true        if   a1    null   amp  amp   a2    null           if  a1 Length    a2 Length              return false            for  int i   0  i  lt  a1 Length  i                if  a1 i     a2 i                   return false                    return true        return false

User · Answer

I have not seen many linq solutions here   I am not sure of the performance implications  however I generally stick to linq as rule of thumb and then optimize later if necessary   public bool CompareTwoArrays byte   array1  byte   array2        return  array1 Where  t  i    gt  t    array2 i   Any         Please do note this only works if they are the same size arrays  an extension could look like so  public bool CompareTwoArrays byte   array1  byte   array2        if  array1 Length    array2 Length  return false     return  array1 Where  t  i    gt  t    array2 i   Any

User · Answer

For those of you that care about order  i e  want your memcmp to return an int like it should instead of nothing    NET Core 3 0  and presumably  NET Standard 2 1 aka  NET 5 0  will include a Span SequenceCompareTo      extension method  plus a Span SequenceEqualTo  that can be used to compare two ReadOnlySpan lt T gt  instances  where T  IComparable lt T gt      In the original GitHub proposal  the discussion included approach comparisons with jump table calculations  reading a byte   as long    SIMD usage  and p invoke to the CLR implementation s memcmp   Going forward  this should be your go-to method for comparing byte arrays or byte ranges  as should using Span lt byte gt  instead of byte   for your  NET Standard 2 1 APIs   and it is sufficiently fast enough that you should no longer care about optimizing it  and no  despite the similarities in name it does not perform as abysmally as the horrid Enumerable SequenceEqual     if NETCOREAPP3 0    Using the platform-native Span lt T gt  SequenceEqual lt T gt      public static int Compare byte   range1  int offset1  byte   range2  int offset2  int count        var span1   range1 AsSpan offset1  count       var span2   range2 AsSpan offset2  count        return span1 SequenceCompareTo span2          or  if you don t care about ordering        return span1 SequenceEqual span2      else    The most basic implementation  in platform-agnostic  safe C  public static bool Compare byte   range1  int offset1  byte   range2  int offset2  int count           Working backwards lets the compiler optimize away bound checking after the first loop     for  int i   count - 1  i  gt   0  --i                if  range1 offset1   i     range2 offset2   i                         return false                       return true     endif

User · Answer

Use SequenceEquals for this to comparison

User · Answer

User gil suggested unsafe code which spawned this solution      Copyright  c  2008-2013 Hafthor Stefansson    Distributed under the MIT X11 software license    Ref  http   www opensource org licenses mit-license php  static unsafe bool UnsafeCompare byte   a1  byte   a2      if a1  a2  return true    if a1  null    a2  null    a1 Length  a2 Length      return false    fixed  byte  p1 a1  p2 a2        byte  x1 p1  x2 p2      int l   a1 Length      for  int i 0  i  lt  l 8  i    x1  8  x2  8        if     long  x1        long  x2   return false      if   l  amp  4   0    if     int  x1      int  x2   return false  x1  4  x2  4        if   l  amp  2   0    if     short  x1      short  x2   return false  x1  2  x2  2        if   l  amp  1   0  if     byte  x1        byte  x2   return false      return true          which does 64-bit based comparison for as much of the array as possible  This kind of counts on the fact that the arrays start qword aligned  It ll work if not qword aligned  just not as fast as if it were   It performs about seven timers faster than the simple for loop  Using the J  library performed equivalently to the original for loop  Using  SequenceEqual runs around seven times slower  I think just because it is using IEnumerator MoveNext  I imagine LINQ-based solutions being at least that slow or worse

User · Answer

Couldn t find a solution I m completely happy with  reasonable performance  but no unsafe code pinvoke  so I came up with this  nothing really original  but works            lt summary gt                    lt  summary gt           lt param name  array1  gt  lt  param gt           lt param name  array2  gt  lt  param gt           lt param name  bytesToCompare  gt  0 means compare entire arrays lt  param gt           lt returns gt  lt  returns gt      public static bool ArraysEqual byte   array1  byte   array2  int bytesToCompare   0                if  array1 Length    array2 Length  return false           var length    bytesToCompare    0    array1 Length   bytesToCompare          var tailIdx   length - length   sizeof Int64              check in 8 byte chunks         for  var i   0  i  lt  tailIdx  i    sizeof Int64                         if  BitConverter ToInt64 array1  i     BitConverter ToInt64 array2  i   return false                       check the remainder of the array  always shorter than 8 bytes         for  var i   tailIdx  i  lt  length  i                          if  array1 i     array2 i   return false                     return true          Performance compared with some of the other solutions on this page   Simple Loop  19837 ticks  1 00   BitConverter  4886 ticks  4 06  UnsafeCompare  1636 ticks  12 12  EqualBytesLongUnrolled  637 ticks  31 09  P Invoke memcmp  369 ticks  53 67  Tested in linqpad  1000000 bytes identical arrays  worst case scenario   500 iterations each

User · Answer

The short answer is this       public bool Compare byte   b1  byte   b2                return Encoding ASCII GetString b1     Encoding ASCII GetString b2           In such a way you can use the optimized  NET string compare to make a byte array compare without the need to write unsafe code  This is how it is done in the background   private unsafe static bool EqualsHelper String strA  String strB        Contract Requires strA    null       Contract Requires strB    null       Contract Requires strA Length    strB Length        int length   strA Length       fixed  char  ap    amp strA m firstChar  fixed  char  bp    amp strB m firstChar                char  a   ap          char  b   bp              Unroll the loop           if AMD64                For the AMD64 bit platform we unroll by 12 and                check three qwords at a time  This is less code                than the 32 bit case and is shorter                pathlength               while  length  gt   12                                if    long  a          long  b      return false                  if    long   a 4       long   b 4   return false                  if    long   a 8       long   b 8   return false                  a    12  b    12  length -  12                        else            while  length  gt   10                              if    int  a      int  b  return false                 if    int   a 2       int   b 2   return false                 if    int   a 4       int   b 4   return false                 if    int   a 6       int   b 6   return false                 if    int   a 8       int   b 8   return false                 a    10  b    10  length -  10                       endif             This depends on the fact that the String objects are            always zero terminated and that the terminating zero is not included            in the length  For odd string sizes  the last compare will include            the zero terminator          while  length  gt  0                        if    int  a      int  b  break              a    2  b    2  length -  2                     return  length  lt   0

User · Answer

I thought about block-transfer acceleration methods built into many graphics cards  But then you would have to copy over all the data byte-wise  so this doesn t help you much if you don t want to implement a whole portion of your logic in unmanaged and hardware-dependent code     Another way of optimization similar to the approach shown above would be to store as much of your data as possible in a long   rather than a byte   right from the start  for example if you are reading it sequentially from a binary file  or if you use a memory mapped file  read in data as long   or single long values  Then  your comparison loop will only need 1 8th of the number of iterations it would have to do for a byte   containing the same amount of data  It is a matter of when and how often you need to compare vs  when and how often you need to access the data in a byte-by-byte manner  e g  to use it in an API call as a parameter in a method that expects a byte    In the end  you only can tell if you really know the use case

User · Answer

Span lt T gt  offers an extremely competitive alternative without having to throw confusing and or non-portable fluff into your own application s code base     byte   is implicitly convertible to ReadOnlySpan lt byte gt  static bool ByteArrayCompare ReadOnlySpan lt byte gt  a1  ReadOnlySpan lt byte gt  a2        return a1 SequenceEqual a2      The  guts of the  implementation as of  NET 5 0 0 can be found here  I ve revised  EliArbel s gist to add this method as SpansEqual  drop most of the less interesting performers in others  benchmarks  run it with different array sizes  output graphs  and mark SpansEqual as the baseline so that it reports how the different methods compare to SpansEqual  The below numbers are from the results  lightly edited to remove  quot Error quot  column           Method    ByteCount                 Mean              StdDev   Ratio   RatioSD    --------------  -----------  -------------------  ------------------  ------  --------        SpansEqual           15             4 629 ns           0 0289 ns    1 00      0 00      LongPointers           15             4 598 ns           0 0416 ns    0 99      0 01          Unrolled           15            18 199 ns           0 0291 ns    3 93      0 02     PInvokeMemcmp           15             9 872 ns           0 0441 ns    2 13      0 02                                                                                                  SpansEqual         1026            19 965 ns           0 0880 ns    1 00      0 00      LongPointers         1026            63 005 ns           0 5217 ns    3 16      0 04          Unrolled         1026            38 731 ns           0 0166 ns    1 94      0 01     PInvokeMemcmp         1026            40 355 ns           0 0202 ns    2 02      0 01                                                                                                  SpansEqual      1048585        43 761 339 ns          30 8744 ns    1 00      0 00      LongPointers      1048585        59 585 479 ns          17 3907 ns    1 36      0 00          Unrolled      1048585        54 646 243 ns          35 7638 ns    1 25      0 00     PInvokeMemcmp      1048585        55 198 289 ns          23 9732 ns    1 26      0 00                                                                                                  SpansEqual   2147483591   240 607 692 857 ns   2 733 489 4894 ns    1 00      0 00      LongPointers   2147483591   238 223 478 571 ns   2 033 769 5979 ns    0 99      0 02          Unrolled   2147483591   236 227 340 000 ns   2 189 627 0164 ns    0 98      0 00     PInvokeMemcmp   2147483591   238 724 660 000 ns   3 726 140 4720 ns    0 99      0 02    I was surprised to see SpansEqual not come out on top for the max-array-size methods  but the difference is so minor that I don t think it ll ever matter  My system info  BenchmarkDotNet v0 12 1  OS Windows 10 0 19042 Intel Core i7-6850K CPU 3 60GHz  Skylake   1 CPU  12 logical and 6 physical cores  NET Core SDK 5 0 100    Host         NET Core 5 0 0  CoreCLR 5 0 20 51904  CoreFX 5 0 20 51904   X64 RyuJIT   DefaultJob    NET Core 5 0 0  CoreCLR 5 0 20 51904  CoreFX 5 0 20 51904   X64 RyuJIT

User · Answer

Let s add one more   Recently Microsoft released a special NuGet package  System Runtime CompilerServices Unsafe  It s special because it s written in IL  and provides low-level functionality not directly available in C    One of its methods  Unsafe As lt T gt  object  allows casting any reference type to another reference type  skipping any safety checks  This is usually a very bad idea  but if both types have the same structure  it can work  So we can use this to cast a byte   to a long     bool CompareWithUnsafeLibrary byte   a1  byte   a2        if  a1 Length    a2 Length  return false       var longSize    int Math Floor a1 Length   8 0       var long1   Unsafe As lt long   gt  a1       var long2   Unsafe As lt long   gt  a2        for  var i   0  i  lt  longSize  i                  if  long1 i     long2 i   return false             for  var i   longSize   8  i  lt  a1 Length  i                  if  a1 i     a2 i   return false             return true      Note that long1 Length would still return the original array s length  since it s stored in a field in the array s memory structure   This method is not quite as fast as other methods demonstrated here  but it is a lot faster than the naive method  doesn t use unsafe code or P Invoke or pinning  and the implementation is quite straightforward  IMO   Here are some BenchmarkDotNet results from my machine   BenchmarkDotNet v0 10 3 0  OS Microsoft Windows NT 6 2 9200 0 Processor Intel R  Core TM  i7-4870HQ CPU 2 50GHz  ProcessorCount 8 Frequency 2435775 Hz  Resolution 410 5470 ns  Timer TSC    Host        Clr 4 0 30319 42000  64bit RyuJIT-v4 6 1637 0   DefaultJob   Clr 4 0 30319 42000  64bit RyuJIT-v4 6 1637 0                   Method            Mean      StdDev   -----------------------  --------------  ----------             UnsafeLibrary     125 8229 ns   0 3588 ns             UnsafeCompare      89 9036 ns   0 8243 ns              JSharpEquals   1 432 1717 ns   1 3161 ns    EqualBytesLongUnrolled      43 7863 ns   0 8923 ns                 NewMemCmp      65 4108 ns   0 2202 ns               ArraysEqual     910 8372 ns   2 6082 ns             PInvokeMemcmp      52 7201 ns   0 1105 ns     I ve also created a gist with all the tests

User · Answer

NET 3 5 and newer have a new public type  System Data Linq Binary that encapsulates byte    It implements IEquatable lt Binary gt  that  in effect  compares two byte arrays  Note that System Data Linq Binary also has implicit conversion operator from byte     MSDN documentation System Data Linq Binary  Reflector decompile of the Equals method   private bool EqualsTo Binary binary        if  this    binary                if  binary    null                        return false                    if  this bytes Length    binary bytes Length                        return false                    if  this hashCode    binary hashCode                        return false                    int index   0          int length   this bytes Length          while  index  lt  length                        if  this bytes index     binary bytes index                                 return false                            index                        return true      Interesting twist is that they only proceed to byte-by-byte comparison loop if hashes of the two Binary objects are the same  This  however  comes at the cost of computing the hash in constructor of Binary objects  by traversing the array with for loop  -      The above implementation means that in the worst case you may have to traverse the arrays three times  first to compute hash of array1  then to compute hash of array2 and finally  because this is the worst case scenario  lengths and hashes equal  to compare bytes in array1 with bytes in array 2   Overall  even though System Data Linq Binary is built into BCL  I don t think it is the fastest way to compare two byte arrays  -

User · Answer

I did some measurements using attached program  net 4 7 release build without the debugger attached  I think people have been using the wrong metric since what you are about if you care about speed here is how long it takes to figure out if two byte arrays are equal  i e  throughput in bytes   StructuralComparison                4 6 MiB s for                               274 5 MiB s ToUInt32                          263 6 MiB s ToUInt64                          474 9 MiB s memcmp                           8500 8 MiB s   As you can see  there s no better way than memcmp and it s orders of magnitude faster  A simple for loop is the second best option  And it still boggles my mind why Microsoft cannot simply include a Buffer Compare method    Program cs    using System  using System Collections  using System Collections Generic  using System Diagnostics  using System Linq  using System Runtime InteropServices  using System Text  using System Threading Tasks   namespace memcmp       class Program               static byte   TestVector int size                        var data   new byte size               using  var rng   new System Security Cryptography RNGCryptoServiceProvider                                  rng GetBytes data                             return data                     static TimeSpan Measure string testCase  TimeSpan offset  Action action  bool ignore   false                        var t   Stopwatch StartNew                var n   0L              while  t Elapsed  lt  TimeSpan FromSeconds 10                                 action                    n                              var elapsed   t Elapsed - offset              if   ignore                                Console WriteLine    testCase -16     n   elapsed TotalSeconds 16 0 0  MiB s                              return elapsed                      DllImport  msvcrt dll   CallingConvention   CallingConvention Cdecl           static extern int memcmp byte   b1  byte   b2  long count            static void Main string   args                           how quickly can we establish if two sequences of bytes are equal                  note that we are testing the speed of different comparsion methods              var a   TestVector 1024   1024      1 MiB             var b    byte   a Clone                    was meant to offset the overhead of everything but copying but my attempt was a horrible mistake    should have reacted sooner due to the initially ridiculous throughput values                   Measure  offset   new TimeSpan         gt    return     ignore  true               var offset   TimeZone Zero              Measure  StructuralComparison   offset       gt                                StructuralComparisons StructuralEqualityComparer Equals a  b                                Measure  for   offset       gt                                for  int i   0  i  lt  a Length  i                                          if  a i     b i   break                                                 Measure  ToUInt32   offset       gt                                for  int i   0  i  lt  a Length  i    4                                        if  BitConverter ToUInt32 a  i     BitConverter ToUInt32 b  i   break                                                 Measure  ToUInt64   offset       gt                                for  int i   0  i  lt  a Length  i    8                                        if  BitConverter ToUInt64 a  i     BitConverter ToUInt64 b  i   break                                                 Measure  memcmp   offset       gt                                memcmp a  b  a Length

User · Answer

For comparing short byte arrays the following is an interesting hack   if myByteArray1 Length    myByteArray2 Length  return false  if myByteArray1 Length    8     return BitConverter ToInt64 myByteArray1  0     BitConverter ToInt64 myByteArray2  0    else if myByteArray Length    4     return BitConverter ToInt32 myByteArray2  0     BitConverter ToInt32 myByteArray2  0      Then I would probably fall out to the solution listed in the question   It d be interesting to do a performance analysis of this code

User · Answer

It seems that EqualBytesLongUnrolled is the best from the above suggested   Skipped methods  Enumerable SequenceEqual StructuralComparisons StructuralEqualityComparer Equals    were not-patient-for-slow  On 265MB arrays I have measured this   Host Process Environment Information  BenchmarkDotNet Core v0 9 9 0 OS Microsoft Windows NT 6 2 9200 0 Processor Intel R  Core TM  i7-3770 CPU 3 40GHz  ProcessorCount 8 Frequency 3323582 ticks  Resolution 300 8802 ns  Timer TSC CLR MS NET 4 0 30319 42000  Arch 64-bit RELEASE  RyuJIT  GC Concurrent Workstation JitModules clrjit-v4 6 1590 0  Type CompareMemoriesBenchmarks  Mode Throughput                     Method        Median      StdDev   Scaled   Scaled-SD   -----------------------  ------------  ----------  -------  ----------                NewMemCopy    30 0443 ms   1 1880 ms     1 00        0 00    EqualBytesLongUnrolled    29 9917 ms   0 7480 ms     0 99        0 04             msvcrt memcmp    30 0930 ms   0 2964 ms     1 00        0 03             UnsafeCompare    31 0520 ms   0 7072 ms     1 03        0 04          ByteArrayCompare   212 9980 ms   2 0776 ms     7 06        0 25       OS Windows Processor    ProcessorCount 8 Frequency 3323582 ticks  Resolution 300 8802 ns  Timer TSC CLR CORE  Arch 64-bit    RyuJIT  GC Concurrent Workstation dotnet cli version  1 0 0-preview2-003131  Type CompareMemoriesBenchmarks  Mode Throughput                     Method        Median      StdDev   Scaled   Scaled-SD   -----------------------  ------------  ----------  -------  ----------                NewMemCopy    30 1789 ms   0 0437 ms     1 00        0 00    EqualBytesLongUnrolled    30 1985 ms   0 1782 ms     1 00        0 01             msvcrt memcmp    30 1084 ms   0 0660 ms     1 00        0 00             UnsafeCompare    31 1845 ms   0 4051 ms     1 03        0 01          ByteArrayCompare   212 0213 ms   0 1694 ms     7 03        0 01

User · Answer

I developed a method that slightly beats memcmp    plinth s answer  and very slighly beats EqualBytesLongUnrolled    Arek Bulski s answer  on my PC  Basically  it unrolls the loop by 4 instead of 8   Update 30 Mar  2019   Starting in  NET core 3 0  we have SIMD support   This solution is fastest by a considerable margin on my PC    if NETCOREAPP3 0 using System Runtime Intrinsics X86   endif      public static unsafe bool Compare byte   arr0  byte   arr1        if  arr0    arr1                return true            if  arr0    null    arr1    null                return false            if  arr0 Length    arr1 Length                return false            if  arr0 Length    0                return true            fixed  byte  b0   arr0  b1   arr1         if NETCOREAPP3 0         if  Avx2 IsSupported                        return Compare256 b0  b1  arr0 Length                     else if  Sse2 IsSupported                        return Compare128 b0  b1  arr0 Length                     else  endif                       return Compare64 b0  b1  arr0 Length                      if NETCOREAPP3 0 public static unsafe bool Compare256 byte  b0  byte  b1  int length        byte  lastAddr   b0   length      byte  lastAddrMinus128   lastAddr - 128      const int mask   -1      while  b0  lt  lastAddrMinus128     unroll the loop so that we are comparing 128 bytes at a time                if  Avx2 MoveMask Avx2 CompareEqual Avx LoadVector256 b0   Avx LoadVector256 b1       mask                        return false                    if  Avx2 MoveMask Avx2 CompareEqual Avx LoadVector256 b0   32   Avx LoadVector256 b1   32       mask                        return false                    if  Avx2 MoveMask Avx2 CompareEqual Avx LoadVector256 b0   64   Avx LoadVector256 b1   64       mask                        return false                    if  Avx2 MoveMask Avx2 CompareEqual Avx LoadVector256 b0   96   Avx LoadVector256 b1   96       mask                        return false                    b0    128          b1    128            while  b0  lt  lastAddr                if   b0     b1  return false          b0            b1              return true    public static unsafe bool Compare128 byte  b0  byte  b1  int length        byte  lastAddr   b0   length      byte  lastAddrMinus64   lastAddr - 64      const int mask   0xFFFF      while  b0  lt  lastAddrMinus64     unroll the loop so that we are comparing 64 bytes at a time                if  Sse2 MoveMask Sse2 CompareEqual Sse2 LoadVector128 b0   Sse2 LoadVector128 b1       mask                        return false                    if  Sse2 MoveMask Sse2 CompareEqual Sse2 LoadVector128 b0   16   Sse2 LoadVector128 b1   16       mask                        return false                    if  Sse2 MoveMask Sse2 CompareEqual Sse2 LoadVector128 b0   32   Sse2 LoadVector128 b1   32       mask                        return false                    if  Sse2 MoveMask Sse2 CompareEqual Sse2 LoadVector128 b0   48   Sse2 LoadVector128 b1   48       mask                        return false                    b0    64          b1    64            while  b0  lt  lastAddr                if   b0     b1  return false          b0            b1              return true     endif public static unsafe bool Compare64 byte  b0  byte  b1  int length        byte  lastAddr   b0   length      byte  lastAddrMinus32   lastAddr - 32      while  b0  lt  lastAddrMinus32     unroll the loop so that we are comparing 32 bytes at a time                if    ulong  b0      ulong  b1  return false          if    ulong   b0   8       ulong   b1   8   return false          if    ulong   b0   16       ulong   b1   16   return false          if    ulong   b0   24       ulong   b1   24   return false          b0    32          b1    32            while  b0  lt  lastAddr                if   b0     b1  return false          b0            b1              return true

[c#] Comparing two byte arrays in .NET

Examples related to c#

Examples related to .net

Examples related to arrays

Examples related to performance

Examples related to j#