Fastest way to serialize and deserialize NET objects

Question

I m looking for the fastest way to serialize and deserialize  NET objects  Here is what I have so far   public class TD       public List lt CT gt  CTs   get  set        public List lt TE gt  TEs   get  set        public string Code   get  set        public string Message   get  set        public DateTime StartDate   get  set        public DateTime EndDate   get  set         public static string Serialize List lt TD gt  tData                var serializer   new XmlSerializer typeof List lt TD gt              TextWriter writer   new StringWriter            serializer Serialize writer  tData            return writer ToString               public static List lt TD gt  Deserialize string tData                var serializer   new XmlSerializer typeof List lt TD gt              TextReader reader   new StringReader tData            return  List lt TD gt  serializer Deserialize reader

User · Answer

Yet another serializer out there that claims to be super fast is netserializer.

The data given on their site shows performance of 2x - 4x over protobuf, I have not tried this myself, but if you are evaluating various options, try this as well

User · Answer

I removed the bugs in above code and got below results   Also I am unsure given how NetSerializer requires you to register the types you are serializing  what kind of compatibility or performance differences that could potentially make   Generating 100000 arrays of data    Test data generated  Testing BinarySerializer    BinaryFormatter  Serializing took 508 9773ms  BinaryFormatter  Deserializing took 371 8499ms   Testing ProtoBuf serializer    ProtoBuf  Serializing took 3280 9185ms  ProtoBuf  Deserializing took 3190 7899ms   Testing NetSerializer serializer    NetSerializer  Serializing took 427 1241ms  NetSerializer  Deserializing took 78 954ms  Press any key to end    Modified Code  using System  using System Collections Generic  using System Diagnostics  using System IO  using System Linq  using System Runtime Serialization Formatters Binary  using System Text  using System Threading Tasks   namespace SerializationTests       class Program               static void Main string   args                        var count   100000              var rnd   new Random  int DateTime UtcNow Ticks  amp  0xFF               Console WriteLine  Generating  0  arrays of data      count               var arrays   new List lt int   gt                 for  int i   0  i  lt  count  i                                  var elements   rnd Next 1  100                   var array   new int elements                   for  int j   0  j  lt  elements  j                                          array j    rnd Next                                      arrays Add array                             Console WriteLine  Test data generated                 var stopWatch   new Stopwatch                 Console WriteLine  Testing BinarySerializer                   var binarySerializer   new BinarySerializer                var binarySerialized   new List lt byte   gt                 var binaryDeserialized   new List lt int   gt                  stopWatch Reset                stopWatch Start                foreach  var array in arrays                                binarySerialized Add binarySerializer Serialize array                              stopWatch Stop                Console WriteLine  BinaryFormatter  Serializing took  0 ms    stopWatch Elapsed TotalMilliseconds                stopWatch Reset                stopWatch Start                foreach  var serialized in binarySerialized                                binaryDeserialized Add binarySerializer Deserialize lt int   gt  serialized                              stopWatch Stop                Console WriteLine  BinaryFormatter  Deserializing took  0 ms    stopWatch Elapsed TotalMilliseconds                 Console WriteLine                Console WriteLine  Testing ProtoBuf serializer                   var protobufSerializer   new ProtoBufSerializer                var protobufSerialized   new List lt byte   gt                 var protobufDeserialized   new List lt int   gt                  stopWatch Reset                stopWatch Start                foreach  var array in arrays                                protobufSerialized Add protobufSerializer Serialize array                              stopWatch Stop                Console WriteLine  ProtoBuf  Serializing took  0 ms    stopWatch Elapsed TotalMilliseconds                stopWatch Reset                stopWatch Start                foreach  var serialized in protobufSerialized                                protobufDeserialized Add protobufSerializer Deserialize lt int   gt  serialized                              stopWatch Stop                Console WriteLine  ProtoBuf  Deserializing took  0 ms    stopWatch Elapsed TotalMilliseconds                Console WriteLine                Console WriteLine  Testing NetSerializer serializer                   var netSerializerSerialized   new List lt byte   gt                 var netSerializerDeserialized   new List lt int   gt                  stopWatch Reset                stopWatch Start                var netSerializerSerializer   new NS                foreach  var array in arrays                                netSerializerSerialized Add netSerializerSerializer Serialize array                              stopWatch Stop                Console WriteLine  NetSerializer  Serializing took  0 ms    stopWatch Elapsed TotalMilliseconds                stopWatch Reset                stopWatch Start                foreach  var serialized in netSerializerSerialized                                netSerializerDeserialized Add netSerializerSerializer Deserialize lt int   gt  serialized                              stopWatch Stop                Console WriteLine  NetSerializer  Deserializing took  0 ms    stopWatch Elapsed TotalMilliseconds                Console WriteLine  Press any key to end                 Console ReadKey                       public class BinarySerializer                       private static readonly BinaryFormatter Formatter   new BinaryFormatter                 public byte   Serialize object toSerialize                                using  var stream   new MemoryStream                                          Formatter Serialize stream  toSerialize                       return stream ToArray                                                 public T Deserialize lt T gt  byte   serialized                                using  var stream   new MemoryStream serialized                                         var result    T Formatter Deserialize stream                       return result                                                     public class ProtoBufSerializer                       public byte   Serialize object toSerialize                                using  var stream   new MemoryStream                                          ProtoBuf Serializer Serialize stream  toSerialize                       return stream ToArray                                                 public T Deserialize lt T gt  byte   serialized                                using  var stream   new MemoryStream serialized                                         var result   ProtoBuf Serializer Deserialize lt T gt  stream                       return result                                                     public class NS                       NetSerializer Serializer Serializer   new NetSerializer Serializer new Type     typeof int   typeof int                     public byte   Serialize object toSerialize                                using  var stream   new MemoryStream                                          Serializer Serialize stream  toSerialize                       return stream ToArray                                                 public T Deserialize lt T gt  byte   serialized                                using  var stream   new MemoryStream serialized                                         Serializer Deserialize stream  out var result                       return  T result

User · Answer

Here s your model  with invented CT and TE  using protobuf-net  yet retaining the ability to use XmlSerializer  which can be useful - in particular for migration   I humbly submit  with lots of evidence if you need it  that this is the fastest  or certainly one of the fastest  general purpose serializer in  NET   If you need strings  just base-64 encode the binary    XmlType  public class CT        XmlElement Order   1       public int Foo   get  set       XmlType  public class TE        XmlElement Order   1       public int Bar   get  set       XmlType  public class TD        XmlElement Order 1       public List lt CT gt  CTs   get  set         XmlElement Order 2       public List lt TE gt  TEs   get  set         XmlElement Order   3       public string Code   get  set         XmlElement Order   4       public string Message   get  set         XmlElement Order   5       public DateTime StartDate   get  set         XmlElement Order   6       public DateTime EndDate   get  set         public static byte   Serialize List lt TD gt  tData            using  var ms   new MemoryStream                  ProtoBuf Serializer Serialize ms  tData               return ms ToArray                                     public static List lt TD gt  Deserialize byte   tData            using  var ms   new MemoryStream tData                 return ProtoBuf Serializer Deserialize lt List lt TD gt  gt  ms

User · Answer

Having an interest in this  I decided to test the suggested methods with the closest  apples to apples  test I could   I wrote a Console app  with the following code   using System  using System Collections Generic  using System Diagnostics  using System IO  using System Linq  using System Runtime Serialization Formatters Binary  using System Text  using System Threading Tasks   namespace SerializationTests       class Program               static void Main string   args                        var count   100000              var rnd   new Random DateTime UtcNow GetHashCode                 Console WriteLine  Generating  0  arrays of data      count               var arrays   new List lt int   gt                 for  int i   0  i  lt  count  i                                  var elements   rnd Next 1  100                   var array   new int elements                   for  int j   0  j  lt  elements  j                                          array j    rnd Next                                         arrays Add array                             Console WriteLine  Test data generated                 var stopWatch   new Stopwatch                 Console WriteLine  Testing BinarySerializer                   var binarySerializer   new BinarySerializer                var binarySerialized   new List lt byte   gt                 var binaryDeserialized   new List lt int   gt                  stopWatch Reset                stopWatch Start                foreach  var array in arrays                                binarySerialized Add binarySerializer Serialize array                              stopWatch Stop                Console WriteLine  BinaryFormatter  Serializing took  0 ms    stopWatch Elapsed TotalMilliseconds                stopWatch Reset                stopWatch Start                foreach  var serialized in binarySerialized                                binaryDeserialized Add binarySerializer Deserialize lt int   gt  serialized                              stopWatch Stop                Console WriteLine  BinaryFormatter  Deserializing took  0 ms    stopWatch Elapsed TotalMilliseconds                 Console WriteLine                Console WriteLine  Testing ProtoBuf serializer                   var protobufSerializer   new ProtoBufSerializer                var protobufSerialized   new List lt byte   gt                 var protobufDeserialized   new List lt int   gt                  stopWatch Reset                stopWatch Start                foreach  var array in arrays                                protobufSerialized Add protobufSerializer Serialize array                              stopWatch Stop                Console WriteLine  ProtoBuf  Serializing took  0 ms    stopWatch Elapsed TotalMilliseconds                stopWatch Reset                stopWatch Start                foreach  var serialized in protobufSerialized                                protobufDeserialized Add protobufSerializer Deserialize lt int   gt  serialized                              stopWatch Stop                Console WriteLine  ProtoBuf  Deserializing took  0 ms    stopWatch Elapsed TotalMilliseconds                Console WriteLine                Console WriteLine  Testing NetSerializer serializer                   var netSerializerSerializer   new ProtoBufSerializer                var netSerializerSerialized   new List lt byte   gt                 var netSerializerDeserialized   new List lt int   gt                  stopWatch Reset                stopWatch Start                foreach  var array in arrays                                netSerializerSerialized Add netSerializerSerializer Serialize array                              stopWatch Stop                Console WriteLine  NetSerializer  Serializing took  0 ms    stopWatch Elapsed TotalMilliseconds                stopWatch Reset                stopWatch Start                foreach  var serialized in netSerializerSerialized                                netSerializerDeserialized Add netSerializerSerializer Deserialize lt int   gt  serialized                              stopWatch Stop                Console WriteLine  NetSerializer  Deserializing took  0 ms    stopWatch Elapsed TotalMilliseconds                Console WriteLine  Press any key to end                 Console ReadKey                       public class BinarySerializer                       private static readonly BinaryFormatter Formatter   new BinaryFormatter                 public byte   Serialize object toSerialize                                using  var stream   new MemoryStream                                          Formatter Serialize stream  toSerialize                       return stream ToArray                                                 public T Deserialize lt T gt  byte   serialized                                using  var stream   new MemoryStream serialized                                         var result    T Formatter Deserialize stream                       return result                                                     public class ProtoBufSerializer                       public byte   Serialize object toSerialize                                using  var stream   new MemoryStream                                          ProtoBuf Serializer Serialize stream  toSerialize                       return stream ToArray                                                 public T Deserialize lt T gt  byte   serialized                                using  var stream   new MemoryStream serialized                                         var result   ProtoBuf Serializer Deserialize lt T gt  stream                       return result                                                     public class NetSerializer                       private static readonly NetSerializer Serializer   new NetSerializer                public byte   Serialize object toSerialize                                return Serializer Serialize toSerialize                              public T Deserialize lt T gt  byte   serialized                                return Serializer Deserialize lt T gt  serialized                                     The results surprised me  they were consistent when run multiple times   Generating 100000 arrays of data    Test data generated  Testing BinarySerializer    BinaryFormatter  Serializing took 336 8392ms  BinaryFormatter  Deserializing took 208 7527ms   Testing ProtoBuf serializer    ProtoBuf  Serializing took 2284 3827ms  ProtoBuf  Deserializing took 2201 8072ms   Testing NetSerializer serializer    NetSerializer  Serializing took 2139 5424ms  NetSerializer  Deserializing took 2113 7296ms  Press any key to end    Collecting these results  I decided to see if ProtoBuf or NetSerializer performed better with larger objects   I changed the collection count to 10 000 objects  but increased the size of the arrays to 1-10 000 instead of 1-100   The results seemed even more definitive   Generating 10000 arrays of data    Test data generated  Testing BinarySerializer    BinaryFormatter  Serializing took 285 8356ms  BinaryFormatter  Deserializing took 206 0906ms   Testing ProtoBuf serializer    ProtoBuf  Serializing took 10693 3848ms  ProtoBuf  Deserializing took 5988 5993ms   Testing NetSerializer serializer    NetSerializer  Serializing took 9017 5785ms  NetSerializer  Deserializing took 5978 7203ms  Press any key to end    My conclusion  therefore  is  there may be cases where ProtoBuf and NetSerializer are well-suited to  but in terms of raw performance for at least relatively simple objects    BinaryFormatter is significantly more performant  by at least an order of magnitude   YMMV

User · Answer

I took the liberty of feeding your classes into the CGbR generator  Because it is in an early stage it doesn t support DateTime yet  so I simply replaced it with long  The generated serialization code looks like this   public int Size       get                 var size   24             Add size for collections and strings         size    Cts    null   0   Cts Count   4          size    Tes    null   0   Tes Count   4          size    Code    null   0   Code Length          size    Message    null   0   Message Length           return size                         public byte   ToBytes byte   bytes  ref int index        if  index   Size  gt  bytes Length          throw new ArgumentOutOfRangeException  index    Object does not fit in array            Convert Cts        Two bytes length information for each dimension     GeneratorByteConverter Include  ushort  Cts    null   0   Cts Count   bytes  ref index       if  Cts    null                for var i   0  i  lt  Cts Count  i                          var value   Cts i               value ToBytes bytes  ref index                          Convert Tes        Two bytes length information for each dimension     GeneratorByteConverter Include  ushort  Tes    null   0   Tes Count   bytes  ref index       if  Tes    null                for var i   0  i  lt  Tes Count  i                          var value   Tes i               value ToBytes bytes  ref index                          Convert Code     GeneratorByteConverter Include Code  bytes  ref index          Convert Message     GeneratorByteConverter Include Message  bytes  ref index          Convert StartDate     GeneratorByteConverter Include StartDate ToBinary    bytes  ref index          Convert EndDate     GeneratorByteConverter Include EndDate ToBinary    bytes  ref index       return bytes     public Td FromBytes byte   bytes  ref int index           Read Cts     var ctsLength   GeneratorByteConverter ToUInt16 bytes  ref index       var tempCts   new List lt Ct gt  ctsLength       for  var i   0  i  lt  ctsLength  i                  var value   new Ct   FromBytes bytes  ref index           tempCts Add value             Cts   tempCts         Read Tes     var tesLength   GeneratorByteConverter ToUInt16 bytes  ref index       var tempTes   new List lt Te gt  tesLength       for  var i   0  i  lt  tesLength  i                  var value   new Te   FromBytes bytes  ref index           tempTes Add value             Tes   tempTes         Read Code     Code   GeneratorByteConverter GetString bytes  ref index          Read Message     Message   GeneratorByteConverter GetString bytes  ref index          Read StartDate     StartDate   DateTime FromBinary GeneratorByteConverter ToInt64 bytes  ref index           Read EndDate     EndDate   DateTime FromBinary GeneratorByteConverter ToInt64 bytes  ref index         return this      I created a list of sample objects like this   var objects   new List lt Td gt     for  int i   0  i  lt  1000  i          var obj   new Td               Message    Hello my friend           Code    Some code that can be put here           StartDate   DateTime Now AddDays -7           EndDate   DateTime Now AddDays 2           Cts   new List lt Ct gt             Tes   new List lt Te gt               for  int j   0  j  lt  10  j                  obj Cts Add new Ct   Foo   i   j             obj Tes Add new Te   Bar   i   j               objects Add obj       Results on my machine in Release build   var watch   new Stopwatch    watch Start    var bytes   BinarySerializer SerializeMany objects   watch Stop      Size  149000 bytes  Time  2 059ms 3 13ms  Edit  Starting with CGbR 0 4 3 the binary serializer supports DateTime  Unfortunately the DateTime ToBinary method is insanely slow  I will replace it with somehting faster soon   Edit2  When using UTC DateTime by invoking ToUniversalTime   the performance is restored and clocks in at 1 669ms

User · Answer

You can try Salar Bois serializer which has a decent performance  Its focus is on payload size but it also offers good performance   There are benchmarks in the Github page if you wish to see and compare the results by yourself   https   github com salarcode Bois

User · Answer

Protobuf is very very fast   See http   code google com p protobuf-net wiki Performance for in depth information concerning the performance of this system  and an implementation

User · Answer

A comprehansive comparison between diffreent formats made by me in this post- https   maxondev com serialization-performance-comparison-c-net-formats-frameworks-xmldatacontractserializer-xmlserializer-binaryformatter-json-newtonsoft-servicestack-text   Just one sample from the post-

User · Answer

The binary serializer included with  net should be faster that the XmlSerializer  Or another serializer for protobuf  json       But for some of them you need to add Attributes  or some other way to add metadata  For example ProtoBuf uses numeric property IDs internally  and the mapping needs to be somehow conserved by a different mechanism  Versioning isn t trivial with any serializer

[c#] Fastest way to serialize and deserialize .NET objects

Examples related to c#

Examples related to .net

Examples related to serialization

Examples related to deserialization