Java JSON - Protobuf back conversion

Question

I have an existing system  which is using protobuf-based communication protocol between GUI and server  Now I would like to add some persistence  but at the moment protobuf messages are straight converted to a third-party custom objects   Is there a way to convert proto messages to json  which could be then persisted to database   N B   I don t much like an idea of writing binary protobuf to database  because it can one day become not backward-compatible with newer versions and break the system that way

User · Answer

Adding to Ophirs answer  JsonFormat is available even before protobuf 3 0  However  the way to do it differs a little bit  In Protobuf 3 0   the JsonFormat class is a singleton and therefore do something like the below String jsonString    quot  quot   JsonFormat parser   ignoringUnknownFields   merge jsonString yourObjectBuilder    In Protobuf 2 5   the below should work String jsonString    quot  quot   JsonFormat jsonFormat   new JsonFormat    jsonString   jsonFormat printToString yourProtobufMessage    Here is a link to a tutorial I wrote that uses the JsonFormat class in a TypeAdapter that can be registered to a GsonBuilder object  You can then use Gson s toJson and fromJson methods to convert the proto data to Java and back  Replying to jean   If we have the protobuf data in a file and want to parse it into a protobuf message object  use the merge method TextFormat class  See the below snippet     Let your proto text data be in a file MessageDataAsProto prototxt    Read it into string   String protoDataAsString   FileUtils readFileToString new File  quot MessageDataAsProto prototxt quot         Create an object of the message builder MyMessage Builder myMsgBuilder   MyMessage newBuilder        Use text format to parse the data into the message builder TextFormat merge protoDataAsString  ExtensionRegistry getEmptyRegistry    myMsgBuilder       Build the message and return return myMsgBuilder build

User · Answer

I don t much like an idea of writing binary protobuf to database  because it can one day become not backward-compatible with newer versions and break the system that way    Converting protobuf to JSON for storage and then back to protobuf on load is much more likely to create compatibility problems  because    If the process which performs the conversion is not built with the latest version of the protobuf schema  then converting will silently drop any fields that the process doesn t know about  This is true both of the storing and loading ends  Even with the most recent schema available  JSON  lt -  Protobuf conversion may be lossy in the presence of imprecise floating-point values and similar corner cases  Protobufs actually have  slightly  stronger backwards-compatibility guarantees than JSON  Like with JSON  if you add a new field  old clients will ignore it  Unlike with JSON  Protobufs allow declaring a default value  which can make it somewhat easier for new clients to deal with old data that is otherwise missing the field  This is only a slight advantage  but otherwise Protobuf and JSON have equivalent backwards-compatibility properties  therefore you are not gaining any backwards-compatibility advantages from storing in JSON    With all that said  there are many libraries out there for converting protobufs to JSON  usually built on the Protobuf reflection interface  not to be confused with the Java reflection interface  Protobuf reflection is offered by the com google protobuf Message interface

User · Answer

Well  there is no shortcut to do it as per my findings  but somehow you an achieve it in few simple steps    First you have to declare a bean of type  ProtobufJsonFormatHttpMessageConverter      Bean    Primary   public ProtobufJsonFormatHttpMessageConverter protobufHttpMessageConverter         return new ProtobufJsonFormatHttpMessageConverter JsonFormat parser    JsonFormat printer             Then you can just write an Utility class like ResponseBuilder  because   it can parse the request by default but without these changes it can not   produce Json response  and then you can write few methods to convert the response types to its   related object type   public static  lt T gt  T build Message message  Class lt T gt  type      Printer printer   JsonFormat printer      Gson gson   new Gson      try       return gson fromJson printer print message   type       catch  JsonSyntaxException   InvalidProtocolBufferException e        throw new ApiException HttpStatus INTERNAL SERVER ERROR   Response   conversion Error   e           Then you can call this method from your controller class as last line like   -  return ResponseBuilder build  lt returned service object gt    lt Type gt      Hope this will help you to implement protobuf in json format

User · Answer

Here is my utility class  you may use  package  lt removed gt   import com google protobuf Message  import com google protobuf MessageOrBuilder  import com google protobuf util JsonFormat         Author  espresso stackoverflow     Sample use          Model Person reqObj   ProtoUtil toProto reqJson  Model Person getDefaultInstance             Model Person res   personSvc update reqObj           final String resJson   ProtoUtil toJson res        public class ProtoUtil       public static  lt T extends Message gt  String toJson T obj           try              return JsonFormat printer   print obj            catch Exception e               throw new RuntimeException  quot Error converting Proto to json quot   e                      public static  lt T extends MessageOrBuilder gt  T toProto String protoJsonStr  T message           try              Message Builder builder   message getDefaultInstanceForType   toBuilder                JsonFormat parser   ignoringUnknownFields   merge protoJsonStr builder               T out    T  builder build                return out           catch Exception e               throw new RuntimeException   quot Error converting Json to proto quot   e

User · Answer

As mentioned in an answer to a similar question  since v3 1 0 this is a supported feature of ProtocolBuffers  For Java  include the extension module com google protobuf protobuf-java-util and use JsonFormat like so   JsonFormat parser   ignoringUnknownFields   merge json  yourObjectBuilder   YourObject value   yourObjectBuilder build

User · Answer

We are currently using protobuf-java-format to convert our Protobuf messages  anything subclass of Message  into a JSON format to send over our web API   Simply do     JsonFormat printToString protoMessage

User · Answer

Try JsonFormat printer   print MessageOrBuilder   it looks good for proto3  Yet  it is unclear how to convert the actual protobuf message  which is provided as the java package of my choice defined in the  proto file   to a com google protbuf Message object

User · Answer

Generics Solution  Here s a generic version of Json converter  package com github platform util   import java io IOException  import java lang reflect InvocationTargetException  import com google protobuf AbstractMessage Builder  import com google protobuf Message  import com google protobuf MessageOrBuilder  import com google protobuf util JsonFormat          Generic ProtoJsonUtil to be used to serialize and deserialize Proto to json         author Marcello deeSales gmail com        public final class ProtoJsonUtil               Makes a Json from a given message or builder             param messageOrBuilder is the instance       return The string representation       throws IOException if any error occurs         public static String toJson MessageOrBuilder messageOrBuilder  throws IOException       return JsonFormat printer   print messageOrBuilder                   Makes a new instance of message based on the json and the class       param  lt T gt  is the class type       param json is the json instance       param clazz is the class instance       return An instance of T based on the json values       throws IOException if any error occurs          SuppressWarnings   unchecked    rawtypes      public static  lt T extends Message gt  T fromJson String json  Class lt T gt  clazz  throws IOException          https   stackoverflow com questions 27642021 calling-parsefrom-method-for-generic-protobuffer-class-in-java 33701202 33701202     Builder builder   null      try            Since we are dealing with a Message type  we can call newBuilder         builder    Builder  clazz getMethod  newBuilder   invoke null          catch  IllegalAccessException   IllegalArgumentException   InvocationTargetException           NoSuchMethodException   SecurityException e          return null                The instance is placed into the builder values     JsonFormat parser   ignoringUnknownFields   merge json  builder           the instance will be from the build     return  T  builder build             Using it is as simple as follows   Message instance  GetAllGreetings Builder allGreetingsBuilder   GetAllGreetings newBuilder     allGreetingsBuilder addGreeting makeNewGreeting  Marcello    Hi  s  how are you   Language EN            addGreeting makeNewGreeting  John    Today is hot   s  get some ice   Language ES            addGreeting makeNewGreeting  Mary     s  summer is here  Let s go surfing    Language PT     GetAllGreetings allGreetings   allGreetingsBuilder build      To Json Generic  String json   ProtoJsonUtil toJson allGreetingsLoaded   log info  Json format      json     From Json Generic  GetAllGreetings parsed   ProtoJsonUtil fromJson json  GetAllGreetings class   log info  The Proto deserialized from Json     parsed

User · Answer

For protobuf 2 5  use the dependency    com googlecode protobuf-java-format     protobuf-java-format     1 2    Then use the code   com googlecode protobuf format JsonFormat merge json  builder  com googlecode protobuf format JsonFormat printToString proto

[java] Java: JSON -> Protobuf & back conversion

Examples related to java

Examples related to json

Examples related to serialization

Examples related to persistence

Examples related to protocol-buffers