How to define an optional field in protobuf 3

Question

I need to specify a message with an optional field in protobuf  proto3 syntax   In terms of proto 2 syntax  the message I want to express is something like   message Foo       required int32 bar   1      optional int32 baz   2      From my understanding  optional  concept has been removed from syntax proto 3  along with required concept   Though it is not clear the alternative - using the default value to state that a field has not been specified from the sender  leaves an ambiguity if the default value belongs to the valid values domain  consider for example a boolean type    So  how am I supposed to encode the message above  Thank you

User · Answer

There is a good post about this  https   itnext io protobuf-and-null-support-1908a15311b6  The solution depends on your actual use case    Handling partial update Support null

User · Answer

In proto3  all fields are  optional   in that it is not an error if the sender fails to set them   But  fields are no longer  nullable   in that there s no way to tell the difference between a field being explicitly set to its default value vs  not having been set at all   If you need a  null  state  and there is no out-of-range value that you can use for this  then you will instead need to encode this as a separate field  For instance  you could do   message Foo     bool has baz   1      always set this to  true  when using baz   int32 baz   2      Alternatively  you could use oneof   message Foo     oneof baz       bool baz null   1      always set this to  true  when null     int32 baz value   2          The oneof version is more explicit and more efficient on the wire but requires understanding how oneof values work   Finally  another perfectly reasonable option is to stick with proto2  Proto2 is not deprecated  and in fact many projects  including inside Google  very much depend on proto2 features which are removed in proto3  hence they will likely never switch  So  it s safe to keep using it for the foreseeable future

User · Answer

Another way is that you can use bitmask for each optional field  and set those bits if values are set and reset those bits which values are not set  enum bitsV       baz present   1     0x01     baz1 present   2     0x02    message Foo       uint32 bitMask      required int32 bar   1      optional int32 baz   2      optional int32 baz1   3      On parsing check for value of bitMask   if  bitMask  amp  baz present      baz is present  if  bitMask  amp  baz1 present      baz1 is present

User · Answer

To expand on  cybersnoopy  s suggestion here  if you had a  proto file with a message like so   message Request       oneof option           int64 option value   1            You can make use of the case options provided  java generated code    So we can now write some code as follows   Request OptionCase optionCase   request getOptionCase    OptionCase optionNotSet   OPTION NOT SET   if  optionNotSet equals optionCase           value not set   else          value set

User · Answer

Based on Kenton s answer  a simpler yet working solution looks like   message Foo       oneof optional baz       optional   prefix here just serves as an indicator  not keyword in proto2         int32 baz   1

User · Answer

Since protobuf release 3 15  proto3 supports using the optional keyword  just as in proto2  to give a scalar field presence information  syntax    quot proto3 quot    message Foo       int32 bar   1      optional int32 baz   2     A has baz   hasBaz   method is generated for the optional field above  just as it was in proto2  Under the hood  protoc effectively treats an optional field as if it were declared using a oneof wrapper  as CyberSnoopy   s answer suggested  message Foo       int32 bar   1      oneof optional baz           int32 baz   2           If you   ve already used that approach  you can now simplify your message declarations  switch from oneof to optional  and code  since the wire format is the same  The nitty-gritty details about field presence and optional in proto3 can be found in the Application note  Field presence doc  Historical note  Experimental support for optional in proto3 was first announced on Apr 23  2020 in this comment  Using it required passing protoc the --experimental allow proto3 optional flag in releases 3 12-3 14

User · Answer

Another way to encode the message you intend is to add another field to track  quot set quot  fields  syntax  quot proto3 quot    package qtprotobuf examples   message SparseMessage       repeated uint32 fieldsUsed   1      bool   attendedParty   2      uint32 numberOfKids    3      string nickName        4     message ExplicitMessage       enum PARTY STATUS  ATTENDED 0  DIDNT ATTEND 1  DIDNT ASK 2        PARTY STATUS attendedParty   1      bool   indicatedKids   2      uint32 numberOfKids    3      enum NO NICK STATUS  HAS NO NICKNAME 0  WOULD NOT ADMIT TO HAVING HAD NICKNAME 1        NO NICK STATUS noNickStatus   4      string nickName        5     This is especially appropriate if there is a large number of fields and only a small number of them have been assigned  In python  usage would look like this  import field enum example pb2 m   field enum example pb2 SparseMessage   m attendedParty   True m fieldsUsed append field enum example pb2 SparseMessages ATTENDEDPARTY FIELD NUMBER

User · Answer

One way is to optional like described in the accepted answer  https   stackoverflow com a 62566052 1803821 Another one is to use wrapper objects  You don t need to write them yourself as google already provides them  At the top of your  proto file add this import  import  quot google protobuf wrappers proto quot   Now you can use special wrappers for every simple type  DoubleValue FloatValue Int64Value UInt64Value Int32Value UInt32Value BoolValue StringValue BytesValue  So to answer the original question a usage of such a wrapper could be like this  message Foo       int32 bar   1      google protobuf Int32Value baz   2     Now for example in Java I can do stuff like  if foo hasBaz

User · Answer

you can find if one has been initialized by comparing the references with the default instance   GRPCContainer container   myGrpcResponseBean getContainer    if  container getDefaultInstanceForType      container

[protocol-buffers] How to define an optional field in protobuf 3

Examples related to protocol-buffers

Examples related to protocol-buffers-3