[protocol-buffers] How to define an optional field in protobuf 3

I need to specify a message with an optional field in protobuf (proto3 syntax). In terms of proto 2 syntax, the message I want to express is something like:

message Foo {
    required int32 bar = 1;
    optional int32 baz = 2;
}

From my understanding "optional" concept has been removed from syntax proto 3 (along with required concept). Though it is not clear the alternative - using the default value to state that a field has not been specified from the sender, leaves an ambiguity if the default value belongs to the valid values domain (consider for example a boolean type).

So, how am I supposed to encode the message above? Thank you.

Another way to encode the message you intend is to add another field to track "set" fields:

syntax="proto3";

package qtprotobuf.examples;

message SparseMessage {
    repeated uint32 fieldsUsed = 1;
    bool   attendedParty = 2;
    uint32 numberOfKids  = 3;
    string nickName      = 4;
}

message ExplicitMessage {
    enum PARTY_STATUS {ATTENDED=0; DIDNT_ATTEND=1; DIDNT_ASK=2;};
    PARTY_STATUS attendedParty = 1;
    bool   indicatedKids = 2;
    uint32 numberOfKids  = 3;
    enum NO_NICK_STATUS {HAS_NO_NICKNAME=0; WOULD_NOT_ADMIT_TO_HAVING_HAD_NICKNAME=1;};
    NO_NICK_STATUS noNickStatus = 4;
    string nickName      = 5;
}

This is especially appropriate if there is a large number of fields and only a small number of them have been assigned.

In python, usage would look like this:

import field_enum_example_pb2
m = field_enum_example_pb2.SparseMessage()
m.attendedParty = True
m.fieldsUsed.append(field_enum_example_pb2.SparseMessages.ATTENDEDPARTY_FIELD_NUMBER)

Another way is that you can use bitmask for each optional field. and set those bits if values are set and reset those bits which values are not set

enum bitsV {
    baz_present = 1; // 0x01
    baz1_present = 2; // 0x02

}
message Foo {
    uint32 bitMask;
    required int32 bar = 1;
    optional int32 baz = 2;
    optional int32 baz1 = 3;
}

On parsing check for value of bitMask.

if (bitMask & baz_present)
    baz is present

if (bitMask & baz1_present)
    baz1 is present

There is a good post about this: https://itnext.io/protobuf-and-null-support-1908a15311b6

The solution depends on your actual use case:

• Handling partial update

• Support null

Based on Kenton's answer, a simpler yet working solution looks like:

message Foo {
    oneof optional_baz { // "optional_" prefix here just serves as an indicator, not keyword in proto2
        int32 baz = 1;
    }
}

you can find if one has been initialized by comparing the references with the default instance:

GRPCContainer container = myGrpcResponseBean.getContainer();
if (container.getDefaultInstanceForType() != container) {
...
}

To expand on @cybersnoopy 's suggestion here

if you had a .proto file with a message like so:

message Request {
    oneof option {
        int64 option_value = 1;
    }
}

You can make use of the case options provided (java generated code):

So we can now write some code as follows:

Request.OptionCase optionCase = request.getOptionCase();
OptionCase optionNotSet = OPTION_NOT_SET;

if (optionNotSet.equals(optionCase)){
    // value not set
} else {
    // value set
}

One way is to optional like described in the accepted answer: https://stackoverflow.com/a/62566052/1803821

Another one is to use wrapper objects. You don't need to write them yourself as google already provides them:

At the top of your .proto file add this import:

import "google/protobuf/wrappers.proto";

Now you can use special wrappers for every simple type:

DoubleValue
FloatValue
Int64Value
UInt64Value
Int32Value
UInt32Value
BoolValue
StringValue
BytesValue

So to answer the original question a usage of such a wrapper could be like this:

message Foo {
    int32 bar = 1;
    google.protobuf.Int32Value baz = 2;
}

Now for example in Java I can do stuff like:

if(foo.hasBaz()) { ... }

In proto3, all fields are "optional" (in that it is not an error if the sender fails to set them). But, fields are no longer "nullable", in that there's no way to tell the difference between a field being explicitly set to its default value vs. not having been set at all.

If you need a "null" state (and there is no out-of-range value that you can use for this) then you will instead need to encode this as a separate field. For instance, you could do:

message Foo {
  bool has_baz = 1;  // always set this to "true" when using baz
  int32 baz = 2;
}

Alternatively, you could use oneof:

message Foo {
  oneof baz {
    bool baz_null = 1;  // always set this to "true" when null
    int32 baz_value = 2;
  }
}

The oneof version is more explicit and more efficient on the wire but requires understanding how oneof values work.

Finally, another perfectly reasonable option is to stick with proto2. Proto2 is not deprecated, and in fact many projects (including inside Google) very much depend on proto2 features which are removed in proto3, hence they will likely never switch. So, it's safe to keep using it for the foreseeable future.