Introduction
This post utilizes Protocol Buffers and GRPC in the context of creating a key value store in Go. Protocol Buffers are a great way to serialize message data for transmission and GRPC builds on Protocol Buffers to describe services and how the messages passed between them. A simplistic put request for a key value store could be described with the following protocol buffer code:
|
|
The PutRequest
has two fields, key
and a value
, which are assigned numbers for encoding the Protocol Buffer wire format. Generating Go code will create a struct looking like this:
|
|
The client code can easily access the Value
field. However, This PutRequest
isn’t ideal in the context of the key value store because it only stores strings. Procol Buffers provide a field type called Any
to allow a field to represent multiple types.
Using protobuf.Any
Using the protobuf.Any
type requires changing the protocol buffer file:
- Setting the syntax to
proto3
- Including
any.proto
- Changing
value
field type togoogle.protobuf.Any
So now the protocol buffer file looks like this:
|
|
That was easy. However, getting the value from the generated code is more complicated. The Value field now expects a pointer to a anypb.Any
type.
|
|
And the anypb.Any
struct is defined like this:
|
|
The anypb.Any
struct contains two fields. The TypeUrl
field is a string containing the message type in the form of a URL: type.googleapis.com/google.protobuf.StringValue
. The Value
field is a slice of bytes. So the TypeUrl
field indicates how to interpret the slice of bytes in Value
.
Working with the anypb.Any
type requires two Go packages to be installed: anypb and wrapperspb. The anypb
package defines the anypb.Any
struct and the methods for marshalling values through it. The wrapperspb
package provides structs for wrapping simple scalar types.
Marshalling Values
Marshalling an interface{}
value through an anypb.Any
field requires 3 steps:
- A type switch to determine which wrapper to use (line 3)
- Creating the wrapper message (line 5)
- Calling
anypb.New()
on the wrapper message (line 10)
|
|
Unmarshalling Values
Unmarshalling is also accomplished in 3 steps:
- Calling UnmarshalNew() method on the value field (line 2)
- Determine the type of the field in a type switch (line 5)
- Calling UnmarshalTo() method (line 9)
|
|
Handling Array Types
The wrapperspb
package doesn’t provide any way to wrap complex or array types, but it’s straight forward to support. First, define a message with a repeated type:
message StringArray {
repeated string value = 1;
}
Next, add a case to the type switch:
|
|
Conclusion
Protocol Buffers provide a field type google.protobuf.Any
which is one way model a field with multiple types. Using an Any
type requires a bit more work to get values in and out of a message.
- Marshaling and unmarshal requires a wrapper messages.
- Primitive types can use messages from wrapperspb
- Complex or array types need wrapped in custom messages
- Use anypb.New() to create
anypb.Any
values - Use anypb.Any.UnmarshalNew() and anypb.Any.UnmarshalTo to unmarshal messages.
These gists illustrate all the concepts discussed above: