ProtoBuf Utilities

GitLab Docs Pipeline Coverage License

A set of Java utility libraries that make Protocol Buffers easier to work with as a dynamic type. The core module provides convenient wrappers for messages, enums, lists, and maps, while additional modules add JSON serialization (via Gson or Jackson), database integration (via JDBI v3), and test matchers.

See morimekta.net/utils for procedures on releases.

Getting Started

Add the modules you need to your build configuration. Most projects will want proto-core plus one or more of the serialization modules.

Maven (pom.xml):

<dependency>
    <groupId>net.morimekta.utils</groupId>
    <artifactId>proto-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>net.morimekta.utils</groupId>
    <artifactId>proto-gson</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>net.morimekta.utils</groupId>
    <artifactId>proto-jackson</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>net.morimekta.utils</groupId>
    <artifactId>proto-testing</artifactId>
    <version>1.1.0</version>
    <scope>test</scope>
</dependency>

Gradle (build.gradle):

implementation 'net.morimekta.utils:proto-core:1.1.0'
implementation 'net.morimekta.utils:proto-gson:1.1.0'
implementation 'net.morimekta.utils:proto-jackson:1.1.0'
testImplementation 'net.morimekta.utils:proto-testing:1.1.0'

Proto : Core

Provides wrappers and utilities for working with message and enum types, converting values, and accessing fields.

  • ProtoMessage -- Wraps a message instance for convenient field access with value caching and type introspection.
  • ProtoEnum -- Wraps an enum type with cached lookups by name, number, or value descriptor.
  • ProtoList -- Wraps a repeated field as a standard Java List. A mutable variant (ProtoListBuilder) propagates changes back to the underlying message builder.
  • ProtoMap -- Wraps a map field as a standard Java Map. A mutable variant (ProtoMapBuilder) propagates changes back to the underlying message builder.

Additional utilities cover field introspection, message building, reflection-based access to static message methods, value conversion, and helpers for well-known Google types.

Example

import net.morimekta.proto.ProtoMessage;
import net.morimekta.proto.ProtoEnum;
import net.morimekta.proto.ProtoMap;

// Wrap a message for cached field access
var message = new ProtoMessage(myProto);
Object value = message.get(fieldDescriptor);

// Create a builder via reflection from a descriptor
Message.Builder builder = ProtoMessage.newBuilder(descriptor);

// Look up enum values by name or number
var protoEnum = new ProtoEnum<>(MyEnum.getDescriptor(), MyEnum.class);
MyEnum val = protoEnum.findByName("ACTIVE");

// Access map fields as a java.util.Map
var map = new ProtoMap<String, MyMessage>(message, mapFieldDescriptor);
MyMessage entry = map.get("key");

Proto : Gson

Serializes and deserializes proto messages and enums using the Gson JSON library, with fine-grained control over the output format.

Supported features:

  • Write field keys as field numbers or names.
  • Write enum values as numbers or names.
  • Write compact messages using JSON array syntax (fields in numeric order).
  • Write Any messages with the unpacked content and a @type marker.
  • Read back all of the above formats.
  • Choose strict parsing (reject unknown fields and values) or lenient parsing (silently ignore them).

Options

Use ProtoTypeOptions.Option to toggle reading and writing behavior:

  • FAIL_ON_UNKNOWN_ENUM -- Reject unknown enum values during parsing. Otherwise, unknown values resolve to null.
  • FAIL_ON_UNKNOWN_FIELD -- Reject unknown message fields during parsing. Otherwise, unknown fields are skipped.
  • FAIL_ON_NULL_VALUE -- Reject null values (explicit nulls or unparseable field values) during parsing.
  • IGNORE_UNKNOWN_ANY_TYPE -- Skip Any messages whose type is not in the registry instead of failing.
  • LENIENT_READER -- Allow non-standard conversions, such as strings to booleans or numbers.
  • WRITE_FIELD_AS_NUMBER -- Write fields using field numbers instead of names.
  • WRITE_ENUM_AS_NUMBER -- Write enum values as their numeric identifiers instead of names.
  • WRITE_UNPACKED_ANY -- Write Any messages with known types as their unpacked JSON content, prefixed with a @type (or __type) field.
  • WRITE_COMPACT_MESSAGE -- Write messages annotated with morimekta.proto.compact as JSON arrays instead of objects. Trailing nulls are omitted.
  • WRITE_TIMESTAMP_AS_ISO -- Write google.protobuf.Timestamp as an ISO instant string (e.g. 2009-02-13T23:31:30Z).
  • WRITE_DURATION_AS_STRING -- Write google.protobuf.Duration as a duration string (e.g. 3.7s).

Use ProtoTypeOptions.Value to configure string-valued settings:

  • ANY_TYPE_FIELD_NAME -- Field name for the type marker in unpacked Any messages. Defaults to @type.
  • ANY_TYPE_PREFIX -- Prefix before the protobuf type name in the type field value. Defaults to type.googleapis.com/.

To detect types of unwrapped Any messages and extensions, register the appropriate types in the type registry via ProtoTypeOptions.

Example

import net.morimekta.proto.gson.ProtoTypeAdapterFactory;
import net.morimekta.proto.gson.ProtoTypeOptions;
import com.google.gson.GsonBuilder;

// Configure options (defaults include FAIL_ON_NULL_VALUE, WRITE_UNPACKED_ANY,
// WRITE_TIMESTAMP_AS_ISO, and WRITE_DURATION_AS_STRING)
var options = new ProtoTypeOptions()
        .withEnabled(ProtoTypeOptions.Option.WRITE_FIELD_AS_NUMBER)
        .withDisabled(ProtoTypeOptions.Option.FAIL_ON_NULL_VALUE);

var gson = new GsonBuilder()
        .registerTypeAdapterFactory(new ProtoTypeAdapterFactory(options))
        .create();

// Serialize
String json = gson.toJson(myMessage);

// Deserialize
MyMessage message = gson.fromJson(json, MyMessage.class);

Proto : Jackson

Provides the same serialization and deserialization capabilities as the Gson module, using Jackson instead.

  • Register ProtoModule with your ObjectMapper to enable automatic handling of all proto types (Message, ProtocolMessageEnum, ByteString).
  • Use ProtoFeature to toggle boolean behavior flags. The available flags mirror ProtoTypeOptions.Option from proto-gson with matching defaults.
  • Use ProtoStringFeature to configure string-valued settings, mirroring ProtoTypeOptions.Value from proto-gson.

Example

import net.morimekta.proto.jackson.ProtoModule;
import net.morimekta.proto.jackson.ProtoFeature;
import com.fasterxml.jackson.databind.ObjectMapper;

var mapper = new ObjectMapper();
mapper.registerModule(new ProtoModule());

// Optionally enable or disable features
ProtoFeature.enableFeatures(mapper,
        ProtoFeature.WRITE_FIELD_AS_NUMBER);
ProtoFeature.disableFeatures(mapper,
        ProtoFeature.FAIL_ON_NULL_VALUE);

// Serialize
String json = mapper.writeValueAsString(myMessage);

// Deserialize
MyMessage message = mapper.readValue(json, MyMessage.class);

Proto : JDBI

Provides JDBI v3 integration for mapping proto messages, enums, and ByteString values to database columns and rows.

Binding annotations

  • @BindMessage -- Binds a message as a single column value, either as binary (VARBINARY) or as a string (VARCHAR).
  • @BindEnumName -- Binds a proto enum as its string name.
  • @BindEnumNumber -- Binds a proto enum as its numeric value.
  • @BindByteString -- Binds a ByteString value, with support for configuring the column encoding (binary or string-like).

Registration annotations

  • @RegisterMessageMapper -- Registers row and column mappers for a proto Message type.
  • @RegisterEnumMapper -- Registers column mappers for reading enum values from result sets.
  • @RegisterByteString -- Registers ByteString column result and argument mappers.

MessageUpserter

A helper class for inserting or upserting message data into a table. It maps top-level message fields directly to columns (nested structures are not expanded).

Example

import net.morimekta.proto.jdbi.v3.MessageUpserter;
import net.morimekta.proto.jdbi.v3.annotations.*;

// Build an upserter for a table
var upserter = new MessageUpserter.Builder<>("users")
        .set(User.UUID, User.NAME, User.EMAIL)
        .onDuplicateKeyUpdate(User.NAME, User.EMAIL)
        .build();

// Execute with a JDBI handle
try (Handle handle = jdbi.open()) {
    upserter.execute(handle, user1, user2);
}

// Use annotations on a JDBI SqlObject
@RegisterMessageMapper(User.class)
@RegisterEnumMapper(Status.class)
public interface UserDao {
    @SqlQuery("SELECT * FROM users WHERE uuid = :uuid")
    User findByUuid(@Bind("uuid") String uuid);

    @SqlUpdate("INSERT INTO users (uuid, name, status) VALUES (:msg.uuid, :msg.name, :status)")
    void insert(@BindMessage("msg") User user,
                @BindEnumName("status") Status status);
}

Proto : Testing

Provides Hamcrest matchers for comparing proto messages in tests. Differences are displayed in a readable diff-like format, and individual fields can be excluded from comparison using proto FieldMask path syntax.

Example

import static net.morimekta.proto.testing.ProtoMatchers.equalToMessage;
import static org.hamcrest.MatcherAssert.assertThat;

// Exact comparison
assertThat(actual, equalToMessage(expected));

// Ignore specific fields (uses FieldMask path syntax)
assertThat(actual, equalToMessage(expected, "updated_at", "metadata.trace_id"));