How to best deal with conversions/seralization to other formats that byte[]? #196
Description
I'm experimenting with using cloudevents as the storage format for an event sourcing library I'm building (you can find the project here). However when I upgraded to io.cloudevents:cloudevents-json-jackson:2.0.0-milestone1 from 1.3.0 I noticed that the "data" field is no longer generic but is now always byte[]. I.e. where you previously could do:
builder.withData(Map.of("hello", "World")). ..you now need to convert the map into a byte[]:
builder.withData(objectMapper.writeValueAsBytes(Map.of("hello", "World"))). ..Also EventFormat has changed to only deal with byte[].
I actually think I like this change but it comes with drawbacks in some cases. In my case I'm trying to serialize the CloudEvents to be stored in MongoDB. Since MongoDB understands json natively I'd like to convert the CloudEvent instance to a model that MongoDB Java Driver understands, for example a org.bson.Document. Currently, if I want to use the io.cloudevents.jackson.JsonFormat, I need to do double encoding:
CloudEvent ce = ..
byte[] serializedCloudEventAsByteArray = jsonEventFormat.serialize(ce);
// I want to store a Document so I need to convert again
String serializedCloudEventAsString = new String(bytes, UTF_8);
Document cloudEventAsDocument = Document.parse(serializedCloudEventAsString);
mongo.insert(cloudEventAsDocument);This involves several unnecessary steps. Since the Document.parse method only takes a String I need to first create a String from the byte[] produced by the EventFormat and then pass it to Document.parse which will parse the JSON string and convert it into a Document.
Compare that with using jackson directly where you could simply do:
CloudEvent ce = ..
Document document = objectMapper.convertValue(ce, Document.class);
mongo.insert(cloudEventAsDocument);which is much more efficient but prohibited when using an instance of EventFormat.
Questions
How should one think when dealing with issues as the one above? I think the cloud event spec fit well as storage format but the current implementation prohibits efficient mapping to datastore model (org.bson.Document in this case). Should you avoid using EventFormat in these cases and write custom serialization/deserialization code?
For reference I've actually copied io.cloudevents.jackson.JsonFormat into a custom DocumentFormat (which cannot implement EventFormat since the types match) to demonstrate the point. As you can see I cannot implement EventFormat since it requires a byte array. Other than that it's virtually identical to the existing JsonFormat but instead of doing objectMapper.writeValueAsBytes(..) I use the objectMapper.convertValue(..) method.
package se.haleby.occurrent.eventstore.mongodb.converter;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import io.cloudevents.CloudEvent;
import io.cloudevents.core.format.EventDeserializationException;
import io.cloudevents.core.format.EventFormat;
import io.cloudevents.core.format.EventSerializationException;
import io.cloudevents.jackson.CloudEventDeserializer;
import io.cloudevents.jackson.CloudEventSerializer;
import org.bson.Document;
public final class DocumentFormat {
public static final String CONTENT_TYPE = "application/cloudevents+json";
private final ObjectMapper mapper;
private final boolean forceDataBase64Serialization;
private final boolean forceStringSerialization;
public DocumentFormat(boolean forceDataBase64Serialization, boolean forceStringSerialization) {
this.mapper = new ObjectMapper();
this.mapper.registerModule(getCloudEventJacksonModule(forceDataBase64Serialization, forceStringSerialization));
this.forceDataBase64Serialization = forceDataBase64Serialization;
this.forceStringSerialization = forceStringSerialization;
}
public DocumentFormat() {
this(false, false);
}
/**
* @return a copy of this JsonFormat that serialize events with json data with Base64 encoding
*/
public DocumentFormat withForceJsonDataToBase64() {
return new DocumentFormat(true, this.forceStringSerialization);
}
/**
* @return a copy of this JsonFormat that serialize events with non-json data as string
*/
public DocumentFormat withForceNonJsonDataToString() {
return new DocumentFormat(this.forceDataBase64Serialization, true);
}
public Document serialize(CloudEvent event) throws EventSerializationException {
return mapper.convertValue(event, Document.class);
}
public CloudEvent deserialize(Document document) throws EventDeserializationException {
return mapper.convertValue(document, CloudEvent.class);
}
public String serializedContentType() {
return CONTENT_TYPE;
}
/**
* @return a JacksonModule with CloudEvent serializer/deserializer with default values
*/
public static SimpleModule getCloudEventJacksonModule() {
return getCloudEventJacksonModule(false, false);
}
/**
* @return a JacksonModule with CloudEvent serializer/deserializer customizing the data serialization.
* Look at {@link #withForceJsonDataToBase64()} and {@link #withForceNonJsonDataToString()} for more details.
*/
public static SimpleModule getCloudEventJacksonModule(boolean forceDataBase64Serialization, boolean forceStringSerialization) {
final SimpleModule ceModule = new SimpleModule("CloudEvent");
ceModule.addSerializer(CloudEvent.class, new CloudEventSerializer(forceDataBase64Serialization, forceStringSerialization) {
});
ceModule.addDeserializer(CloudEvent.class, new CloudEventDeserializer() {
});
return ceModule;
}
protected static boolean dataIsJsonContentType(String contentType) {
// If content type, spec states that we should assume is json
return contentType == null || contentType.startsWith("application/json") || contentType.startsWith("text/json");
}
}