Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/src/main/java/io/cloudevents/CloudEvent.java
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,5 @@ public interface CloudEvent extends CloudEventAttributes, CloudEventExtensions {
* The event data
*/
@Nullable
byte[] getData();
Object getData();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would make me abandon the SDK altogether and introduce my own type. A strong no (-1).

Motivation:
Currently, one does not need core to serialize an instance of CloudEvent. If you change it to Object, it would mean that one will have to do something like:

if (getData() instanceof byte[]) {
    // Ok, I know how to deal with it
} else {
    // panic!
}

This is very JS-style plus makes the whole point of the api module obsolete.

It is "fine"-ish if getData will return Data type that gives methods like asBytes, asByteBuffer, or a custom type that will let accessing the original object, but making it Object is a no go IMO.

Copy link
Member Author

@slinkydeveloper slinkydeveloper Aug 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok ok i'm interested in exploring the Data alternative, what about (names are provisional):

public interface Data {
  public byte[] asBytes();

  public @Nullable Object raw();
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to understand your idea, what should it look like the Data interface for you?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few things to keep in mind:

  1. Some implementations that care about the performance may want to implement Data on the same class that implements the CloudEvent type, so the method names should be carefully selected to not clash or confuse.

  2. This can as well be a set of methods of CloudEvent like dataAsBytes(), dataAsByteBuffer(), etc etc. not generic dataAs(Class) tho.

  3. before we make 5 more rounds of PRs, I would like to step back and ask the question that was already asked:
    Why do we even need to expose "raw" object access in CloudEvent on a first place? What are we trying to solve?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. so i wonder if dataAsBytes() and dataAsObject() just solves the problem we have here (I still prefer to have a sub interface and then I implement it in the same class of the cloudevent impl).

  2. [WIP] Allow POJOs as CloudEvent data #198 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dataAsObject() would require doing a mandatory instanceof check on the value to proceed with processing. You can't even use it in the JSON format to encode the body because it may return byte[] of already seriallized payload. You must have a concrete type to hint the serializer to use some other field other than dataAsBytes().

How is it better than doing an instanceof check on the CloudEvent itself?

Copy link
Member Author

@slinkydeveloper slinkydeveloper Aug 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't even use it in the JSON format to encode the body because it may return byte[] of already seriallized payload.

I can and the implementation is pretty straightforward (look at the code here https://github.com/cloudevents/sdk-java/pull/211/files#diff-75b992e6fc5d4cd3890ea6e37c7fe2a4R114)

You must have a concrete type to hint the serializer to use some other field other than dataAsBytes().

WDYM? Every serializer just use val.getClass() to start the serialization.

How is it better than doing an instanceof check on the CloudEvent itself?

Of course you always need the instanceof, but makes simpler the usage and the impl:

  • 90% of users won't have to subclass to put a concrete type of their choice, they just use the CloudEvent provided by core
  • 10% of users subclass overriding the signature, and they return the concrete type, removing the need to use the instanceof
  • Makes explicit that the data is dynamic by nature in a CloudEvent. We can't build in this sdk all the different JsonCloudEvent, XmlCloudEvent because data can be anything...
  • Because it doesn't assume anything on the nature of CloudEvent data, the builders/deserializers don't need to return different impls of the event depending on the payload, they always use the same CloudEventV1 and CloudEventV03

My feeling about subclassing CloudEvent depending on the type of data is that you're making the wrong thing "dynamic", while the Data idea makes more sense to me because is exactly the data field the dynamic one. It's another layer of indirection, but makes more explicit the concept

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can and the implementation is pretty straightforward (look at the code here https://github.com/cloudevents/sdk-java/pull/211/files#diff-75b992e6fc5d4cd3890ea6e37c7fe2a4R114)

this is not a "straightforward implementation", this is if {} else {} for a single case (byte[]) while it may also be ByteBuffer, InputStream, or many other possibilities. dataAsBytes/dataAsByteBuffer do provide a straightforward way of handling the case.

90% of users won't have to subclass to put a concrete type of their choice, they just use the CloudEvent provided by core

Then added it to core. I only care about api here.

10% of users subclass overriding the signature, and they return the concrete type, removing the need to use the instanceof

this is a wrong assumption because, in a pipeline, you will get instances of CloudEvent, not SomeCloudEvent, where getData() will be Object, not the concrete type. That's how Java's type system works.

Makes explicit that the data is dynamic by nature in a CloudEvent.

This does not mean that the SDK should be dynamic tho.

We can't build in this sdk all the different JsonCloudEvent, XmlCloudEvent because data can be anything...
Because it doesn't assume anything on the nature of CloudEvent data, the builders/deserializers don't need to return different impls of the event depending on the payload, they always use the same CloudEventV1 and CloudEventV03

That's not what I suggested. Please re-read.

My feeling about subclassing CloudEvent depending on the type of data is that you're making the wrong thing "dynamic", while the Data idea makes more sense to me because is exactly the data field the dynamic one.

Your perception is incorrect. I am saying that, if someone wants to apply an optimization, they can subclass and expose the raw object. Just re-read the original issue ( #196 ), the problem was in "double encoding".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a "straightforward implementation", this is if {} else {} for a single case (byte[]) while it may also be ByteBuffer, InputStream, or many other possibilities. dataAsBytes/dataAsByteBuffer do provide a straightforward way of handling the case.

That's the very same thing i have to do with subclassing CloudEvent. Also, keep in mind that i cannot write directly byte[] into a jackson serializer, i need or a json type or a pojo to map to json (look at the diff).

Then added it to core. I only care about api here.
Makes explicit that the data is dynamic by nature in a CloudEvent.

I know, but the users of this sdk ends up using the interface CloudEvent in api, hence I care to have well aligned interface and impls for 90% of users without letting them downcast because the 10% thinks data is not a dynamic field and we can "statically type it". Just look at the spec of CloudEvents https://github.com/cloudevents/spec/blob/v1.0/spec.md#event-data and you clearly see that "This specification does not place any restriction on the type of this information", hence what is the point of trying to make it look static with subclasses and whatnot?

I am saying that, if someone wants to apply an optimization, they can subclass and expose the raw object

I'm not talking only about optimizations, I'm talking about the fact that if the user is allowed to put pojos inside the event (eg using the event builder providing a non byte array parameter), she/he expects to be able to get it back with a getter, like getDataAsObject().
And as I said, I see no point for the user of the sdk to do Object data = ((BaseCloudEventImpl)event).getRawData() where they build with builder.withData(pojo). That makes the usage unnecessary awful...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the very same thing i have to do with subclassing CloudEvent. Also, keep in mind that i cannot write directly byte[] into a jackson serializer, i need or a json type or a pojo to map to json (look at the diff).

With byte[], you don't need Jackson at all, you have the bytes already. It is up to the implementation to decide how to return byte[]. That's the whole point.

}
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ public enum CloudEventRWExceptionKind {
INVALID_ATTRIBUTE_TYPE,
INVALID_ATTRIBUTE_VALUE,
INVALID_EXTENSION_TYPE,
DATA_CODEC_ERROR,
OTHER
}

Expand Down Expand Up @@ -85,6 +86,14 @@ public static CloudEventRWException newInvalidExtensionType(String extensionName
);
}

public static CloudEventRWException newDataCodecError(Throwable cause) {
return new CloudEventRWException(
CloudEventRWExceptionKind.DATA_CODEC_ERROR,
"Error while serializing data",
cause
);
}

public static CloudEventRWException newOther(Throwable cause) {
return new CloudEventRWException(
CloudEventRWExceptionKind.OTHER,
Expand Down
7 changes: 6 additions & 1 deletion api/src/main/java/io/cloudevents/rw/CloudEventWriter.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@

package io.cloudevents.rw;

import io.cloudevents.lang.Nullable;

/**
* Interface to write the content (CloudEvents attributes, extensions and payload) from a
* {@link io.cloudevents.rw.CloudEventReader} to a new representation.
Expand All @@ -27,10 +29,13 @@ public interface CloudEventWriter<R> extends CloudEventAttributesWriter, CloudEv

/**
* End the visit with a data field
* <p>
* TODO add note that explains that contentType is already provided before visiting the attributes
* TODO explain that value cannot be a base64 serialized byte array
*
* @return an eventual return value
*/
R end(byte[] value) throws CloudEventRWException;
R end(@Nullable String contentType, Object value) throws CloudEventRWException;

/**
* End the visit
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

package io.cloudevents.bench.jackson;

import io.cloudevents.jackson.JsonFormat;
import io.cloudevents.jackson.Json;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;

Expand All @@ -31,7 +31,7 @@ public static class DeserializationState {

public byte[] eventWithJson;
public byte[] eventWithXml;
public JsonFormat format = new JsonFormat();
public Json format = new Json();

public DeserializationState() {
eventWithJson = format.serialize(V1_WITH_JSON_DATA_WITH_EXT);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import io.cloudevents.CloudEvent;
import io.cloudevents.core.builder.CloudEventBuilder;
import io.cloudevents.jackson.JsonFormat;
import io.cloudevents.jackson.Json;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;

Expand All @@ -32,7 +32,7 @@ public class JsonFormatSerializationBenchmark {
public static class SerializationState {
public CloudEvent eventWithJson = CloudEventBuilder.v1(V1_WITH_JSON_DATA_WITH_EXT).build();
public CloudEvent eventWithXml = CloudEventBuilder.v1(V1_WITH_XML_DATA).build();
public JsonFormat format = new JsonFormat();
public Json format = new Json();
}

@Benchmark
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import io.cloudevents.CloudEvent;
import io.cloudevents.core.builder.CloudEventBuilder;
import io.cloudevents.jackson.JsonFormat;
import io.cloudevents.jackson.Json;
import io.cloudevents.kafka.KafkaMessageFactory;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;
Expand Down Expand Up @@ -50,7 +50,7 @@ public void testStructuredJsonEncoding(Event event, Blackhole bh) {
bh.consume(
KafkaMessageFactory
.createWriter("aaa")
.writeStructured(event.event, JsonFormat.CONTENT_TYPE)
.writeStructured(event.event, Json.CONTENT_TYPE)
);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

package io.cloudevents.bench.kafka;

import io.cloudevents.jackson.JsonFormat;
import io.cloudevents.jackson.Json;
import io.cloudevents.kafka.KafkaMessageFactory;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.producer.ProducerRecord;
Expand Down Expand Up @@ -73,7 +73,7 @@ public StructuredJsonMessage() {
// Hack to generate a consumer message
ProducerRecord<Void, byte[]> inRecord = KafkaMessageFactory
.createWriter("aaa")
.writeStructured(V1_WITH_JSON_DATA_WITH_EXT, JsonFormat.CONTENT_TYPE);
.writeStructured(V1_WITH_JSON_DATA_WITH_EXT, Json.CONTENT_TYPE);

this.message = new ConsumerRecord<>(
"aaa",
Expand Down
1 change: 1 addition & 0 deletions core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
<packaging>jar</packaging>

<properties>
<jackson.version>2.11.2</jackson.version>
<module-name>io.cloudevents.core</module-name>
</properties>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ public interface CloudEventBuilder extends CloudEventWriter<CloudEvent> {
* @param data data of the event
* @return self
*/
CloudEventBuilder withData(byte[] data);
CloudEventBuilder withData(Object data);

/**
* Set the {@code datacontenttype} and {@code data} of the event
Expand All @@ -104,7 +104,7 @@ public interface CloudEventBuilder extends CloudEventWriter<CloudEvent> {
* @param data data of the event
* @return self
*/
CloudEventBuilder withData(String dataContentType, byte[] data);
CloudEventBuilder withData(String dataContentType, Object data);

/**
* Set the {@code datacontenttype}, {@code dataschema} and {@code data} of the event
Expand All @@ -114,7 +114,7 @@ public interface CloudEventBuilder extends CloudEventWriter<CloudEvent> {
* @param data data of the event
* @return self
*/
CloudEventBuilder withData(String dataContentType, URI dataSchema, byte[] data);
CloudEventBuilder withData(String dataContentType, URI dataSchema, Object data);

/**
* Set an extension with provided key and string value
Expand Down Expand Up @@ -214,4 +214,22 @@ static CloudEventBuilder fromSpecVersion(@Nonnull SpecVersion version) {
);
}

/**
* Create a new builder with a copy of the content inside the provided {@code event}.
*
* @param event event to copy
* @return a new builder
*/
static CloudEventBuilder from(@Nonnull CloudEvent event) {
switch (event.getSpecVersion()) {
case V1:
return CloudEventBuilder.v1(event);
case V03:
return CloudEventBuilder.v03(event);
}
throw new IllegalStateException(
"The provided spec version doesn't exist. Please make sure your io.cloudevents deps versions are aligned."
);
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
* Copyright 2018-Present The CloudEvents Authors
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/

package io.cloudevents.core.codec;

import io.cloudevents.core.format.EventFormat;

/**
* Exception representing a serialization error while using an {@link EventFormat}.
*/
public class DataSerializationException extends RuntimeException {
public DataSerializationException(Throwable e) {
super(e);
}
}
31 changes: 31 additions & 0 deletions core/src/main/java/io/cloudevents/core/codec/EventDataCodec.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package io.cloudevents.core.codec;

import io.cloudevents.CloudEvent;
import io.cloudevents.core.format.EventSerializationException;
import io.cloudevents.lang.Nullable;

import javax.annotation.ParametersAreNonnullByDefault;

/**
* This interface represents how to encode an event data to bytes, in order to write the payload on the wire.
*/
@ParametersAreNonnullByDefault
public interface EventDataCodec {

/**
* Serialize a {@link CloudEvent} to a byte array.
*
* @param data the data to serialize.
* @return the byte representation of the provided event.
* @throws EventSerializationException if something goes wrong during serialization.
* @throws IllegalArgumentException if this codec cannot serialize the provided data
*/
byte[] serialize(@Nullable String dataContentType, Object data) throws DataSerializationException, IllegalArgumentException;

/**
* @param dataContentType the eventual content type to serialize
* @return true if this event codec instance can serialize the provided content type.
*/
boolean canSerialize(@Nullable String dataContentType);

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package io.cloudevents.core.codec.impl;

import io.cloudevents.core.codec.DataSerializationException;
import io.cloudevents.core.codec.EventDataCodec;

public class TextEventDataCodec implements EventDataCodec {
@Override
public byte[] serialize(String dataContentType, Object data) throws DataSerializationException, IllegalArgumentException {
try {
return data.toString().getBytes();
} catch (Throwable e) {
throw new DataSerializationException(e);
}
}

@Override
public boolean canSerialize(String dataContentType) {
return "text/plain".equals(dataContentType);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,16 @@

public abstract class BaseCloudEvent implements CloudEvent, CloudEventReader {

private final byte[] data;
private final Object data;
protected final Map<String, Object> extensions;

protected BaseCloudEvent(byte[] data, Map<String, Object> extensions) {
protected BaseCloudEvent(Object data, Map<String, Object> extensions) {
this.data = data;
this.extensions = extensions != null ? extensions : new HashMap<>();
}

@Override
public byte[] getData() {
public Object getData() {
return this.data;
}

Expand All @@ -55,7 +55,7 @@ public <T extends CloudEventWriter<V>, V> V read(CloudEventWriterFactory<T, V> w
this.readExtensions(visitor);

if (this.data != null) {
return visitor.end(this.data);
return visitor.end(this.getDataContentType(), this.data);
}

return visitor.end();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,12 @@
import java.util.HashMap;
import java.util.Map;

public abstract class BaseCloudEventBuilder<SELF extends BaseCloudEventBuilder<SELF, T>, T extends CloudEvent> implements CloudEventBuilder {
public abstract class BaseCloudEventBuilder<SELF extends BaseCloudEventBuilder<SELF>> implements CloudEventBuilder {

// This is a little trick for enabling fluency
private final SELF self;

protected byte[] data;
protected Object data;
protected Map<String, Object> extensions;

@SuppressWarnings("unchecked")
Expand All @@ -55,21 +55,18 @@ public BaseCloudEventBuilder(CloudEvent event) {

protected abstract void setAttributes(CloudEvent event);

//TODO builder should accept data as Object and use data codecs (that we need to implement)
// to encode data

public SELF withData(byte[] data) {
public SELF withData(Object data) {
this.data = data;
return this.self;
}

public SELF withData(String dataContentType, byte[] data) {
public SELF withData(String dataContentType, Object data) {
withDataContentType(dataContentType);
withData(data);
return this.self;
}

public SELF withData(String dataContentType, URI dataSchema, byte[] data) {
public SELF withData(String dataContentType, URI dataSchema, Object data) {
withDataContentType(dataContentType);
withDataSchema(dataSchema);
withData(data);
Expand Down Expand Up @@ -117,7 +114,7 @@ public void setExtension(String name, Boolean value) throws CloudEventRWExceptio
}

@Override
public CloudEvent end(byte[] value) throws CloudEventRWException {
public CloudEvent end(String contentType, Object value) throws CloudEventRWException {
this.data = value;
return build();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

public class CloudEventReaderAdapter implements CloudEventReader {

private CloudEvent event;
private final CloudEvent event;

CloudEventReaderAdapter(CloudEvent event) {
this.event = event;
Expand All @@ -35,7 +35,7 @@ public <V extends CloudEventWriter<R>, R> R read(CloudEventWriterFactory<V, R> w
this.readExtensions(visitor);

if (event.getData() != null) {
return visitor.end(event.getData());
return visitor.end(event.getDataContentType(), event.getData());
}

return visitor.end();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ private CloudEventUtils() {}

/**
* Convert a {@link CloudEvent} to a {@link CloudEventReader}. This method provides a default implementation
* for CloudEvent that doesn't implement CloudEventVisitable
* for {@link CloudEvent} implementations that don't implement {@link CloudEventReader}
*
* @param event the event to convert
* @return the visitable implementation
Expand Down
Loading