Immutability of event context

[deissnerk]

In the discussion of #277 I raised the question, if extension attributes can be changed or added by middleware, or if the event context is immutable. I remember that during the f2f meeting in Copenhagen @clemensv proposed to have two property bags, one for properties added by the source, and one for annotations added by middleware. AFAIK this is quite similar to what AMQP 1.0 supports. If we decide to abandon the idea of property bags for extensions, how do we handle the question of information added to the context by middleware? There are quite a few options:

1. Don't support information added by middleware. An event is immutable. If anything is changed, it becomes a new event with a new ID.
2. Allow adding of context attributes by any component on the way.
3. Add a dedicated property bag for information that is added after event creation. This could be part of the context or on the same level as data and context.

[jdstuart]

Vote would be for #3.

#1 - yes, events are and should be immutable, so no changes allowed to the original properties.
#2 - the original context property values can be overwritten accidentally if the middleware does not perform checks before setting property values.
#3 - same problem as with #2, but the original context and event data is preserved.

[cneijenhuis]

A question regarding 1. - if the middleware produces a new event, then the middleware is the (new) event producer. Therefore, wouldn't it be forced to set itself as the source?

On one hand, that looks right (one shouldn't be allowed to "impersonate" another event producer), on the other hand it will pose problems to the consumer (expected "example.org/user/...", got "middleware.com/foo").

[Vlaaaaaaad]

Regarding option 1( immutable events), if a new event is to be created two options seem to be available:

• make a bad clone of the initial event( reuse time and event id for example, see Eliminating the who - no middleware concept idea section in Add property-bucket to correlate multiple events associated with a serverless application #128 (comment))
• some type of embedding would have to be done which would leak transport and middleware details into the event

The security( signing and encryption) discussion will also have an impact on this. Also, #266 seems to be related

[duglin]

Do we even need to make a statement about this at all? Whether a middleware acts like "a new source" or it acts more like a proxy is really up to it. I'm not sure CloudEvents itself needs to say anything either way.

There might be some best practices that middleware might want to think about/do - but for the most part it seems like our spec should not get into the processing model side of things, and stick with just how things appear on the wire.

[deissnerk]

@jdstuart I agree with your assessment of the options. My preference would also be option 3.

@Vlaaaaaaad I also agree to you. I have my doubts about middleware cloning the original event. If an event is propagated through multiple channels, you might end up seeing different "versions" of the same event. I also agree on the nesting option.

@cneijenhuis The questions regarding the event source is a good point.

@duglin Our design goals state:

The metadata is not meant to duplicate any of the application data of the event itself, rather it is just the minimal information needed to route the request to the proper component that will process the event.

So the whole point of CloudEvents is to define some data that is needed for processing the event on its way from producer to consumers. To do so, we will need some assumptions about this processing.

[BenBeattieHood]

Isn't any middleware essentially an event source in itself? An event that has been enriched would be considered a new event, emitted from a separate domain responsibility. I'm unsure why this wouldn't apply to metadata too?

[duglin]

I tend to think of middleware as an event source too - however, whether it changes the source attribute is up to it.

I guess I don't yet see the need to differentiate who added an attribute to an event. To the event consumer all they should see (and care about) is the data itself and not who added it, or modified it, along the way. If we start to put data into different bags based on who is adding it then we need to add complex rules for which takes precedence and I think a much simpler model would be to just let the new value replace the old one - then there's no ambiguity or complexity in the processing model. And, keep in mind, there will be some middleware (like proxies) that really will want to modify the original metadata and I don't think its our place to ban those.

I guess I'd like to understand the usecase for why we're even considering such processing rules.

[sslavic]

By https://github.com/cloudevents/spec/blob/master/primer.md middleware is, among other things, expected to

Transformation that changes the event's structure, like mapping from a proprietary format to CloudEvents, while preserving the identity and semantic integrity of the event.

IMO changing event source changes the identity and semantic integrity of the event - it should be a new event with new event ID even if only change is that event source is different.

[BenBeattieHood]

@duglin I essentially agree. The way I see it is that the source represents the authority for an event. I agree that the middleware should replace (or update, or transform, or augment) the event data; and as doing so it is the new authority for the updated data in the event, and so should be considered the source of the resulting event. Hope that makes sense: not a processing rule, more of a principle for trust/authority.

@sslavic I believe the eventID should be a composite of [source]/[event sequence number] so agree that the eventID would therefore be different if an event has gone via middleware, as it's [source] component should be updated.

[duglin]

Discussed a bit on today's call. @deissnerk will continue to think about possible next steps, but please comment here is you have any comments or thoughts on this topic.