Transforms: Move Message instead of Copy Message

[geudrik]

A bit of a weird one - I've got a usecase where I want to use a transform to add a field to a message. The field is static data, but I'm not able to change the source to have it added there. This same pattern is copy/paste across N sources.

The idea is 1 topic per source, and per-topic add a field that labels the message per source. Once that field is added to the message, we can just dump it into a single topic where all messages mixed together.

Using the copy message to a new topic example as the ref, is there a way to just "move" the message to the new topic, and bypass the original topic entirely ("move" vs. clone to a second topic)?

A different way of thinking about this is "drop the message on the floor (instead of adding it to the original topic), after copying the mutated message to the --dest-topic topic"

If this isn't natively doable using just transforms, is there a pattern for doing this by setting (for example) retention.ms?

[dotnwat]

cc @rockwotj

[rockwotj]

Thanks @geudrik! Currently deleting the data isn't supported directly after a transform. As you mentioned, what we recommend is a low retention.ms value for the input topic in the case where you don't want to keep the original data

[geudrik]

Ack, appreciate the info! Thanks!