Send Custom Header only to DLT Topic.

[wojciechcwek]

Hi All,

I have added the following logic to add custom header in events that are sent to retry and dlt topics. I use spring-kafa 2.8.8.

@Bean(RetryTopicInternalBeanNames.DEAD_LETTER_PUBLISHING_RECOVERER_FACTORY_BEAN_NAME)
public DeadLetterPublishingRecovererFactory factory(DestinationTopicResolver destinationTopicResolver) {
    DeadLetterPublishingRecovererFactory factory = new DeadLetterPublishingRecovererFactory(destinationTopicResolver);
    factory.setDeadLetterPublishingRecovererCustomizer(dlpr -> dlpr.addHeadersFunction((consumerRecord, exception) -> {
     
return new RecordHeaders().add(EXTERNAL_SYSTEM_EXCEPTION_MESSAGE_HEADER, exc.getMessage().getBytes(StandardCharsets.UTF_8));

    ));
    return factory;
}

Unfortunately the following header is passed to another topic. I'll give you an example: I have 2 retry topics and dlt. The exception is thrown and the header is added at main topic, then at retry-0 I receive the event, where that header already is and then I add it again, because the exception was thrown once again. So at retry-1 I have 2 same headers. It ends in dlt topic, where are 3 headers with the same name. It happens, because "Apache Kafka supports multiple headers with the same name" as it is written here: https://docs.spring.io/spring-kafka/reference/html/#dlpr-headers

I think of two solutions and I'm not sure how to achieve them:

1. The header can be added at each level, but before addition, it should be checked, if it exists. If yes than should be removed and then added. -> Here I think I could override exceptionHeadersCreator, but it would not attach the default exception headers.
2. The header should be added only to DLT topic.

Do you know if the second solution is possible?

Thanks.

[garyrussell]

In your custom headers function, you can check the consumerRecord.topic() to see if it is the one before the DLT and only add the header in that case.

[garyrussell]

However, please add a new feature issue to enable header deletion.

[wojciechcwek]

Thank you. Issue added: #2528