Kafka partition key clarification

[slinkydeveloper]

In Kafka binding specification, it's specified that the message key should be included in the event as key extension. But then, the specification links to the partitioning extension specification, which states that the key extension should be called partitionKey.

Which of the ones should be used?

[duglin]

@bluemonk3y any thoughts on this?

[duglin]

On 3/5 call we agreed to "PartitionKey"

[slinkydeveloper]

Now that I look again at this issue, partitionKey sounds a little bit confusing for Kafka specific: the Kafka Consumer in Java SDK returns ConsumerRecord, which exposes both the key of the message itself (which is what we mean as partitionKey) and the topic partition id, which is the number that identifies the partition where the message is stored. The same if for ProducerRecord, and generally other clients also exposes and allows you to set the partition id.
So looking at partitionKey, It's confusing for a Kafka user to understand if we're talking about the topic partition or the message key

[duglin]

@slinkydeveloper what do you suggest we do?

[slinkydeveloper]

The key in a kafka message is well defined by the kafka message specification already: I don't see the reason to add the partitioning extension, nor the partitionKeyExtractor thing which is just confusing. I think we should reword, removing both the partitioning spec link and the partitionKeyExtractor, and just state that the Kafka message key maps to the key extension of the event.

[OneCricketeer]

Or messageKey & partitionId if you want to be verbose

[duglin]

@clemensv took an AI to write-up a proposal

[duglin]

@slinkydeveloper we can close this now, right?