Kafka client fine tuning

Kafka client fine-tuning

Kafka.EventLoop is built on top of the Confluent's .NET Client for Apache Kafka.

You have access to its consumer and producer configurations through the helper method HasKafkaConfig.

Consumer fine-tuning:

services.AddKafkaEventLoop(ctx.Configuration, o => o
  .HasConsumerGroup("<your-consumer-group-id>", cgOptions => cgOptions
    .HasMessageType<MyMessage>()
    .HasJsonMessageDeserializer()
    .HasController<MyController>()
    .HasKafkaConfig(c => c.AutoOffsetReset = AutoOffsetReset.Earliest)  // <--
    .HasKafkaConfig(c => c.MaxPollIntervalMs = 900000)                  // <--
    .Build())
  .Build());

Limitations

• EnableAutoCommit - you cannot set true because the library makes manual commits after each successful message processing.
• EnableAutoOffsetStore - you cannot set true because the library disables auto-offset store as it's not needed with manual commits.

Dead-lettering producer fine-tuning:

services.AddKafkaEventLoop(ctx.Configuration, o => o
  .HasConsumerGroup("<your-consumer-group-id>", cgOptions => cgOptions
    .HasMessageType<MyMessage>()
    .HasJsonMessageDeserializer()
    .HasController<MyController>()
    .HasDeadLettering(dlOptions => dlOptions
      .HasJsonDeadLetterMessageSerializer()
      .HasKafkaConfig(p => p.Acks = Acks.All) // <-- example
      .Build())
    .Build())
  .Build());

Streaming producer fine-tuning:

services.AddKafkaEventLoop(ctx.Configuration, o => o
  .HasConsumerGroup("<your-consumer-group-id>", cgOptions => cgOptions
    .HasMessageType<MyMessage>()
    .HasJsonMessageDeserializer()
    .HasController<MyController>()
    .HasStreaming<MyExtendedMessage>(sOptions => sOptions
      .HasJsonOutMessageSerializer()
      .HasKafkaConfig(p => p.RequestTimeoutMs = 2000) // <-- example
      .Build())
    .Build())
  .Build());