Skip to content

Conversation

@lucasbru
Copy link
Member

@lucasbru lucasbru commented Feb 9, 2026

Summary

This refactoring simplifies the streams group coordinator by replacing
the ConfiguredTopology, ConfiguredSubtopology, and
ConfiguredInternalTopic classes with a simpler
TopologyValidationResult record.

TopologyValidationResult is a new record that captures the validation
state of a topology, including the topology epoch, internal topics to be
created, any configuration exceptions, and an optional
maxPartitionsPerSubtopology map when the topology is ready. Storing
only the pre-computed map (instead of TopologyMetadata) avoids
retaining a reference to CoordinatorMetadataImage in Timeline
collections.

TopologyMetadata is now a lightweight class that accepts pre-computed
partition counts and StreamsTopology. It is constructed on-the-fly in
TargetAssignmentBuilder when needed, rather than being stored in
TopologyValidationResult. The computeAllMaxPartitions method was
moved from TopologyMetadata to InternalTopicManager.

InternalTopicManager.configureTopics now returns
TopologyValidationResult with maxPartitionsPerSubtopology directly.
The topology metadata is only created when all required topics exist.

StreamsGroup stores TopologyValidationResult instead of
ConfiguredTopology, with renamed accessor methods.

TargetAssignmentBuilder now accepts both TopologyValidationResult
and StreamsTopology via withTopology(), and constructs
TopologyMetadata on the fly when the topology is ready.

EndpointToPartitionsManager now takes StreamsTopology directly as a
parameter instead of fetching it from StreamsGroup, which fixes a
timing issue during heartbeat processing.

The three Configured* classes and their tests are deleted as they are
no longer needed.

The jmh-benchmarks module has been updated to use the new
TopologyMetadata constructor with StreamsTopology and pre-computed
max partitions.

Test plan

  • Unit tests pass: TopologyValidationResultTest,
    TopologyMetadataTest, InternalTopicManagerTest,
    TargetAssignmentBuilderTest, EndpointToPartitionsManagerTest,
    StreamsGroupTest
  • All GroupMetadataManagerTest tests pass (410+ tests)
  • All streams group coordinator tests pass
  • Checkstyle and spotless checks pass
  • JMH benchmarks compile and pass style checks

@lucasbru lucasbru force-pushed the lucasbru/replace-configured-topology-with-validation-result branch 2 times, most recently from c4f2973 to b8306f3 Compare February 9, 2026 19:22
This refactoring simplifies the streams group coordinator by replacing the ConfiguredTopology, ConfiguredSubtopology, and ConfiguredInternalTopic classes with a simpler TopologyValidationResult record.

The key changes are:

TopologyValidationResult is a new record that captures the validation state of a topology, including the topology epoch, internal topics to be created, any configuration exceptions, and an optional maxPartitionsPerSubtopology map when the topology is ready. Storing only the pre-computed map instead of TopologyMetadata avoids retaining a reference to CoordinatorMetadataImage in Timeline collections.

TopologyMetadata is now a lightweight class that accepts pre-computed partition counts and StreamsTopology. It is constructed on-the-fly in TargetAssignmentBuilder when needed, rather than being stored in TopologyValidationResult.

InternalTopicManager.configureTopics now returns TopologyValidationResult with maxPartitionsPerSubtopology directly instead of TopologyMetadata. The computeAllMaxPartitions method was moved from TopologyMetadata to InternalTopicManager.

StreamsGroup stores TopologyValidationResult instead of ConfiguredTopology, with renamed accessor methods.

TargetAssignmentBuilder now accepts both TopologyValidationResult and StreamsTopology via withTopology(), and constructs TopologyMetadata on the fly when the topology is ready.

EndpointToPartitionsManager now takes StreamsTopology directly as a parameter instead of fetching it from StreamsGroup, which fixes a timing issue during heartbeat processing.

The three Configured* classes and their tests are deleted as they are no longer needed.
@lucasbru lucasbru force-pushed the lucasbru/replace-configured-topology-with-validation-result branch from b8306f3 to bb0d8af Compare February 9, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant