Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI never ends for unit tests with manual test container lifecycle control #782

Conversation

guillermocalvo
Copy link
Contributor

Steps to reproduce the problem:

  • A unit test must create a singleton kafka container.
  • There must be some kafka consumer in the application context.

Example:

@MicronautTest
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class MyTest implements TestPropertyProvider {

  static final KafkaContainer MY_KAFKA = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:latest"));
  static { MY_KAFKA.start(); }

  @Override
  public Map<String, String> getProperties() {
    return Map.of("kafka.bootstrap.servers", MY_KAFKA.getBootstrapServers());
  }

  @Test
  void testKafkaRunning(MyProducer producer) {
    producer.produce("hello");
  }

  @KafkaClient interface MyProducer {
    @Topic("my-topic")
    void produce(String message);
  }

  @KafkaListener static class MyConsumer {
    @Topic("my-topic")
    public void consume(String message) { }
  }
}

At the end of the test, the "main" thread will try to detect if the kafka consumer is closed:

    @Override
    @PreDestroy
    public void close() {
        for (ConsumerState consumerState : consumers.values()) {
            consumerState.kafkaConsumer.wakeup();
        }
        for (ConsumerState consumerState : consumers.values()) {
            while (consumerState.closedState == ConsumerCloseState.POLLING) {
                LOG.trace("consumer not closed yet");
            }
        }
        consumers.clear();
    }

Since consumerState.closedState was updated by another thread, the "main" thread will use a cached version of this value indefinitely (ie. "consumer not closed yet").

By declaring this field as volatile we will prevent the "main" thread from caching its value.

The lifecycle of this field is NOT_STARTED -> POLLING -> CLOSED. It's set when the state changes and it's only queried when the kafka consumer is destroyed, so the performance penalty should be negligible.

@guillermocalvo guillermocalvo mentioned this pull request Jul 20, 2023
@sdelamo sdelamo requested review from yawkat and removed request for jeremyg484 and yawkat July 20, 2023 16:05
@sdelamo sdelamo changed the base branch from master to 5.0.x July 20, 2023 16:05
@sdelamo sdelamo requested review from yawkat and removed request for sdelamo July 20, 2023 16:05
@sdelamo sdelamo added the type: bug Something isn't working label Jul 20, 2023
@sonarcloud
Copy link

sonarcloud bot commented Jul 21, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sdelamo
Copy link
Contributor

sdelamo commented Jul 27, 2023

@yawkat can you review this PR?

@sdelamo sdelamo merged commit 9641015 into 5.0.x Jul 28, 2023
7 checks passed
@sdelamo sdelamo deleted the 781-ci-never-ends-for-unit-tests-with-manual-test-container-lifecycle-control branch July 28, 2023 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

CI never ends for unit tests with manual test container lifecycle control
3 participants