Skip to content

False negatives in CI marked as "Flakes" while running PR tests #44422

Open

Description

Describe the bug

Hello,

While working on #44348 it has been bewildering to me that the CI was all green while locally it was broken.
The way the PR was done, it could not have passed, it was missing Brotli library loading, yet the CI was green 🟢

The reason seems to be that Errors are marked as "Flakes", because 2nd run PASSED:

2024-11-09T14:15:53.4363212Z [ERROR]   Run 1: RESTEndpointsIT>RESTEndpointsTest.testDecompressors:120 » UnsatisfiedLink 'java.nio.ByteBuffer com.aayushatharva.brotli4j.encoder.EncoderJNI.nativeCreate(long[])'
2024-11-09T14:15:53.4365271Z [INFO]    Run 2: PASS

Full: https://karms.biz/pastebin/errors-labeled-as-flakes.txt

It seems that the way Re-Run is configured, it recycles the whole context and the second time around, the Brotli4jLoader got called?

It might hide, as this case shows, important issues with integration tests. Can we take a step back and rather fix tests than to enable this re-run feature en mass?

How about we enable re-runs only for a subset of tests that are notorious for race conditions, e.g. some Mongo or DB or messaging scenarios.

Expected behavior

No response

Actual behavior

No response

How to Reproduce?

No response

Output of uname -a or ver

Linux amd64

Output of java -version

21

Quarkus version or git rev

main

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

area/housekeepingIssue type for generalized tasks not related to bugs or enhancementskind/bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions