Acceptance tests do not stop Besu subprocesses causing resource exhaustion #7596
Description
Description
When running ATs that spawn Besu subprocesses, these subprocesses are not correctly stopped and the linger on, eventually causing resource exhaustion on the system before the tests end.
See this video for an example of what is happening: https://github.com/user-attachments/assets/19ed4ac9-50d8-4fd0-960b-3d510c3cbbb2
you can see that the number of Java Besu processes spawn is always increasing as the tests go, while they should be closed as soon as a test is done.
This is at least present on Linux and Windows, but could depends also on system configuration, because the current way to stop the Besu subprocess is to kill (TERM) the PID related to the start script, but this does not cause the kill (TERM) of the children of that script, including the Java Besu process.
A solution for this problem was implemented in this PR #6820, but it was reverted since it made the ATs take longer on CI, this probably because without the fix, there was already a Besu up and running and able to response to the requests, while with the PR we need to wait for the Besu process to stop and restart, and that adds a delay.
Ideas on how to remove the delay?
- Force kill the subprocesses, without waiting for the Besu stop, should not be a problem because these Besus are single use
- Refactors ATs to avoid having to stop and start Besu every time, this requires grouping tests that can run with the same configuration, if 2 tests require configurations that are incompatible they cannot be grouped
- other ideas?
Acceptance Criteria
Running ATs does not consume all the system resources, because subprocesses are closed when not needed
Steps to Reproduce (Bug)
./gradlew --max-workers=1 acceptanceTestNoPrivacy
Versions (Add all that apply)
- Software version: 24.9
- Java version: Java 17
- OS Name & Version: Ubuntu 22.04.4 LTS
Activity