Closed
Description
Right now, the tests are somewhat stateful: Tests modify inputs/outputs of other tests and/or don't clean up after themselves properly. This can cause problems when JSON-Fortran is added as an ExternalProject
in a CMake "SuperBuild" with testing enabled.
What happens is that subsequent runs of the build will see that JSON-Fortran is up to date, skipping the configure and build steps, but will run the test step. (Or, perhaps it will re-run the configure step I'm not 100% clear on this, but I think my initial understanding is correct.) Because the test inputs/outputs are stateful, the initial run after configuring and building will pass, but subsequent runs will fail.
To resolve this the following steps should be taken:
- Analyze inter-test run order dependencies. CTest allows tests to be run in parallel, and some tests may be taking the output of others as inputs. This should be explicitly declared.
- CMake/CTest has added support for "test fixtures" which allow setup/teardown tasks for groups of tests to be declared. Use this to copy inputs (and cleanup outputs) between test runs.