Skip to content

Migrate test setup/teardown to fixtures #413

Closed
@zbeekman

Description

@zbeekman

Right now, the tests are somewhat stateful: Tests modify inputs/outputs of other tests and/or don't clean up after themselves properly. This can cause problems when JSON-Fortran is added as an ExternalProject in a CMake "SuperBuild" with testing enabled.

What happens is that subsequent runs of the build will see that JSON-Fortran is up to date, skipping the configure and build steps, but will run the test step. (Or, perhaps it will re-run the configure step I'm not 100% clear on this, but I think my initial understanding is correct.) Because the test inputs/outputs are stateful, the initial run after configuring and building will pass, but subsequent runs will fail.

To resolve this the following steps should be taken:

  1. Analyze inter-test run order dependencies. CTest allows tests to be run in parallel, and some tests may be taking the output of others as inputs. This should be explicitly declared.
  2. CMake/CTest has added support for "test fixtures" which allow setup/teardown tasks for groups of tests to be declared. Use this to copy inputs (and cleanup outputs) between test runs.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions