Tests run in parallel?

[jvdp1]

I am buzy to move some tests for using test-drive in one of my personal project.
The library, as well as the tests are compiled with the OpenMP and paralle options. This leads to quite random outputs because it seems that the new_unittests are run in parallel.

E.g.:

# Testing: modtest_example
  Starting dummy ... (1/3)
  Starting test_info ... (2/3)
       ... dummy [PASSED]
  Starting test_ex1 ... (3/3)
       ... test_info [PASSED]
       ... test_ex1 [PASSED]
All tests passed!

Per se, it is not a problem, if the tests can run independently (it might not be always the case, e.g., it 2 tests call a procedure relying on a module 's global variable). It is mainly the output that is difficult to read.

Or did I miss something?

[awvwgk]

I think parallel CTest runs work similar in the printout, you get output once the test starts and once it finishes. At least that was my reference for implementing the printout. Now seeing something like the new meson test printout (since 0.59), having it in full color and always in deterministic order seems like a much better representation for the printout.

For the parallel runs I'm currently assuming all tests are independent and can run in any order, i.e. there is no global state. If tests access shared resources you could add a named !$omp critical (...) section to create a mutex.

[jvdp1]

For the parallel runs I'm currently assuming all tests are independent and can run in any order, i.e. there is no global state. If tests access shared resources you could add a named !$omp critical (...) section to create a mutex.

Thanks for your quick answer.
Actually the parallelism appears when creating the array testsuite as testsuite = [new_unittest(...), new_unittest(...), ... ]. So I am not sure how to use !$omp critical(...) in such a case, e.g.,:

....
!> Collect all exported unit tests
subroutine collect_snpblup(testsuite)
  !> Collection of tests
  type(unittest_type), allocatable, intent(out) :: testsuite(:)

  testsuite = [ &
    new_unittest("dummy", dummy) &
    , new_unittest("dummy1", dummy1) &
    ]

end subroutine

subroutine dummy(error)
 type(error_type), allocatable, intent(out) :: error
 
 print*,burn_cpu(huge(1))

 call check(error, .true., 'dummy  logical')
 if(allocated(error))return

end subroutine

subroutine dummy1(error)
 type(error_type), allocatable, intent(out) :: error

 print*,burn_cpu(huge(1)/2)

 call check(error, .true., 'dummy1  logical')
 if(allocated(error))return

end subroutine
...

[jvdp1]

Compiling only testdrive.F90 without openmp and parallel options is also a solution (the most appropriate for the moment probably?).

I need to look further in the testdrive code to better understand it.

[awvwgk]

Maybe it would be better if test-drive could offer different drivers for running testsuites. Additionally, the unit tests could get more attributes which allow the general driver to decide whether the suite can be run in parallel or the user can manually decide which driver to use for all suites.

[jvdp1]

Solved by #11