Slow tests

[dennisvang]

Tests are incredibly slow.

A few simple "acceptance" tests added in #853 increased the total time by another couple of minutes, causing the slowest runners (windows) to take up to 25 minutes.

Something needs to be done about this, although I'm afraid it will require some serious refactoring.

Also see

• Reset entire test db before each test #841
• Clean up test annotations #816

Discussed in #598

^{Originally posted by dennisvang January 20, 2025}
@MarekSuchanek looking at a typical test run on github actions, we can see that the FDP test suite takes >10 minutes on posix, and >15 minutes on windows. On my local windows system, it can take up to 25 minutes.

My experience with java/spring/junit is limited, so I don't know if this is normal, but I've seen similar sized projects, with comparable test suites in e.g. python/django staying under five minutes.

Do you know if this can be improved?

Perhaps we could separate unit tests from integration tests?

[dennisvang]

Here's an example of a flame graph showing that the SHACL validation during RDF migrations executed by RdfDevelopmentMigrationRunner is taking up a lot of time:

Specifically the connection.commit() calls are slow:

FAIRDataPoint/src/main/java/org/fairdatateam/fairdatapoint/service/rdf/ShaclValidator.java

Line 55 in 67ec393

connection.commit();

Perhaps standalone validation would be an option here?

For non-transactional or one-off validation you can use the standalone ShaclValidator API. It lets you load shapes and data from common inputs without creating a ShaclSail, while still giving access to the same validation options (parallel validation, logging, RDFS reasoning, extensions, and limits).

Also see rdf4j shacl performance section.

[dennisvang]

Another problem is in the WebIntegrationTest class, which is used as the parent for most of the "acceptance" tests:

FAIRDataPoint/src/test/java/org/fairdatateam/fairdatapoint/WebIntegrationTest.java

Lines 38 to 39 in bfb5d49

	@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
	@AutoConfigureMockMvc

As explained in the spring boot testing docs, This enables a full web server and network setup via the SpringBootTest.WebEnvironment.DEFINED_PORT, which is typically used with TestRestTemplate, and consequently is relatively slow. The effect can be seen in the logs, which show threads like [http-nio-auto-1-exec-3].

In addition, this uses @AutoConfigureMockMvc, which is intended for testing without a full web server setup, using MockMvc (or MockMvcTester). However, a search for MockMvc in the codebase comes up empty, showing that it is not used anywhere (!).

In my opinion, using a full http web server environment for the acceptance tests, with TestRestTemplate, is not necessary at all. MockMvc could be used in most cases, and would probably speed up tests considerably.