Closed
Description
Our CI is getting slow and sometimes unstable.
This is mostly related to integrations tests that:
- call unstable external services (e.g., Hugging Face)
- perform model inference using CPU
A possible approach
While in the long run we could think about moving some components to core integrations, we can put a mitigation strategy in place.
- Mark slow/unstable tests with a specific marker
- Create a workflow to run these tests, nightly and on PRs that touch specific files
- Include this workflow in branch protection rules to block merging if it fails (related: create a skipper workflow)
- bonus: if we carefully select tests, we might avoid running Tika and installing system dependencies for Whisper during ordinary integration tests; these 2 steps make integration tests considerably slower