Skip to content

CI: Run slow/unstable tests nightly #9266

Closed
@anakin87

Description

@anakin87

Our CI is getting slow and sometimes unstable.

This is mostly related to integrations tests that:

  • call unstable external services (e.g., Hugging Face)
  • perform model inference using CPU

A possible approach

While in the long run we could think about moving some components to core integrations, we can put a mitigation strategy in place.

  • Mark slow/unstable tests with a specific marker
  • Create a workflow to run these tests, nightly and on PRs that touch specific files
  • Include this workflow in branch protection rules to block merging if it fails (related: create a skipper workflow)
  • bonus: if we carefully select tests, we might avoid running Tika and installing system dependencies for Whisper during ordinary integration tests; these 2 steps make integration tests considerably slower

Metadata

Metadata

Assignees

Labels

P2Medium priority, add to the next sprint if no P1 availabletopic:CItopic:tests

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions