This repository has been archived by the owner on Sep 17, 2024. It is now read-only.
This repository has been archived by the owner on Sep 17, 2024. It is now read-only.
Audit test pipelines #3053
Open
Description
openedon Sep 28, 2022
This is a master tracking issue issue for auditing which E2E test pipelines need to remain enabled.
Beats CI pipelines
| Pipeline | Main Health | Triggers | Stakeholders | Issue(s) | Removal planned |
|---|---|---|---|---|---|
| Docker images | ⭕ Stale | None | Robots [@cachedout and @kuisathaverat ] | ❌ | |
| Fleet E2E | 🔴 Broken | Daily build | Fleet [@joshdover] | elastic/elastic-agent#1174 | |
| Observability Helm Charts | 🟢 Healthy | Daily build | Robots [@cachedout and @kuisathaverat ] | Issue located in private repo | ✅ |
| K8S Autodiscover | 🟡 Flakey | Daily build | Cloud Native Monitoring [@gizas] | ||
| Observability MacOS | 🔴 Broken | Daily build | Elastic Agent [@cmacknz and @jlind23 ] | https://github.com/elastic/ci/issues/705 | |
| Fleet Server | ⭕ Stale | None | Fleet [@joshdover ] | elastic/fleet-server#1927 | |
| Fleet UI | ⭕ Stale | None | Fleet and Integrations [@kpollich ] |
Fleet CI pipelines
| Pipeline | Main Health | Triggers | Stakeholders | Issue |
|---|---|---|---|---|
| Pipeline helper | 🔴 Broken | Push to main; PR labeled | Elastic Agent[@cmacknz and @jlind23 ] | elastic/elastic-agent#1174 |
- Should the pipeline be removed from the CI or should it remain?
1.1 If the pipeline remains and is broken, what is the link to an issue tracking a fix?
1.2 If the pipeline should remain, how is it monitored by the team to ensure that build artifacts are not produced when the tests fail?
Next steps
Proposed pipeline criteria
I am proposing that we remove all pipelines which do not meet any of the following criteria:
- Necessary for the ongoing health of the E2E test suite itself
- Used by a product team as a quality gateway. Concretely, this means that a failing test blocks a PR from being merged or a build artifact from being produced.
- Exist to ensure the quality of a supported product.
Timeline
- All existing E2E pipelines have stakeholders assigned no later than: October 1, 2022
- All stakeholder agree upon proposed pipeline criteria no later than: October 20, 2022
- Non-confirming pipelines will be removed from Jenkins and code will be removed from the E2E test suite beginning on: Nov 1st, 2022
Related efforts
There is a separate effort to try and reduce the scope of E2E testing back to a point where stability can be maintained, but it is limited to tests for the Agent. That effort can be found here: elastic/elastic-agent#1174
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Assignees
Labels
No labels