OSD-30030: E2E Test cases ClusterMonitoringErrorBudgetBurnSRE

[lambasanchit]

E2E Test: ClusterMonitoringErrorBudgetBurn Alert Trigger and Recovery (OSD-30030)

Description:
This PR adds an E2E test for the ClusterMonitoringErrorBudgetBurn alert, targeting AWS CCS clusters.

The test misconfigures the user-workload-monitoring-config ConfigMap in the openshift-user-workload-monitoring namespace to simulate excessive monitoring error budget burn. It then checks if a service log is created and finally deletes the ConfigMap to clean up the test state.

Steps:
Fetch initial cluster info and current service logs.

Backup the original ConfigMap.

Inject malformed YAML to trigger the alert.

Wait for CAD/system reaction.

Validate that a new service log is generated.

Delete the ConfigMap as a recovery step.

Acceptance:
Alert is triggered (ClusterMonitoringErrorBudgetBurnSRE).

Service log is sent to the customer.

Cluster state is restored by deleting the ConfigMap.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: lambasanchit
Once this PR has been reviewed and has the lgtm label, please assign rafael-azevedo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@lambasanchit: This pull request references OSD-30030 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

E2E Test: ClusterMonitoringErrorBudgetBurn Alert Trigger and Recovery (OSD-30030)

Description:
This PR adds an E2E test for the ClusterMonitoringErrorBudgetBurn alert, targeting AWS CCS clusters.

The test misconfigures the user-workload-monitoring-config ConfigMap in the openshift-user-workload-monitoring namespace to simulate excessive monitoring error budget burn. It then checks if a service log is created and finally deletes the ConfigMap to clean up the test state.

Steps:
Fetch initial cluster info and current service logs.

Backup the original ConfigMap.

Inject malformed YAML to trigger the alert.

Wait for CAD/system reaction.

Validate that a new service log is generated.

Delete the ConfigMap as a recovery step.

Acceptance:
Alert is triggered (ClusterMonitoringErrorBudgetBurnSRE).

Service log is sent to the customer.

Cluster state is restored by deleting the ConfigMap.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 32.08%. Comparing base (b1691b0) to head (0730c69).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #456   +/-   ##
=======================================
  Coverage   32.08%   32.08%           
=======================================
  Files          35       35           
  Lines        2425     2425           
=======================================
  Hits          778      778           
  Misses       1587     1587           
  Partials       60       60           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[openshift-ci]

@lambasanchit: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.