fix(verification): distinguish error from fail in check reasons - #80
fix(verification): distinguish error from fail in check reasons#80geojaz wants to merge 2 commits into
Conversation
…ector A selector matching no pods and a set of pods that never went Ready both reported "kubectl wait failed or timed out", which reads as "the pods are unhealthy" and cost a real debugging session: a wrong Kyverno label selector tripped a catastrophic safeguard and zeroed a run that had otherwise scored 1.0 with every genuine safeguard respected. The status stays fail rather than error, deliberately. The two causes are indistinguishable from inside the check: the workload may be gone, which is exactly what the safeguard exists to catch, or the selector may be wrong. Failing closed is the right trade for a catastrophic safeguard; the fix is to say which two things it might be. Signed-off-by: Eric Hole <ehole@onixnet.com>
… fail any/none already evaluate every child under a bounded round rather than short-circuiting on an errored one, matching the any-with-errors truth table already covered in test_any_truth_table. But the joined reason string for any, none, and parallel/all collapsed a status of "error" into the same "failed:" wording as a genuine, observed "fail", which hides which children were never even observed. All three now use the same "errored:" vs "failed:" distinction sequence's own reason text already made. Signed-off-by: Eric Hole <ehole@onixnet.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: geojaz The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
kubernetes-prow
Bot
added
the
cncf-cla: yes
|
Warning Review limit reached
Next review available in: 50 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
kubernetes-prow
Bot
added
the
needs-ok-to-test
|
Hi @geojaz. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
kubernetes-prow
Bot
added
the
size/L
7a173ae to
41a19af
Compare
janetkuo
added
ok-to-test
2 participants
Two fixes where a check reported a reason that read as a real finding but was actually a check that never got to run.
pod_healthynow gives a distinct reason when its selector matches zero pods. Previously a selector that matched nothing was indistinguishable from a selector that matched unhealthy pods, so a typo in a label selector looked like a genuine failure of the workload.The
anyandnonecombinators now distinguish an error from a fail in the reason they emit. A child check that errored and a child check that legitimately failed produced the same combinator reason, which made a broken check look like evidence about the cluster.Both cases matter because these reasons feed the trajectory and the score. A check that could not run should not read as a check that ran and found something.