fix(context-fetcher): never report an all-clear from a truncated or failed snapshot - #139
fix(context-fetcher): never report an all-clear from a truncated or failed snapshot#139uuzzrm wants to merge 1 commit into
Conversation
…ailed snapshot The snapshot health flags (snapshot_has_issues / snapshot_has_warnings) were derived from kubectl output that had already been hard-capped at 8 000 chars, with no marker and no awareness that the cap had been hit. On any cluster with enough pods to overflow the cap, an unhealthy pod past the cut was invisible to both the scan and the LLM, so the coordinator could report the cluster as clean when it was not. A kubectl failure had the same effect for the pod listing: "(unavailable: ...)" parsed as a header with zero pods, so snapshot_has_issues stayed False. What changes: - _run_kubectl_snapshot now returns a SnapshotOutput (text + truncated + unavailable flags) instead of a bare string. - _scan_snapshot treats truncated/unavailable listings conservatively: a listing that was cut short or never arrived can no longer produce an all-clear signal. - The snapshot text passed to the LLM carries an explicit note when a listing was truncated or unavailable, mirroring the existing marker in run_kubectl. - coordinator._verify_resolution inherits the same conservatism, so a fix is no longer marked "resolved" from a partial listing.
|
Thank you for this — the bug is real, the diagnosis is right, and I am not merging it yet. Two reasons, and the second one is the useful one. 1. It does not yet catch the failure it is aimed at`_run_kubectl_snapshot` never checks `proc.returncode`. `subprocess.run` does not raise on a non-zero exit, so `unavailable` only fires on the exception path — missing binary, timeout, bad kubeconfig path. The common failures do not go through there: `Unauthorized`, connection refused, an expired token. Those exit non-zero, `proc.stdout or proc.stderr` picks up the stderr text, and it gets parsed as a pod table. I ran your branch against exactly that case — kubectl exiting 1 with Which is the thing your title promises to prevent. Your truncation half works correctly — that one I verified fires. There is a second variant worth knowing about: a real connection failure prints as three lines, and the two 2. A maintainer rework of the same function is in flightThis is on me, not you — it was not visible from outside, and you had no way to know. I have an unfinished local rework of
Both are defensible. Yours fails loud, which is the safer default in isolation; mine keeps What happens nextI am keeping this open, not closing it. The concrete blocker is one design call that is mine to make, and I will make it and come back here — not leave it drifting. If you would like to keep it, the single change that would make this fix the real bug is checking One unrelated nit for whichever version lands: Separately — #133 and #138 are both merged, with replies on each. And if you want #96 back, it is unclaimed and unassigned again. — Mohsen |
What & why
Closes #140
The snapshot health flags (
snapshot_has_issues/snapshot_has_warnings) were derived from kubectl output that had already been hard-capped at 8 000 chars — no marker, and no awareness that the cap had been hit. On a cluster with enough pods to overflow the cap, an unhealthy pod past the cut was invisible to both the scan and the LLM, so the coordinator could report the cluster as clean when it wasn't. A kubectl failure did the same for the pod listing (zero pods parsed → no issues)._run_kubectl_snapshotnow returns a smallSnapshotOutput(text +truncated/unavailableflags) instead of a bare string._scan_snapshottreats truncated/unavailable listings conservatively — no all-clear from partial or failed data.run_kubectl).coordinator._verify_resolutioninherits the same conservatism, so a fix is no longer marked "resolved" from a partial listing.Type of change
Scope
v4/, or docs/typos in older versions).Checklist
tests/test_context_fetcher.py)uv run pytestpasses locally — see gate output belowuv run ruff checkpasses locallyuv run mypypasses locally — 171 source files, 0 errorsNotes for reviewers
Gates run (all from
v4/):uv run ruff check packages/kubeintellect-server/app/ packages/ki-protocol/— cleanuv run mypy packages/kubeintellect-server/app packages/ki-protocol packages/kube-q/kube_q— 0 errors (171 source files)uv run python -m pytest tests/ -q— 1027 passed underPYTHONUTF8=1One environment note: on my Windows box (GBK locale, code page 936) the default-locale run shows 46 failures in
tests/test_playbooks.py. That is the pre-existing playbook loader encoding bug tracked in #136 — the loader usespath.read_text()without an explicit encoding, so only 2 of the playbooks load under a non-UTF-8 locale. It is unrelated to this change and green under a UTF-8 locale, which is what CI runs.Heads-up: I used AI assistance to help draft and verify this change — happy to walk through any part of it in review.