You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[CI] alpha.211 installer and upgrade blocked by hermes-agent CrashLoopBackOff #791
Cloud Installer And Upgrade Test Ref:v2.3.0-alpha.211 run 30972437222 failed all three GitHub-hosted matrices:
upgrade: job 92199515843
fresh K3s v1.33: job 92199515848
fresh K3s v1.22: job 92199515856
In each matrix, the surrounding KB Cloud pods become ready while hermes-agent alternates between RunContainerError and CrashLoopBackOff until the bounded readiness wait fails. Fresh v1.22 reaches at least 7 restarts; upgrade reaches at least 8.
The workflow then runs kubectl logs, but the captured hermes-agent log is empty. It does not preserve the container state reason/message, exit code, command/args, image digest, events, or previous-container log, so the first actionable runtime error is currently lost.
This is scoped to the alpha.211 product input: alpha.209 run 30890210350 and alpha.210 run 30895858238 both succeeded, and alpha.210 used the same apecloud-cd workflow head 69779e56bdf56764e941e1311362f5e0f6c7475e. It is not a self-hosted runner or workflow-code regression signal.
Closeout
Resolve and record the exact alpha.211 hermes-agent image tag and digest before install/upgrade.
On RunContainerError or CrashLoopBackOff, capture sanitized kubectl describe pod, container current/last state reason and message, exit code, command/args, image ID, events, and both current and --previous logs.
Fix the image entrypoint/runtime/configuration defect identified by that first error; do not hide it by extending the readiness timeout.
Validate the corrected immutable image through both fresh matrices and the upgrade matrix.
Keep alpha.210 as a known-good comparison and do not silently fall back to it.
Patrol did not rerun/cancel the workflow or modify images, charts, workflow, secrets, clusters, release state, or production.
Patrol update for the next release inputs on the same workflow head 69779e56bdf56764e941e1311362f5e0f6c7475e:
alpha.212 run 30981983923 reproduced the same failure in all three independent GitHub-hosted matrices: fresh K3s v1.22 job 92228211175, fresh K3s v1.33 job 92228211142, and upgrade job 92228211155. In each case, hermes-agent remained in RunContainerError / CrashLoopBackOff until the bounded readiness check failed; the captured container log was empty.
alpha.213 run 30984159719 then completed naturally with all three matrices green: v1.22 job 92235033293, v1.33 job 92235033386, and upgrade job 92235033448.
This narrows the product-input regression window to alpha.211-alpha.212 and provides a green successor at alpha.213. It does not close the workflow diagnostic gap: the failed runs still did not preserve the kubelet current/last-state message, exit code, image digest, events, or previous-container log, so the exact alpha.211/212 container-start cause cannot be recovered from Actions evidence alone.
No rerun/cancel, image/chart/workflow mutation, or release action was performed by patrol.
Activity
khalildai commentedon Aug 5, 2026
Patrol update for the next release inputs on the same workflow head
69779e56bdf56764e941e1311362f5e0f6c7475e:30981983923reproduced the same failure in all three independent GitHub-hosted matrices: fresh K3s v1.22 job92228211175, fresh K3s v1.33 job92228211142, and upgrade job92228211155. In each case,hermes-agentremained inRunContainerError/CrashLoopBackOffuntil the bounded readiness check failed; the captured container log was empty.30984159719then completed naturally with all three matrices green: v1.22 job92235033293, v1.33 job92235033386, and upgrade job92235033448.This narrows the product-input regression window to alpha.211-alpha.212 and provides a green successor at alpha.213. It does not close the workflow diagnostic gap: the failed runs still did not preserve the kubelet current/last-state message, exit code, image digest, events, or previous-container log, so the exact alpha.211/212 container-start cause cannot be recovered from Actions evidence alone.
No rerun/cancel, image/chart/workflow mutation, or release action was performed by patrol.