Repository navigation
test: stabilize OPA timeout regression - #3848
MohammadHaroonAbuomar merged 1 commit into
Conversation
Signed-off-by: dpaul0501 <deb.jyoti93.paul@gmail.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Welcome to the Agent Governance Toolkit! Thanks for your first pull request. |
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
在本 fork 實測重現:已註冊的 server 用 http:// 被 entry.require_tls 擋下,但同一個 URL 換成未註冊的名字,落到 fallback 路徑就完全沒有 TLS 檢查、直接放行。也就是說 allowlist 的鍵打錯一個字,傳輸層下限就等於關掉——治理工具本身的護欄失效。 落地照上游:McpAuthPolicy 新增 default_require_tls=True(fail-closed)、fallback 分支補上與已註冊條目相同的 scheme 檢查(只認 https/wss)、from_yaml 讀該欄位。 只在真的有給 URL 時才擋,與已註冊條目一致,所以不誤擋從不傳 URL 的呼叫端。 測試四條(含上游沒有的「未註冊+不給 URL 仍放行」),21 passed。 同輪另三筆: - microsoft#3848(OPA timeout 測試 30ms 太緊)採用:假 opa sleep 30 秒,30ms 預算可能在 行程還沒 spawn 完就到期,失敗的是 spawn 不是逾時,測到的不是這條要測的東西。 本 fork 的 ci.yml 與 policy-engine-ci.yml 都跑 cargo test --workspace,opa.rs:254 就是同一行。驗證限制照實說:本機沒裝 cargo,這一行沒實跑過,靠 CI。 - microsoft#3846(土耳其文翻譯)不引用:本 fork 沒有 docs/i18n/,維護一份讀不懂也無法審校的 翻譯只會變成長期漂移的死文件。 - microsoft#3850(清理 python-app.yml 的 flake8)不引用:本 fork 沒有那支 workflow。 水位 PR 3845 → 3850。順帶把 test_fork_overlay 釘字面日期與字面水位的斷言改成釘性質 (第五個 repo 出現同一條)。 驗證:compileall / ruff / fork tests / 連結檢查 / mcp_auth 測試全部 exit 0。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Imran Siddique (imran-siddique)
left a comment
There was a problem hiding this comment.
Read the test rather than the number, and the diagnosis in your PR body is exactly right. Worth
spelling out on the thread, because a one-line timeout bump looks like the usual "make the flake go
away" change and this one is not.
The fake OPA script is:
echo $$ > "$ACS_FAKE_OPA_PID_FILE"
exec sleep 30and the assertion that actually matters is the last one, that
/proc/<pid> no longer exists after the timeout. That assertion needs the PID file to have been
written. With a 30 ms budget the runner can reach its timeout and kill the process group before
/bin/sh has spawned, written the file and reached exec, at which point
fs::read_to_string(&pid_file).unwrap() panics on a missing file. So the failure was never in the
timeout behaviour being tested, it was the harness racing its own fixture, which is why it presents
as a mysterious flake on slower machines and never on a warm CI runner.
500 ms is the right value for the same reason. It is far enough above shell startup to make the race
disappear and still two orders of magnitude below the fixture's sleep 30, so
error.detail().contains("OPA eval exceeded timeout") still asserts a real timeout rather than a
completed run. Nothing about what the test proves changes.
Nothing blocking. The cost is up to half a second on one #[cfg(unix)] test, which is a good trade
for an assertion that is currently unreliable.
|
Thanks, that framing is exactly what I was trying to preserve here. To make the intent explicit: the timeout value is not the behavior under test. The behavior under test is that an OPA child process which has entered the fake long-running command is killed after the evaluator timeout, and the final assertion verifies that The old 30 ms value let the harness race its own fixture: on a slower host, the timeout could fire before The 500 ms budget gives the fixture enough time to publish the PID deterministically, while remaining far below the fake command's |
MohammadHaroonAbuomar
left a comment
There was a problem hiding this comment.
Approved after group integration review (tests, gates and adversarial pass on the combined change).
Signed-off-by: dpaul0501 <deb.jyoti93.paul@gmail.com>
Signed-off-by: dpaul0501 <deb.jyoti93.paul@gmail.com> Signed-off-by: yuvrajsingh2428 <offcyuvi2428@gmail.com>
Summary
Repro
Before this change, running the core package tests on this machine failed in
opa_timeout_kills_child_process_without_lingeringbecausecore/tests/opa.rsattempted to read a PID file that had not been written yet.Test
cargo test -p agent_control_specification_core --test opa --quietcargo test -p agent_control_specification_core --quietSigned-off-by: Debjyoti Paul debjyotipaul@users.noreply.github.com