feat: NVIDIA NIM model discovery (issue #86 scaffold) - #115
Conversation
Add discover_nim_models + agent-pool mapping for issue #86, reading NVIDIA_NIM_API_KEY from the KV (not COPILOT_GITHUB_TOKEN). Offline fixtures keep CI hermetic; optional live path seeds KV from bootstrap env when present.
📝 WalkthroughWalkthroughNIM 모델 카탈로그 검색, capability inventory와 probe, benchmark dry-run, 오프라인 비용 품질 평가 CLI를 추가했다. 오케스트레이션 정책에 역할별 sampling temperature를 추가했다. 테스트, 문서, 예제, 변경 로그와 Semgrep 주석을 갱신했다. ChangesNIM 평가 및 오케스트레이션
Estimated code review effort: 4 (Complex) | ~45 minutes Mergeability Score: 🟠 High · up to This scaffold adds NIM discovery and offline cost-quality behavior, but the current head still has unresolved input-validation, capability-reporting, and external-provider security issues; required checks are not yet passing and the implementation is explicitly superseded, so it is not merge-ready and should remain unmerged. Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Merge order note: land after or with #114 if agent pool entries should include |
Pull request was closed
|
Closing unmerged after live-state revalidation. Exact contributor head |
Pull request was closed
|
Do not close: active product work for issue #86 (NIM discovery + CLI + role temperature). Autonomous commercial loop will re-open if closed without merge. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
contextual_orchestrator/orchestrator.py (1)
332-336: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift보호된 provider egress를 transport 경계에서 강제하십시오.
_validate_provider()는 검증 시점의 DNS 결과만 검사하고,_open_provider()는 hostname을urllib.request.urlopen()에 다시 전달합니다. DNS 재바인딩으로 private/reserved 주소에 연결될 수 있습니다.CONTEXTUAL_ORCHESTRATOR_ALLOWED_PROVIDER_HOSTS가 비어 있으면 allowlist도 적용되지 않습니다.기본
HTTPRedirectHandler는 POST의 301/302/303 redirect를 따르고Authorization헤더를 새 요청에 복사합니다. Redirect 대상의 host와 DNS 결과를 재검증하지 않으므로 자격 증명이 검증되지 않은 대상에 전송될 수 있습니다.비모의 provider에는 필수 allowlist를 적용하고, 검증된 주소를 transport에서 고정하거나 trusted egress proxy를 사용하십시오. Redirect를 비활성화하거나 각
Location을 재검증하고 다른 origin에는Authorization을 제거하십시오.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@contextual_orchestrator/orchestrator.py` around lines 332 - 336, Update _open_provider and its opener/transport flow to enforce provider egress at connection time, not only through _validate_provider’s initial DNS lookup: require a non-empty CONTEXTUAL_ORCHESTRATOR_ALLOWED_PROVIDER_HOSTS allowlist for non-mock providers and connect using the validated addresses or a trusted egress proxy. Disable automatic redirects or revalidate every Location against host and DNS restrictions, removing Authorization when the redirect changes origin; preserve existing timeout and SSL behavior.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Around line 11-12: Remove both changelog entries for the superseded PR: delete
the discover-nim-models/nim_discovery issue `#86` entry and the
role-differentiated sampling temperatures entry, leaving these changes to be
documented through the active PR `#90` if appropriate.
In `@contextual_orchestrator/cost_ledger.py`:
- Line 625: Replace the suppression-based SQL handling at
contextual_orchestrator/cost_ledger.py:625-625 so Ruff S608 is resolved through
a statically verifiable query construction, and apply the same security
validation to the SELECT path at contextual_orchestrator/cost_ledger.py:586-586
and INSERT path at contextual_orchestrator/cost_ledger.py:605-605; if
suppressions remain, verify their exact accepted format across all scanners.
Retain CHANGELOG.md:14-15’s Security entry only if the implementation includes
an actual security remediation; otherwise remove it.
In `@contextual_orchestrator/nim_discovery.py`:
- Around line 77-93: The NIM discovery result currently labels test fixture
transport data as live catalog data. In contextual_orchestrator/nim_discovery.py
lines 77-93, update the result-building logic around the transport path to
return an explicit offline_fixture source status for fixture calls, or accept
and apply an explicit source-status argument while preserving live transport
support; update tests/test_nim_discovery.py lines 51-71 so the fixture
invocation expects offline_fixture.
- Around line 108-119: Update the agent-entry construction around _slug_model_id
and the 48-character truncation so every model receives a unique deterministic
"id". Track previously generated IDs and append a stable suffix when slug or
truncation collisions occur, while preserving the existing ID format for
non-colliding models; add tests covering both normalized-slug collisions and
long-model truncation collisions.
- Around line 68-84: Update the model-discovery flow around models_url and the
urllib.request.Request call so NVIDIA_NIM_API_KEY is never sent to a
user-controlled URL: remove the configurable --models-url path or replace it
with a fixed NVIDIA origin, and validate HTTPS, allowed host, port, and
/v1/models before constructing the authenticated request. Remove the nosec
suppression rather than relying on it, while preserving the existing transport
and response-reading behavior.
In `@contextual_orchestrator/orchestrator.py`:
- Around line 172-174: Remove the role-specific temperature behavior introduced
through role_temperature and default_role_temperature, preserving
ModelClient.chat’s existing default temperature of 0.2 for normal route/conduct
responses. If the experiment must remain, isolate it behind an explicit opt-in
policy so OrchestrationPolicy() defaults retain existing behavior.
- Around line 185-207: Normalize role temperatures once before serialization or
request execution, validating that values are finite numbers and keys are
supported roles. Update as_dict() and temperature_for_role() to consume the same
normalized role-temperature table, preserving default values for invalid or
unsupported configuration entries so policy_snapshot matches _invoke() exactly.
- Around line 1573-1577: 모든 provider 호출 경로에서 역할 기반 온도를 사용하도록 수정하십시오.
stream_route의 stream_chat(), batch_route의 batch_chat(), _plan_generated()와
_model_judge_verification()의 client.chat() 호출에 해당 역할의
policy.temperature_for_role(role) 결과를 명시적으로 전달하고, 기존 _invoke() 동작은 유지하십시오. 각 경로가
기본값 0.2 대신 계산된 온도를 전달하는지 검증하는 테스트를 추가하십시오.
- Line 255: Update the --serve request path to reject verify_tls=False from
--insecure-skip-tls-verify before constructing ModelClient or making provider
requests. Preserve certificate and hostname verification for
Bearer-authenticated external providers, and require self-signed providers to
configure ca_bundle instead. Do not rely on the SAST suppression comment to
alter this behavior.
In `@tests/test_nim_discovery.py`:
- Around line 142-160: Update tests/test_nim_discovery.py lines 142-160 in
test_live_nim_catalog_when_env_seeded_into_kv to require an explicit opt-in such
as RUN_LIVE_NIM_TESTS=1 before reading NVIDIA_NIM_API_KEY or invoking
discover_nim_models; otherwise return without making a live request. Update
docs/architecture.md lines 59-64 to state that live NIM checks require explicit
opt-in and default CI uses only offline fixtures.
---
Outside diff comments:
In `@contextual_orchestrator/orchestrator.py`:
- Around line 332-336: Update _open_provider and its opener/transport flow to
enforce provider egress at connection time, not only through
_validate_provider’s initial DNS lookup: require a non-empty
CONTEXTUAL_ORCHESTRATOR_ALLOWED_PROVIDER_HOSTS allowlist for non-mock providers
and connect using the validated addresses or a trusted egress proxy. Disable
automatic redirects or revalidate every Location against host and DNS
restrictions, removing Authorization when the redirect changes origin; preserve
existing timeout and SSL behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: e0fd3f79-5f88-4eae-a556-56e95b881317
📒 Files selected for processing (7)
CHANGELOG.mdcontextual_orchestrator/__main__.pycontextual_orchestrator/cost_ledger.pycontextual_orchestrator/nim_discovery.pycontextual_orchestrator/orchestrator.pydocs/architecture.mdtests/test_nim_discovery.py
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Semgrep multi-language SAST flagged dynamic-urllib-use on the validated NIM catalog client; annotate Request/urlopen with the full rule id after HTTPS host allowlist validation.
Validate models catalog URLs against an NVIDIA HTTPS allowlist before attaching NVIDIA_NIM_API_KEY, label fixture transports as offline_fixture, dedupe agent ids on slug/truncation collision, and gate live NIM tests on RUN_LIVE_NIM_TESTS=1. Completes Semgrep suppressions for ledger executes.
Classify chat/embeddings/rate-limit/timeout/unsupported probe outcomes from offline fixtures for issue #86 without network or secrets.
There was a problem hiding this comment.
Actionable comments posted: 8
🧹 Nitpick comments (1)
tests/test_nim_discovery.py (1)
419-454: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winprobe 계획의 fail-closed 검증 분기에 테스트를 추가하십시오.
현재 테스트는 정상 경로와 예산 초과만 검증합니다.
build_capability_probe_plan의 나머지 거부 분기는 검증되지 않습니다. 다음 입력에 대한NimDiscoveryError테스트를 추가하십시오.
hard_request_budget=0probe_kinds=()probe_kinds=("",)
classify_probe_http_status가 지원하지 않는 상태 코드에 대해 예외를 발생시키는지도 검증하십시오.♻️ 제안 추가 테스트
with pytest.raises(NimDiscoveryError): run_capability_probes_dry_run(fixtures, hard_request_budget=1) + with pytest.raises(NimDiscoveryError): + build_capability_probe_plan(["a"], hard_request_budget=0) + with pytest.raises(NimDiscoveryError): + build_capability_probe_plan(["a"], probe_kinds=()) + with pytest.raises(NimDiscoveryError): + build_capability_probe_plan(["a"], probe_kinds=("",))🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_nim_discovery.py` around lines 419 - 454, Extend test_capability_probe_plan_and_dry_run_budget to assert NimDiscoveryError from build_capability_probe_plan for hard_request_budget=0, an empty probe_kinds tuple, and probe_kinds containing an empty string. Also import and test classify_probe_http_status with an unsupported HTTP status code, verifying it raises NimDiscoveryError.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Around line 14-19: CHANGELOG.md의 [Unreleased] 섹션에서 중복된 `### Added` 제목을 하나로
통합하십시오. 두 제목 아래의 모든 항목을 단일 `### Added` 블록에 유지하고, 다른 변경 유형 제목이나 항목은 수정하지 마십시오.
In `@contextual_orchestrator/__main__.py`:
- Around line 126-137: Validate the fixture JSON root in the
capability_probe_dry_run path before accessing dict methods: accept only list or
dict values, normalize supported dict keys as currently intended, and route
invalid structures through parser.error without allowing AttributeError. In
contextual_orchestrator/__main__.py lines 126-137, update the fixture handling
accordingly; in lines 239-250, wrap the load_agents call so KeyError is
converted to parser.error.
In `@contextual_orchestrator/nim_cost_quality.py`:
- Around line 312-334: Update direct_worker and route_once in the runner factory
to record that they share the same orchestrator.route_once(_messages(prompt))
execution path, using an explicit alias indicator in their returned
dictionaries. Preserve the existing mode values and bounded_conduct behavior so
report consumers can distinguish aliased results from independently executed
policies.
In `@contextual_orchestrator/nim_discovery.py`:
- Around line 477-484: Update the outcome mapping in the body-is-None branch to
use PROBE_OUTCOME_MALFORMED as the fallback for unknown kind values instead of
PROBE_OUTCOME_CHAT. Preserve the existing mappings for registered probe kinds
and keep behavior consistent with the malformed handling for dictionary bodies.
- Around line 576-594: Update the fixture-row validation around
classify_probe_result to require both model_id and probe_kind, rejecting missing
values with NimDiscoveryError that includes the row index. Remove the "chat"
fallback in both error_class and status_code branches, while preserving the
existing status_code validation and classification flow for valid rows.
- Around line 415-429: Update the status classification logic near the existing
200/4xx/5xx branches so every 2xx response other than 200 maps explicitly to
PROBE_OUTCOME_MALFORMED, and every 3xx response maps to PROBE_OUTCOME_FAILED.
Preserve the existing mappings for 200, 4xx, 5xx, and the unsupported-status
error.
In `@docs/architecture.md`:
- Around line 106-109: Update the outcome list in the architecture documentation
describing build_capability_probe_plan and run_capability_probes_dry_run to
include unavailable, malformed, image, audio, and video alongside the existing
outcomes, matching all values produced by
contextual_orchestrator/nim_discovery.py.
In `@tests/test_nim_cost_quality.py`:
- Around line 388-389: Remove the redundant `or True` condition from the
assertion over `report["cells"]` in the test, making it explicitly validate that
every cell has an `"outcome"` of `"success"` without filtering by `policy_name`.
---
Nitpick comments:
In `@tests/test_nim_discovery.py`:
- Around line 419-454: Extend test_capability_probe_plan_and_dry_run_budget to
assert NimDiscoveryError from build_capability_probe_plan for
hard_request_budget=0, an empty probe_kinds tuple, and probe_kinds containing an
empty string. Also import and test classify_probe_http_status with an
unsupported HTTP status code, verifying it raises NimDiscoveryError.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 7e386afe-0d2c-4177-b175-7f6fb7622751
📒 Files selected for processing (9)
CHANGELOG.mdREADME.mdcontextual_orchestrator/__main__.pycontextual_orchestrator/nim_cost_quality.pycontextual_orchestrator/nim_discovery.pydocs/architecture.mdexamples/nim_capability_probe_fixtures.jsontests/test_nim_cost_quality.pytests/test_nim_discovery.py
🚧 Files skipped from review as they are similar to previous changes (1)
- README.md
| ### Added | ||
| - Offline NIM capability probe plan + fixture classification (issue #86). | ||
| - Offline cost-quality `--use-mock-orchestrator` path: Fugu `route_once` and | ||
| Conductor/TRINITY `conduct` via `mock://` agents (issue #86 paper-path exercise). | ||
|
|
||
| ### Added |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
중복된 ### Added 제목을 하나로 병합하십시오.
[Unreleased] 섹션에 ### Added 제목이 line 14와 line 19에 두 번 나옵니다. Line 5-6은 이 파일이 Keep a Changelog 형식을 따른다고 선언합니다. 이 형식은 릴리스 섹션마다 변경 유형별로 하나의 제목을 사용합니다. 중복 제목은 변경 로그 파서와 릴리스 노트 생성을 혼란시킵니다.
두 블록의 항목을 하나의 ### Added 아래로 병합하십시오.
♻️ 제안 수정
### Added
- Offline NIM capability probe plan + fixture classification (issue `#86`).
- Offline cost-quality `--use-mock-orchestrator` path: Fugu `route_once` and
Conductor/TRINITY `conduct` via `mock://` agents (issue `#86` paper-path exercise).
-
-### Added
- Offline NIM cost-quality comparison harness (`nim_cost_quality` +📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ### Added | |
| - Offline NIM capability probe plan + fixture classification (issue #86). | |
| - Offline cost-quality `--use-mock-orchestrator` path: Fugu `route_once` and | |
| Conductor/TRINITY `conduct` via `mock://` agents (issue #86 paper-path exercise). | |
| ### Added | |
| ### Added | |
| - Offline NIM capability probe plan + fixture classification (issue #86). | |
| - Offline cost-quality `--use-mock-orchestrator` path: Fugu `route_once` and | |
| Conductor/TRINITY `conduct` via `mock://` agents (issue #86 paper-path exercise). |
🧰 Tools
🪛 markdownlint-cli2 (0.23.2)
[warning] 19-19: Multiple headings with the same content
(MD024, no-duplicate-heading)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@CHANGELOG.md` around lines 14 - 19, CHANGELOG.md의 [Unreleased] 섹션에서 중복된 `###
Added` 제목을 하나로 통합하십시오. 두 제목 아래의 모든 항목을 단일 `### Added` 블록에 유지하고, 다른 변경 유형 제목이나
항목은 수정하지 마십시오.
| if args.capability_probe_dry_run: | ||
| try: | ||
| with open(args.capability_probe_dry_run, encoding="utf-8") as handle: | ||
| fixtures = json.load(handle) | ||
| plan = run_capability_probes_dry_run( | ||
| fixtures if isinstance(fixtures, list) else fixtures.get("probe_rows") or fixtures.get("fixtures") or [], | ||
| hard_request_budget=args.hard_request_budget, | ||
| ) | ||
| except (NimDiscoveryError, OSError, ValueError, TypeError) as exc: | ||
| parser.error(str(exc)) | ||
| print(json.dumps(plan, ensure_ascii=False, indent=2)) | ||
| return |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
두 서브커맨드에서 운영자 입력 오류가 선언된 예외 계약을 벗어납니다. 각 명령은 입력 오류를 parser.error로 변환하는 계약을 선언합니다. 그러나 두 경로 모두 JSON 입력의 내부 구조를 검사하지 않으므로 except 목록에 없는 예외가 발생하고 CLI가 traceback으로 종료됩니다.
contextual_orchestrator/__main__.py#L126-L137: fixture JSON의 최상위 타입을 검사하십시오. list와 dict가 아닌 값에서.get호출로 발생하는AttributeError를 제거하십시오.contextual_orchestrator/__main__.py#L239-L250:load_agents호출을 감싸서KeyError를parser.error로 변환하십시오.
🧰 Tools
🪛 ast-grep (0.45.1)
[warning] 127-127: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(args.capability_probe_dry_run, encoding="utf-8")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[info] 135-135: use jsonify instead of json.dumps for JSON output
Context: json.dumps(plan, ensure_ascii=False, indent=2)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
📍 Affects 1 file
contextual_orchestrator/__main__.py#L126-L137(this comment)contextual_orchestrator/__main__.py#L239-L250
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@contextual_orchestrator/__main__.py` around lines 126 - 137, Validate the
fixture JSON root in the capability_probe_dry_run path before accessing dict
methods: accept only list or dict values, normalize supported dict keys as
currently intended, and route invalid structures through parser.error without
allowing AttributeError. In contextual_orchestrator/__main__.py lines 126-137,
update the fixture handling accordingly; in lines 239-250, wrap the load_agents
call so KeyError is converted to parser.error.
| def direct_worker(prompt: str) -> dict[str, Any]: | ||
| result = orchestrator.route_once(_messages(prompt)) | ||
| result = dict(result) | ||
| result["mode"] = "direct_worker" | ||
| return result | ||
|
|
||
| def route_once(prompt: str) -> dict[str, Any]: | ||
| result = orchestrator.route_once(_messages(prompt)) | ||
| result = dict(result) | ||
| result["mode"] = "route_once" | ||
| return result | ||
|
|
||
| def bounded_conduct(prompt: str) -> dict[str, Any]: | ||
| result = orchestrator.conduct(_messages(prompt)) | ||
| result = dict(result) | ||
| result["mode"] = "bounded_conduct" | ||
| return result | ||
|
|
||
| return { | ||
| "direct_worker": direct_worker, | ||
| "route_once": route_once, | ||
| "bounded_conduct": bounded_conduct, | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
direct_worker와 route_once가 동일한 경로이므로 보고서에 명시하십시오.
Line 312-322의 두 러너는 같은 orchestrator.route_once(_messages(prompt)) 호출을 실행합니다. 차이는 result["mode"] 문자열뿐입니다. 그러나 run_offline_cost_quality는 두 정책을 독립 정책으로 셀에 기록하고, summarize_policy_cells와 build_pareto_frontiers는 두 개의 별도 행을 만듭니다. 독자는 이를 서로 다른 두 정책의 비교 근거로 해석할 수 있습니다.
또한 hindsight_best_single은 direct_worker 셀에서 파생됩니다. 따라서 세 정책 행이 모두 같은 호출 경로에서 나옵니다. 이는 이 모듈이 선언한 공정 비교 계약과 어긋납니다.
최소 수정으로 러너 반환값에 alias 사실을 기록하십시오. 그러면 보고서 소비자가 근거를 구분할 수 있습니다.
🐛 제안 수정
def direct_worker(prompt: str) -> dict[str, Any]:
result = orchestrator.route_once(_messages(prompt))
result = dict(result)
result["mode"] = "direct_worker"
+ # Same provider path as route_once: no independent direct baseline exists yet.
+ result["policy_path_alias"] = "route_once"
return resultmock:// 에이전트 외의 실제 비교가 필요하면, 라우팅 결정을 우회하는 별도 direct 경로를 오케스트레이터에 추가하십시오.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def direct_worker(prompt: str) -> dict[str, Any]: | |
| result = orchestrator.route_once(_messages(prompt)) | |
| result = dict(result) | |
| result["mode"] = "direct_worker" | |
| return result | |
| def route_once(prompt: str) -> dict[str, Any]: | |
| result = orchestrator.route_once(_messages(prompt)) | |
| result = dict(result) | |
| result["mode"] = "route_once" | |
| return result | |
| def bounded_conduct(prompt: str) -> dict[str, Any]: | |
| result = orchestrator.conduct(_messages(prompt)) | |
| result = dict(result) | |
| result["mode"] = "bounded_conduct" | |
| return result | |
| return { | |
| "direct_worker": direct_worker, | |
| "route_once": route_once, | |
| "bounded_conduct": bounded_conduct, | |
| } | |
| def direct_worker(prompt: str) -> dict[str, Any]: | |
| result = orchestrator.route_once(_messages(prompt)) | |
| result = dict(result) | |
| result["mode"] = "direct_worker" | |
| # Same provider path as route_once: no independent direct baseline exists yet. | |
| result["policy_path_alias"] = "route_once" | |
| return result | |
| def route_once(prompt: str) -> dict[str, Any]: | |
| result = orchestrator.route_once(_messages(prompt)) | |
| result = dict(result) | |
| result["mode"] = "route_once" | |
| return result | |
| def bounded_conduct(prompt: str) -> dict[str, Any]: | |
| result = orchestrator.conduct(_messages(prompt)) | |
| result = dict(result) | |
| result["mode"] = "bounded_conduct" | |
| return result | |
| return { | |
| "direct_worker": direct_worker, | |
| "route_once": route_once, | |
| "bounded_conduct": bounded_conduct, | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@contextual_orchestrator/nim_cost_quality.py` around lines 312 - 334, Update
direct_worker and route_once in the runner factory to record that they share the
same orchestrator.route_once(_messages(prompt)) execution path, using an
explicit alias indicator in their returned dictionaries. Preserve the existing
mode values and bounded_conduct behavior so report consumers can distinguish
aliased results from independently executed policies.
| if status_code == 200: | ||
| return PROBE_OUTCOME_CHAT # refined by response body shape in classify_probe_result | ||
| if status_code == 429: | ||
| return PROBE_OUTCOME_RATE_LIMITED | ||
| if status_code in (401, 403): | ||
| return PROBE_OUTCOME_UNAVAILABLE | ||
| if status_code == 404: | ||
| return PROBE_OUTCOME_UNSUPPORTED | ||
| if status_code == 408 or status_code == 504: | ||
| return PROBE_OUTCOME_TIMEOUT | ||
| if 400 <= status_code < 500: | ||
| return PROBE_OUTCOME_UNSUPPORTED | ||
| if 500 <= status_code < 600: | ||
| return PROBE_OUTCOME_FAILED | ||
| raise NimDiscoveryError(f"unsupported probe status_code: {status_code}") |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
200 이외의 2xx와 3xx 상태를 분류하십시오.
현재 분기는 200, 4xx, 5xx만 처리합니다. 202 또는 302 같은 상태는 NimDiscoveryError를 발생시킵니다. run_capability_probes_dry_run은 이 예외를 잡지 않으므로 하나의 fixture 행이 전체 probe 실행을 중단시킵니다. docstring은 이 분류기를 live probe에서도 재사용한다고 명시합니다. live 경로에서는 redirect나 202 응답이 실제로 발생할 수 있습니다.
2xx와 3xx를 명시적 outcome으로 매핑하십시오. 예를 들어 200 이외의 2xx는 PROBE_OUTCOME_MALFORMED, 3xx는 PROBE_OUTCOME_FAILED로 처리할 수 있습니다.
🛡️ 제안 수정
if 400 <= status_code < 500:
return PROBE_OUTCOME_UNSUPPORTED
if 500 <= status_code < 600:
return PROBE_OUTCOME_FAILED
+ if 200 <= status_code < 300:
+ # A non-200 success shape cannot be scored as a capability outcome.
+ return PROBE_OUTCOME_MALFORMED
+ if 300 <= status_code < 400:
+ # Redirects are not followed on the probe boundary.
+ return PROBE_OUTCOME_FAILED
raise NimDiscoveryError(f"unsupported probe status_code: {status_code}")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if status_code == 200: | |
| return PROBE_OUTCOME_CHAT # refined by response body shape in classify_probe_result | |
| if status_code == 429: | |
| return PROBE_OUTCOME_RATE_LIMITED | |
| if status_code in (401, 403): | |
| return PROBE_OUTCOME_UNAVAILABLE | |
| if status_code == 404: | |
| return PROBE_OUTCOME_UNSUPPORTED | |
| if status_code == 408 or status_code == 504: | |
| return PROBE_OUTCOME_TIMEOUT | |
| if 400 <= status_code < 500: | |
| return PROBE_OUTCOME_UNSUPPORTED | |
| if 500 <= status_code < 600: | |
| return PROBE_OUTCOME_FAILED | |
| raise NimDiscoveryError(f"unsupported probe status_code: {status_code}") | |
| if status_code == 200: | |
| return PROBE_OUTCOME_CHAT # refined by response body shape in classify_probe_result | |
| if status_code == 429: | |
| return PROBE_OUTCOME_RATE_LIMITED | |
| if status_code in (401, 403): | |
| return PROBE_OUTCOME_UNAVAILABLE | |
| if status_code == 404: | |
| return PROBE_OUTCOME_UNSUPPORTED | |
| if status_code == 408 or status_code == 504: | |
| return PROBE_OUTCOME_TIMEOUT | |
| if 400 <= status_code < 500: | |
| return PROBE_OUTCOME_UNSUPPORTED | |
| if 500 <= status_code < 600: | |
| return PROBE_OUTCOME_FAILED | |
| if 200 <= status_code < 300: | |
| # A non-200 success shape cannot be scored as a capability outcome. | |
| return PROBE_OUTCOME_MALFORMED | |
| if 300 <= status_code < 400: | |
| # Redirects are not followed on the probe boundary. | |
| return PROBE_OUTCOME_FAILED | |
| raise NimDiscoveryError(f"unsupported probe status_code: {status_code}") |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@contextual_orchestrator/nim_discovery.py` around lines 415 - 429, Update the
status classification logic near the existing 200/4xx/5xx branches so every 2xx
response other than 200 maps explicitly to PROBE_OUTCOME_MALFORMED, and every
3xx response maps to PROBE_OUTCOME_FAILED. Preserve the existing mappings for
200, 4xx, 5xx, and the unsupported-status error.
| if body is None: | ||
| outcome = { | ||
| "chat": PROBE_OUTCOME_CHAT, | ||
| "embeddings": PROBE_OUTCOME_EMBEDDINGS, | ||
| "image": PROBE_OUTCOME_IMAGE, | ||
| "audio": PROBE_OUTCOME_AUDIO, | ||
| "video": PROBE_OUTCOME_VIDEO, | ||
| }.get(kind, PROBE_OUTCOME_CHAT) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
알 수 없는 probe_kind를 chat으로 분류하지 마십시오.
Line 484의 .get(kind, PROBE_OUTCOME_CHAT)는 등록되지 않은 probe kind를 chat capability로 보고합니다. 예를 들어 probe_kind="vision"이고 body가 없는 200 응답은 outcome="chat"이 됩니다. 이 결과는 근거 없는 capability 주장입니다. body가 dict인 동일 입력은 line 498에서 malformed가 되므로 동작도 일관되지 않습니다.
알 수 없는 kind는 PROBE_OUTCOME_MALFORMED로 처리하십시오.
🛡️ 제안 수정
if body is None:
outcome = {
"chat": PROBE_OUTCOME_CHAT,
"embeddings": PROBE_OUTCOME_EMBEDDINGS,
"image": PROBE_OUTCOME_IMAGE,
"audio": PROBE_OUTCOME_AUDIO,
"video": PROBE_OUTCOME_VIDEO,
- }.get(kind, PROBE_OUTCOME_CHAT)
+ }.get(kind, PROBE_OUTCOME_MALFORMED)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if body is None: | |
| outcome = { | |
| "chat": PROBE_OUTCOME_CHAT, | |
| "embeddings": PROBE_OUTCOME_EMBEDDINGS, | |
| "image": PROBE_OUTCOME_IMAGE, | |
| "audio": PROBE_OUTCOME_AUDIO, | |
| "video": PROBE_OUTCOME_VIDEO, | |
| }.get(kind, PROBE_OUTCOME_CHAT) | |
| if body is None: | |
| outcome = { | |
| "chat": PROBE_OUTCOME_CHAT, | |
| "embeddings": PROBE_OUTCOME_EMBEDDINGS, | |
| "image": PROBE_OUTCOME_IMAGE, | |
| "audio": PROBE_OUTCOME_AUDIO, | |
| "video": PROBE_OUTCOME_VIDEO, | |
| }.get(kind, PROBE_OUTCOME_MALFORMED) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@contextual_orchestrator/nim_discovery.py` around lines 477 - 484, Update the
outcome mapping in the body-is-None branch to use PROBE_OUTCOME_MALFORMED as the
fallback for unknown kind values instead of PROBE_OUTCOME_CHAT. Preserve the
existing mappings for registered probe kinds and keep behavior consistent with
the malformed handling for dictionary bodies.
| model_id = row.get("model_id") | ||
| probe_kind = row.get("probe_kind") | ||
| if row.get("error_class"): | ||
| classified = classify_probe_result( | ||
| model_id=str(model_id or ""), | ||
| probe_kind=str(probe_kind or "chat"), | ||
| status_code=0, | ||
| error_class=str(row["error_class"]), | ||
| ) | ||
| else: | ||
| status = row.get("status_code") | ||
| if not isinstance(status, int) or isinstance(status, bool): | ||
| raise NimDiscoveryError(f"fixture row {index} needs int status_code or error_class") | ||
| classified = classify_probe_result( | ||
| model_id=str(model_id or ""), | ||
| probe_kind=str(probe_kind or "chat"), | ||
| status_code=status, | ||
| body=row.get("body"), | ||
| ) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
probe_kind가 없는 fixture 행을 chat으로 기본 처리하지 마십시오.
Line 581과 line 591은 누락된 probe_kind를 "chat"으로 대체합니다. 그 결과 probe kind를 선언하지 않은 행이 chat probe로 집계됩니다. 상태 코드가 200이고 body에 choices가 있으면 이 행은 line 601-607의 chat_eligible_model_ids에도 들어갑니다. chat_eligible_model_ids는 nim_cost_quality의 dry-run 계획 입력이므로 근거 없는 모델이 후속 단계로 전달됩니다. docstring도 probe_kind를 필수 필드로 명시합니다.
누락된 model_id와 probe_kind는 행 index를 포함한 NimDiscoveryError로 거부하십시오.
🛡️ 제안 수정
model_id = row.get("model_id")
probe_kind = row.get("probe_kind")
+ if not isinstance(model_id, str) or not model_id.strip():
+ raise NimDiscoveryError(f"fixture row {index} needs a non-empty model_id")
+ if not isinstance(probe_kind, str) or not probe_kind.strip():
+ raise NimDiscoveryError(f"fixture row {index} needs a non-empty probe_kind")
if row.get("error_class"):
classified = classify_probe_result(
- model_id=str(model_id or ""),
- probe_kind=str(probe_kind or "chat"),
+ model_id=model_id,
+ probe_kind=probe_kind,
status_code=0,
error_class=str(row["error_class"]),
)
else:
status = row.get("status_code")
if not isinstance(status, int) or isinstance(status, bool):
raise NimDiscoveryError(f"fixture row {index} needs int status_code or error_class")
classified = classify_probe_result(
- model_id=str(model_id or ""),
- probe_kind=str(probe_kind or "chat"),
+ model_id=model_id,
+ probe_kind=probe_kind,
status_code=status,
body=row.get("body"),
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| model_id = row.get("model_id") | |
| probe_kind = row.get("probe_kind") | |
| if row.get("error_class"): | |
| classified = classify_probe_result( | |
| model_id=str(model_id or ""), | |
| probe_kind=str(probe_kind or "chat"), | |
| status_code=0, | |
| error_class=str(row["error_class"]), | |
| ) | |
| else: | |
| status = row.get("status_code") | |
| if not isinstance(status, int) or isinstance(status, bool): | |
| raise NimDiscoveryError(f"fixture row {index} needs int status_code or error_class") | |
| classified = classify_probe_result( | |
| model_id=str(model_id or ""), | |
| probe_kind=str(probe_kind or "chat"), | |
| status_code=status, | |
| body=row.get("body"), | |
| ) | |
| model_id = row.get("model_id") | |
| probe_kind = row.get("probe_kind") | |
| if not isinstance(model_id, str) or not model_id.strip(): | |
| raise NimDiscoveryError(f"fixture row {index} needs a non-empty model_id") | |
| if not isinstance(probe_kind, str) or not probe_kind.strip(): | |
| raise NimDiscoveryError(f"fixture row {index} needs a non-empty probe_kind") | |
| if row.get("error_class"): | |
| classified = classify_probe_result( | |
| model_id=model_id, | |
| probe_kind=probe_kind, | |
| status_code=0, | |
| error_class=str(row["error_class"]), | |
| ) | |
| else: | |
| status = row.get("status_code") | |
| if not isinstance(status, int) or isinstance(status, bool): | |
| raise NimDiscoveryError(f"fixture row {index} needs int status_code or error_class") | |
| classified = classify_probe_result( | |
| model_id=model_id, | |
| probe_kind=probe_kind, | |
| status_code=status, | |
| body=row.get("body"), | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@contextual_orchestrator/nim_discovery.py` around lines 576 - 594, Update the
fixture-row validation around classify_probe_result to require both model_id and
probe_kind, rejecting missing values with NimDiscoveryError that includes the
row index. Remove the "chat" fallback in both error_class and status_code
branches, while preserving the existing status_code validation and
classification flow for valid rows.
| `build_capability_probe_plan` and `run_capability_probes_dry_run` classify | ||
| fixture probe rows (HTTP status + optional body shape) into chat / embeddings / | ||
| rate_limited / unsupported / timeout / failed outcomes without network or | ||
| secrets. Live probing remains opt-in via `RUN_LIVE_NIM_TESTS=1`. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
probe outcome 목록을 구현과 일치시키십시오.
Line 106-108은 outcome을 chat, embeddings, rate_limited, unsupported, timeout, failed로 열거합니다. contextual_orchestrator/nim_discovery.py의 구현은 unavailable(HTTP 401 / 403)과 malformed(200 응답의 body 형태 불일치)도 생성합니다. 두 값은 인증 실패와 잘못된 응답 본문을 일반 실패와 구분하므로 운영상 중요합니다. image, audio, video도 생성됩니다.
목록에 빠진 outcome을 추가하십시오.
📝 제안 수정
`build_capability_probe_plan` and `run_capability_probes_dry_run` classify
-fixture probe rows (HTTP status + optional body shape) into chat / embeddings /
-rate_limited / unsupported / timeout / failed outcomes without network or
-secrets. Live probing remains opt-in via `RUN_LIVE_NIM_TESTS=1`.
+fixture probe rows (HTTP status + optional body shape) into chat / embeddings /
+image / audio / video / unavailable / rate_limited / unsupported / timeout /
+malformed / failed outcomes without network or secrets. Live probing remains
+opt-in via `RUN_LIVE_NIM_TESTS=1`.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| `build_capability_probe_plan` and `run_capability_probes_dry_run` classify | |
| fixture probe rows (HTTP status + optional body shape) into chat / embeddings / | |
| rate_limited / unsupported / timeout / failed outcomes without network or | |
| secrets. Live probing remains opt-in via `RUN_LIVE_NIM_TESTS=1`. | |
| `build_capability_probe_plan` and `run_capability_probes_dry_run` classify | |
| fixture probe rows (HTTP status + optional body shape) into chat / embeddings / | |
| image / audio / video / unavailable / rate_limited / unsupported / timeout / | |
| malformed / failed outcomes without network or secrets. Live probing remains | |
| opt-in via `RUN_LIVE_NIM_TESTS=1`. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/architecture.md` around lines 106 - 109, Update the outcome list in the
architecture documentation describing build_capability_probe_plan and
run_capability_probes_dry_run to include unavailable, malformed, image, audio,
and video alongside the existing outcomes, matching all values produced by
contextual_orchestrator/nim_discovery.py.
| # Mock answers echo the prompt; scorers must not invent quality from that alone. | ||
| self.assertTrue(all(c["outcome"] == "success" for c in report["cells"] if c["policy_name"] != "hindsight_best_single" or True)) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
항등 조건 or True를 제거하십시오.
Line 389의 조건 c["policy_name"] != "hindsight_best_single" or True는 항상 참입니다. 따라서 필터가 아무 행도 제외하지 않습니다. 의도가 불명확하게 남고, 이후 유지보수자가 hindsight 셀이 제외된다고 잘못 읽을 수 있습니다. 실제 단정은 모든 셀에 적용됩니다.
필터를 제거하고 단정을 명시하십시오.
💚 제안 수정
- # Mock answers echo the prompt; scorers must not invent quality from that alone.
- self.assertTrue(all(c["outcome"] == "success" for c in report["cells"] if c["policy_name"] != "hindsight_best_single" or True))
+ # Mock answers echo the prompt; scorers must not invent quality from that alone.
+ # Every cell, including derived hindsight_best_single rows, must run without error.
+ self.assertTrue(all(c["outcome"] == "success" for c in report["cells"]))📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Mock answers echo the prompt; scorers must not invent quality from that alone. | |
| self.assertTrue(all(c["outcome"] == "success" for c in report["cells"] if c["policy_name"] != "hindsight_best_single" or True)) | |
| # Mock answers echo the prompt; scorers must not invent quality from that alone. | |
| # Every cell, including derived hindsight_best_single rows, must run without error. | |
| self.assertTrue(all(c["outcome"] == "success" for c in report["cells"])) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_nim_cost_quality.py` around lines 388 - 389, Remove the redundant
`or True` condition from the assertion over `report["cells"]` in the test,
making it explicitly validate that every cell has an `"outcome"` of `"success"`
without filtering by `policy_name`.
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head61a39435117c20ccabcf6246392ff8c89f5d91e3. -
Head SHA:
61a39435117c20ccabcf6246392ff8c89f5d91e3 -
Workflow run: 31779360781
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (10 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (10 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (2 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (2 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (10 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (10 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (2 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (2 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
|
|
Closing this explicitly superseded scaffold. Issue #86 is the durable requirement authority and already records the useful catalog, ID-collision, fixture-classification, and live-opt-in evidence. The current branch remains incomplete for all-model capability probing, equal-budget direct/route/conduct evaluation, strict scoring, cost honesty, uncertainty/Pareto evidence, transactional artifacts, and the protected provider-egress boundary; it also carries unrelated role-temperature and suppression-only changes. Selectively rebuild valid unique work after #96 reaches protected integration rather than keeping a non-mergeable authority in the active PR queue. |
2 participants
Status: Draft superseded scaffold — do not merge
This branch is open, but it is not the accepted implementation authority for issue #86 and it does not close that issue. It targets protected
main@6841b71935e0b7cb98fb52bcb4709cc5100c8d87; the exact current contributor head is325a1a46e7ee2028fe6f7fe2ec7c626e63e4fa2b.PR #90 is now closed unmerged. It remains useful superseded design and test evidence for the complete benchmark contract, but neither #90 nor this scaffold is protected-main authority. Issue #86 is the durable requirement owner and currently has no accepted complete implementation PR.
What this scaffold currently proves
The current head adds a bounded NIM catalog helper and has repaired several initial defects:
offline_fixturerather than live evidence;RUN_LIVE_NIM_TESTS=1opt-in.These improvements are partial evidence only. They do not provide dynamic all-model capability probing, equal-budget direct/route/conduct evaluation, strict complete-answer scoring, actual-vs-hypothetical cost separation, paired uncertainty, Pareto analysis, complete provenance, transactional JSON/CSV/Markdown publication, or the full provider-response/egress boundary required by #86.
Current blockers
Five current, valid unresolved CodeRabbit threads remain:
cost_ledger.pyis not an accepted security remediation;--servestill permitsverify_tls=Falsefor Bearer-authenticated external provider traffic; andThe correct resolution is not to deepen this competing scaffold. Preserve useful unique evidence, remove unrelated and suppression-only changes, and selectively rebuild the complete #86 implementation test-first after PR #96 reaches an accepted protected result.
Exact-head evidence state
For
325a1a46e7ee2028fe6f7fe2ec7c626e63e4fa2b:31594926051succeeded;31596033849, Security31596033834, Security Scan31596034033, Fuzz31596033960, and SAST31596033842are queued and are not passing evidence;Queued, cancelled, predecessor-head, status-only, author-only, synthetic-merge-only, or model-review evidence is never merge authority. Keep this PR Draft and unmerged.
Refs #86; does not close it.
Summary by CodeRabbit
unknown으로 표시합니다.