Experiment: prepare revised HTTP knowledge A/B gate

[Wenjie Fan (gggdttt)]

Current status: v2 preparation only

The revised seven-entry v2 pair is prepared and unrun. It tests the marginal effect of microsoft/BCQuality#196 as a bundle, not an individual HTTP article or routing contribution. A new explicit user approval is required before any reviewer generation, gold-judge call or evaluation dispatch. No retry, replication, result publication, release or product merge is authorized by this preparation.

Final immutable v2 pair

Arm	BC-Bench commit	Declared engine	Engine-owned BCQuality
A, this draft	0cbb709caaf92fcb2017bcfa0b1c77185d37cbc6	da9236a6f43d47c37e0bd59ebced33b6fe73f277	130d5de6c4bd72d2158240fe431862b6434c735a
B, dependent #912	043ddc2c38bd1e71f5dedc5a9487b441d38d13de	1e61e1eacbd7e0a55cb63dc4e9418a8ed9059b19	4287233f80678fd4fb02aca6455d9cf046ca9262

Both engines share source 2ad4f65b231707d5222be27f4a7c6ab442662722 except the declared BCQuality ref; see microsoft/BC-ALAgents#78. Final Bench trees differ only by one SHA replacement in .github/actions/install-agent-harnesses/action.yml and the matching DEFAULT_ENGINE_SHA in tests/test_review_workflows.py. Every other tracked mode/blob, fixture, workflow, scorer, judge and package version 0.14.0.dev1 is identical. The candidate incorporated v2 through a normal merge; published history was not rewritten.

Dataset SHA-256: 027fcf04cdbf506aeada64ef70498b059006d946e6804162a662ccbba45ef1d8.

The shared support selectively integrates the necessary adapter foundation from open, unmerged #881 at 8e2946fd8583dc406d2ecacbaa6ef265326c0db1, not #887. Generic harnesses, unrelated model choices, dependency resolution and runtime/scorer behavior are unchanged by v2. This is not a release.

Fixed scope: seven entries, six target sites

Entry	V2 role
synthetic__privacy-008	Historical target g1
synthetic__privacy-015	Historical targets g2 and g3
synthetic__security-clean-02	Historical target g0
synthetic__errh-tryfunction-swallowed-01	Existing consumed-TryFunction positive control
synthetic__http-optional-clean-01	Controlled bare GET and POST target sites at lines 12 and 27, each with correct HTTP status handling; expected comments remain empty
synthetic__http-consumed-false-01	Revised swallowed-execution-failure positive control
synthetic__http-status-body-01	Completed non-2xx body-processing positive control

synthetic__privacy-010 is removed only from the experimental input, excluding both its historical target and unrelated outbox agent-finding rejection. Its dataset row and scores are untouched. All 145 original rows remain byte-for-byte unchanged. Of the three experimental rows, only the consumed-false row changes.

That row now uses if not Client.Post(HeartbeatUrlTok, Content, Response) then exit(true);, with the expected finding at line 12..12. The successful exchange still falls through to IsSuccessStatusCode; the false path never reads the response. No explanatory steering comments, extra gold or ignored comments were introduced. The compact shape reduces the particular multi-line location ambiguity but does not guarantee future model compliance.

All three HTTP fixtures produce zero diagnostics using the already-installed AL extension 30.0.2804007, parser assembly 30.0.42.51579, via SyntaxFactory.ParseSyntaxTree. A deliberately missing parenthesis in an in-memory sensitivity control produces AL0104. This is syntax only, not symbol resolution, compilation, analyzer execution or runtime validation, and does not prove BC27 application compatibility. No tooling was installed or upgraded. The parent accepted the revised shape for experiment-only use conditional on zero syntax diagnostics; that condition passed. Stochastic Style noise remains possible and must be inventoried rather than hidden.

HTTP article annotations remain empty because those articles do not exist at the pre-196 baseline. There are 8 expected comments across 5 judge-eligible entries. Existing gold and scorer behavior remain unchanged.

Proposed paid gate, not execution permission

Both arms retain CLI 1.0.88, root gpt-5.6-sol, leaf gpt-5.6-luna, serial scheduling, configured concurrency 4/effective 1, timeout 30 minutes, Medium severity floors and identical ordered 19 leaves + 1 root.

A new approval may authorize one paired smoke: 14 entry generations, at most 280 reviewer processes and 10 gold-judge calls. Five further pairs require another approval: 70 generations, 1400 processes and at most 50 judge calls. Combined conditional maximum: 84 / 1680 / 60. These are invocation bounds, not provider-call, token, credit or dollar caps.

Collect and audit every baseline entry before considering B. Baseline eligibility requires all seven valid reports, all three positive controls and at least one optional-return misread at the six fixed sites. Report the four historical and two controlled sites separately. If both controlled sites are silent but a historical target reproduces, the target-signal condition can still pass. Missing signal, invalid output or a missing control means STOP before B, without retry.

Candidate acceptance requires no target or mixed assertion at any of the six sites in original leaf, root and normalized output, with controls preserved. A leaf misread later removed by the root is not a v2 pass, and filtering losses are not suppression. Use the fixed human rubric, no new AI adjudicator. Inventory unrelated findings; normal Bench P/R/F1 remains secondary. Five replications are not authorized now and are not a powered significance claim.

Integrity and no-publish route

Require exact executed/declared pin equality, source and fixture hashes, all ordered process roles, observed models, complete usage, original schema/role checks and nested outcomes. Manifest completion or JSON schema validity alone is insufficient. Startup CLI must be exactly 1.0.88; optional raw OTel CLI versions remain null rather than being filled from configuration. Raw OTel is deleted after ingestion and is not promised as archived.

Any future authorized run must use the exact seven-ID entries array, test-run=true, modified-only=false, repeat=1, and explicit engine-sha equal to its declared pin. Recheck live branch heads before spending. This disables requeue/ephemeral tags and makes summary mock=true, omitting Braintrust/Kusto storage and leaderboard writes. Summary mock does not mock reviewer generation or the gold judge. Preserve the existing concurrency group and matrix cap; no bypass is added.

Separate immutable v1 observation

V1 prepared A 32a4c2190f996ef68e696716fed201e0d005cebd / B 385cf73a9aa217692a43af4ec6c4997967efd668. Its only evaluation was baseline run 36839547719: 6/8 complete observations; two original root reports were partial due to consumer-contract rejections. Candidate B was not dispatched. Old inputs, STOP receipt, run artifacts and hashes remain unchanged.

V2 is informed by this failed preparation/signal attempt and therefore has a revised scope and fixture. Do not pool v1 observations, reuse its authorization, or describe v2 as an unchanged retry. No causal A/B conclusion follows from v1, and v2 has no evaluation results.

Unpaid preparation evidence

• Baseline: 93 focused checks and 1120 full unit tests passed (2 skipped, 1 deselected); focused pre-commit and type check passed.
• Installed AL parser: zero syntax diagnostics for all three fixtures, plus a working negative sensitivity check; not AL-compiled.
• Baseline automatic CI: all 16 checks passed, including CodeQL.
• Exact candidate SHA: the same 93 focused checks passed, plus Ruff and diff checks; independently verified complete tree proof leaves only the two pin lines different.
• Candidate full remote CI is not claimed: its dependent base does not trigger the main-only workflow. CLA, inherited baseline CI, local checks and exact tree equivalence are the evidence. No workflow was dispatched to fill that gap.

PREPARATION ONLY / NOT_AUTHORIZED.