Skip to content

test(golden): drop the Darwin special case on the three VMAFEXEC_score assertions - #1502

Closed
lusoris wants to merge 1 commit into
masterfrom
test/golden-drop-darwin-special-case
Closed

lusoris wants to merge 1 commit into
masterfrom
test/golden-drop-darwin-special-case

Conversation

@lusoris

@lusoris lusoris commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR edits Netflix golden assertions. It does so on the maintainer's explicit authorization (popup answer, 2026-09-19: "Explicitly authorize me to edit them"), and it is the only PR in the queue that touches python/test/.

Three assertAlmostEqual calls in python/test/vmafexec_test.py (lines 941, 1051, 1107) expected 88.030322 on macOS and 88.030463 elsewhere. After the September upstream reconciliation (#1473) macOS produces 88.030459, which is within places=4 of the Linux value and not of the old Darwin value — so the special case itself had become the thing failing the four macOS lanes on #1473, #1474, #1476, #1477, #1478 and #1481:

FAILED test_run_vmafexec_runner_akiyo_multiply_disable_enhn_gain
FAILED test_run_vmafexec_runner_akiyo_multiply_no_enhn_gain_model
FAILED test_run_vmafexec_runner_akiyo_multiply_with_feature_enhn_gain_limit
  'np.float64(88.030459) != 88.030322 within 4 places (0.000137 difference)'

Those three lines now expect 88.030463 everywhere. Exactly three assertion lines change. The other three per-platform values in the file (132.73…, 129.47…, 122.80…) are not failing, still measure as recorded, and are untouched; _IS_DARWIN stays for them. The ADR-0418 header comment gains a note recording the change and why.

What this is not

  • Not a loosening: places=4 is unchanged on every assertion.
  • Not a new value: 88.030463 is the value the Linux lanes have asserted all along.
  • Not a code change: nothing under core/ moves.

Verification

Check Result
Diff against master 3 assertion lines + 8 comment lines, one file
The three failing macOS tests, expected vs measured 88.030463 vs 88.030459, Δ 4e-6, inside places=4
Linux lanes unchanged value, so unchanged result
praetorctl audit passes

.standards-baseline.json is re-recorded (line shifts in this file are line-keyed); it lands at 1414, which is what origin/master actually measures — see ledger L-80 and #1499 for why the committed 1433 was loose.

Type

  • test — test-only change

Checklist

  • Commits follow Conventional Commits.
  • Pre-push hooks pass.
  • I did modify assertAlmostEqual(...) values in the Netflix golden Python tests — with the maintainer's explicit authorization, recorded above and in the commit message.
  • docs/state.md: no row — this closes no bug and opens none; it unblocks the six PRs listed above.

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: trivial. The measurement is the three CI failure lines quoted above.
  • Decision matrix — no alternatives: only-one-way fix. The Darwin value is simply no longer what macOS produces.
  • AGENTS.md invariant note — no rebase-sensitive invariants: an upstream sync never touches this fork-local test file.
  • Reproducer / smoke-test command — below.
  • CHANGELOG fragment — no changelog fragment needed: test-only, no user-visible delta.
  • Rebase note — no rebase impact: test-only, fork-local file.

Reproducer

cd python && python3 -m pytest -q test/vmafexec_test.py -k "akiyo_multiply"

On a macOS host with #1473 applied: fails on master (three != 88.030322 assertions), passes here.

…e assertions

Maintainer-authorized golden change (popup answer, 2026-09-19). The
three assertAlmostEqual calls that expected 88.030322 on macOS and
88.030463 elsewhere now expect 88.030463 everywhere.

After the September upstream reconciliation (#1473) macOS produces
88.030459. That is within places=4 of the Linux value and not of the
old Darwin value, so the special case had become the thing failing the
macOS lanes: test_run_vmafexec_runner_akiyo_multiply_disable_enhn_gain,
_no_enhn_gain_model and _with_feature_enhn_gain_limit, each with
'88.030459 != 88.030322 within 4 places'.

Exactly three lines change. The other three per-platform values in
this file (132.73, 129.47, 122.80) still measure as recorded and are
untouched, and _IS_DARWIN stays for them.
@github-actions github-actions Bot added the type:test Test-only change label Sep 19, 2026
lusoris added a commit that referenced this pull request Sep 19, 2026
…e assertions

Maintainer-authorized golden change (popup answer, 2026-09-19), folded
into the upstream reconciliation because it is only true together with
it: with this port macOS produces 88.030459, within places=4 of the
Linux value 88.030463 and not of the old Darwin value 88.030322, so the
special case was what failed the four macOS lanes on this PR. Without
the port macOS still yields 88.030317, which is why the change could not
merge on its own (#1502, closed in favour of this commit).

Exactly three assertion lines change. The other three per-platform
values in the file still measure as recorded and are untouched.
@lusoris

lusoris commented Sep 19, 2026

Copy link
Copy Markdown
Contributor Author

Folded into #1473 as commit ed77a843b and closed here.

This change is only true together with the September upstream reconciliation. On master macOS still produces 88.030317, so dropping the Darwin special case on its own fails the macOS lanes — which is exactly what this PR's CI showed (88.030317 != 88.030463 within 4 places). With #1473's port applied macOS produces 88.030459, inside places=4 of the Linux value, and the special case is what fails instead.

Same three lines, same maintainer authorization, now carried by the PR that makes them correct.

@lusoris lusoris closed this Sep 19, 2026
lusoris added a commit that referenced this pull request Sep 20, 2026
…e assertions

Maintainer-authorized golden change (popup answer, 2026-09-19), folded
into the upstream reconciliation because it is only true together with
it: with this port macOS produces 88.030459, within places=4 of the
Linux value 88.030463 and not of the old Darwin value 88.030322, so the
special case was what failed the four macOS lanes on this PR. Without
the port macOS still yields 88.030317, which is why the change could not
merge on its own (#1502, closed in favour of this commit).

Exactly three assertion lines change. The other three per-platform
values in the file still measure as recorded and are untouched.
lusoris added a commit that referenced this pull request Sep 20, 2026
…e assertions

Maintainer-authorized golden change (popup answer, 2026-09-19), folded
into the upstream reconciliation because it is only true together with
it: with this port macOS produces 88.030459, within places=4 of the
Linux value 88.030463 and not of the old Darwin value 88.030322, so the
special case was what failed the four macOS lanes on this PR. Without
the port macOS still yields 88.030317, which is why the change could not
merge on its own (#1502, closed in favour of this commit).

Exactly three assertion lines change. The other three per-platform
values in the file still measure as recorded and are untouched.
@lusoris
lusoris deleted the test/golden-drop-darwin-special-case branch October 6, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:test Test-only change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant