Skip to content

fix(rc1): integrate pre-RC1 correctness train - #1561

Merged
lusoris merged 316 commits into
masterfrom
train/pre-rc1-correctness-20260924
Sep 26, 2026
Merged

lusoris merged 316 commits into
masterfrom
train/pre-rc1-correctness-20260924

Conversation

@lusoris

@lusoris lusoris commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Integrates the reviewed pre-RC1 correctness train into one protected merge path. It closes the confirmed metric-failure publication and backend-lifecycle defects, restores fail-closed quality/security gates, applies the Go 1.27.1 modernization sweep, carries the already-reviewed dependency updates, and adds a portable external-hardware report bundle for RC1 testers.

Release sequencing is now explicit:

  • RC1: correctness, exact-candidate verification, and tester/report readiness.
  • RC2: benchmarking, profiling, performance tuning, and optional acceleration work.
  • RC3: the one-shot real model retrain, held-out validation, provenance, and production promotion.

Ordinary Renovate/version PRs are no longer frozen. Each update still has to rerun the affected exact-head evidence. No benchmark, tuning, or real training workload is included here.

Type

  • fix — bug fix
  • feat — RC1 tester/report capability
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation
  • test — tests
  • build / ci — tooling / infrastructure
  • port — verbatim upstream Netflix/vmaf port
  • sycl / cuda / simd — backend-specific

RC1 tester path

tools/rc1-tester/vmaf-rc1-report now:

  • inventories CPU/SIMD, CUDA, SYCL, HIP, Metal, compilers, and repository tools;
  • runs a bounded four-frame explicit-backend correctness smoke with a CPU reference;
  • fails closed on missing metrics, fallback, malformed output, or excessive delta;
  • supports accelerator device ordinals and auto-runs the CPU reference when needed;
  • emits a normalized .tar.gz or .zip containing Markdown, machine-readable JSON, logs, checksums, collector revision, and binary hash;
  • redacts checkout and temporary paths before the archive is created;
  • labels benchmark/tuning tools as RC2 and real-training tools as RC3.

Source-checkout and installed-console entry points are both covered. Linux, macOS, and Windows launch/build guidance is documented in docs/usage/rc1-tester-guide.md; the bounded native suite is routed through make test-fast, not a raw Meson invocation.

Exact-head local evidence

  • Signed candidate head: 9568ab51a3c2dd13c3db33e936b04160d687c987.
  • Complete staged-file pre-commit suite passed, including governance, HISS, Semgrep, dependency locks, generated docs, FFmpeg replay, Meson credential-sanitization, and source-to-ADR provenance.
  • RC1 tester tests: 71 passed; Ruff, Black, C901, and function-length checks passed.
  • Python dependency locks: 26 hash-only locks verified; the tester package installs from its committed lock with --require-hashes, then --no-deps --no-build-isolation.
  • HISS replay contract: 7/7 tests passed; required-check aggregation: 80/80 contexts accounted for.
  • Real end-to-end CPU bundle: PASS, four frames, maximum metric delta 0, archive SHA-256 5a3948bab09f07a66af1063326b573de4aaf580c1f5898f1f83cea2399a441f1; internal checksums and normalized archive metadata verified.
  • RTX 4090 candidate-source CUDA motion parity: motion3 2/2, motion_v2 2/2; CPU/CUDA pooled-score delta 1.0e-6 with no tolerance change.
  • Arc A380 candidate-source SYCL ADM parity: 6/6 cases passed with the existing tolerance.
  • Reviewed train Netflix CPU golden gate: 271 passed, 12 skipped; no Netflix golden assertion changed.
  • Go 1.27.1 modernization covers all 118 Go files, and go fix -diff ./... is now a fail-closed local/CI clean-tree gate.

Hosted-failure follow-ups after 9568ab51a

Hosted CI on the documented head found real defects. Each one was fixed at its root and re-verified locally. No gate was loosened.

  • 7860f93d3, 31492952b, 8114ec846, ce3bbc7f9, 9638e18eb, 807b47cb2: a stale Vulkan docs anchor, eight CodeQL findings, hosted policy gaps, the Meson secret-sanitization timeout and per-shot boundary build, the changed-source clang-tidy findings, and FIFO primary-failure preservation.
  • 9b014f60e: the Windows max macro collision, the macOS case-insensitive Makefile/makefile inventory, and the CodeQL test-shim file mode.
  • 1614fa714: Tidy Changed and Tidy Ratchet.
    • fex_ctx_vector.h no longer includes feature_extractor.h, and with it <atomic>, inside extern "C".
    • The Metal PSNR parity test now uses VmafFeatureDictionary and stops freeing a dictionary that vmaf_use_feature() already consumed.
    • dict_grow_entries replaces an NDEBUG-erased assert with a hard guard, backed by a new zero-capacity test.
    • The CPU baseline was regenerated with make tidy-ratchet-write (620 -> 613) and matches CI's GCC 15 measurement file for file.
  • 6bdbb5a88: Python harness.
    • golden_gate_isolation_test.py checks import-time paths in a child interpreter. Its importlib.reload had replaced sys.modules['vmaf'] and broke later tests' mocks on every tox lane.
    • The bounded FIFO startup wait now observes child exit before it tries the readiness permit. This closes a race that intermittently rejected AssetExtractor on macOS; see T-PY-FIFO-READY-EXIT-RACE-2026-09-26.
    • CodeQL py/empty-except alerts 1290 and 1291 are fixed with explanatory comments.

The full SYCL release configuration exposed a oneAPI 2026.1 linker-plugin/LTO incompatibility; RC1 correctness builds and tester guidance use -Db_lto=false. Characterizing or tuning LTO belongs to RC2 rather than blocking correctness verification.

Checklist

  • Commits follow Conventional Commits and are signed.
  • The complete touched-file/pre-commit contract is green locally.
  • Focused CPU, CUDA, and SYCL correctness evidence is green on available hardware.
  • Feature-extractor twin gaps are fixed or classified exactly once in docs/state.md.
  • New native files carry applicable licence headers.
  • This is not a breaking change.
  • New ADRs use fragment-owned index rows and regenerated consolidated output.
  • User-visible surfaces include human-readable documentation.
  • Required hosted checks are green on this exact head. Normal protected auto-merge will enforce this.

Bug-status hygiene

  • Every remaining first-release row appears exactly once under RC1 hosted verification, RC2 performance, RC3 training, or an evidence-bound deferral.
  • Two stale hardware rows were closed with fresh RTX 4090 and Arc A380 evidence.
  • Pelorus-owned follow-ups were filed as VMAFx/pelorus issues only; no Pelorus code was changed here.

Netflix golden-data gate

  • No Netflix assertAlmostEqual(...) value was modified.
  • Fork-added report and parity assertions remain separate from Netflix golden data.

Deep-dive deliverables

  • Research digests are present under docs/research/.
  • Decision matrix — alternatives and decisions are recorded in the corresponding ADRs.
  • AGENTS.md invariant note — relevant rebase-sensitive invariants are preserved or updated.
  • Reproducer / smoke-test command — concrete commands are included below.
  • CHANGELOG fragments are present and generated output is current.
  • Rebase notes cover affected surfaces.

Reproducer

python3 -m pytest tools/rc1-tester/tests -q
python3 scripts/ci/check_python_dependency_locks.py check
make go-fix-check
(cd python && python3 -m pytest test/golden_gate_isolation_test.py test/python_harness_coverage_test.py test/executor_test.py test/raw_extractor_test.py -m main -q)
python3 scripts/ci/run_meson_test.py -- -C build test_dict test_fex_ctx_vector test_feature_extractor
./tools/rc1-tester/vmaf-rc1-report bundle \
  --vmaf-bin /path/to/vmaf --backend cpu --out-dir /tmp/vmafx-rc1-report

Known hosted boundary

Four rows now require hosted exact-head verification only; the implementation work identified for RC1 is complete locally. Any new correctness defect found by those checks returns to the fix-and-revalidate loop. Performance and training rows stay open in RC2/RC3 and do not masquerade as RC1 completion.

PR #1213 remains operator-held and untouched.

Closes #1525
Closes #1526

lusoris and others added 30 commits September 24, 2026 16:55
…ox locks

Repair all five root-cause blockers for hash-locked Python dependency
enforcement under ADR-1305:

1. Normalize and strictly parse pip option forms: preserve global
   pre-subcommand flags (--trusted-host, -v, etc.), split joined short
   options (-qrfoo, -rmalicious.txt, -cconstraints.txt, -ihttps://...),
   and fail closed on secondary unhashed requirements and constraint flags.
2. Bind manifest install_aliases to explicit consumer paths and context,
   rejecting directory traversal, Windows drive/UNC paths, duplicate JSON
   keys, and unreferenced/dead aliases fail-closed.
3. Generate universal workstation-portable dev locks for all Nox sessions
   without --python-platform, eliminating unconditional NVIDIA/Triton
   residue, and explicitly pin session Python versions matching locks.
4. Restrict Nox AST receiver authority to @nox.session parameters, track
   and reject session/method aliases, and scan literal shell runners
   fail-closed.
5. Fail closed with ContractError on git command and filesystem errors
   during consumer and lock discovery.
Resolves all six live GitHub CodeQL cpp/equality-on-floats alerts on
current origin/master at their semantic root cause without scanner
suppressions, query evasion, public-ABI expansion, score/tolerance
weakening, or Netflix golden changes (ADR-1308, Research-2097):

- core/src/feature/feature_name.cpp:149 (Alert 168): option_double_equals
  compares option doubles considering NaN equals NaN, signed zeros
  +0.0 == -0.0, and 64-bit bit identity for finite values.
- core/src/predict.c:302 (Alert 927): float_values_equal handles sentinel
  equality checks across NaN, signed zeros, and bit patterns.
- core/test/test_svm_api.c:368, 504 (Alert 1101): svm_labels_equal compares
  discrete SVM integer-class labels via constant comparison (== 0.0).
- core/src/feature/brisque_math.h:366 (Alert 1201): asserts span != 0.0 &&
  isfinite(span) on the normalization span; extracts brisque_aggd_accumulate
  to conform to NASA JPL / HISS-04 complexity limits (<= 60 LOC).
- core/src/mcp/3rdparty/cJSON/cJSON.c:615 (Alert 1221): evaluates
  d - (double)item->valueint == 0.0, preserving compiler float model.
- core/test/test_cambi.c:1142 (Alert 1244): float_bits_equal asserts bit-exact
  single-precision kernel equivalence between AVX2 and scalar paths.

All targets verified clean on fresh CodeQL database analysis. Focused unit
tests (6/6), fast suite (145/145), and Netflix golden data gate (271 passed)
all pass green. Pre-commit hooks, docs fragments, ADR links, state rows,
dedupe scan, and make verify-all pass.
… exclude doc recipes

Resolve three root causes identified during independent review of hash-locked
Python dependency enforcement under ADR-1305:

1. Decouple local source validation from ambient working directory: resolve
   local sources and bare package names (such as ai) against explicit repository
   root and consumer context, ensuring check_python_dependency_locks.py --root
   ABS_REPO check and test suites pass from unrelated working directories
   without global chdir.
2. Fail closed on multiple editable targets: extract and validate all editable
   targets rather than only the first, ensuring every target is proven local
   and authorized, and rejecting remote, unhashed, dangling, or traversal
   targets across joined, long, and reordered flag forms.
3. Fix tracked_consumer_paths container recipe classification: distinguish
   Markdown and documentation paths (e.g. docs/adr/by-tag/containerfile.md,
   changelog.d/fixed/containerfile-dpkg-gitam-bash.md) from genuine Dockerfile
   and Containerfile recipes, while preserving full scanning of Dockerfile,
   Dockerfile.production, Containerfile, and repository variants.
…entation

- Correct contract descriptions for option_double_equals and float_values_equal
  to accurately state that NaN is never equal even to NaN (preserving IEEE-754
  semantics), signed zeros match, same infinities match, and finite values compare
  via 64-bit IEEE bit identity.
- Correct svm_labels_equal description to specify 64-bit bit identity via memcpy
  with signed-zero equivalence and same-infinity behavior, rejecting NaN, rather
  than a-minus-b or finiteness checks.
- Accurately name CAMBI parity check check_c_values_avx2_parity (c_scalar[i] vs
  c_avx2[i]) and BRISQUE normalization brisque_range_scale.
- Record actual CodeQL 2.27.0 CLI toolchain and database source hash
  (d850ee7bdf63ec8287922e8ed0f25009b0dd18a5a8765ef93a18f6e6412c593c).
- Add item 6 (d - (double)item->valueint == 0.0) to the numbered re-vendoring
  delta list in core/src/mcp/3rdparty/cJSON/AGENTS.md.
- Document investigation of reviewer golden discrepancy: running without required
  CPU environment (CUDA_VISIBLE_DEVICES="" VMAF_FORCE_BACKEND=cpu) triggered
  CUDA feature extraction kernels differing on 10 tests, whereas running with
  CPU environment passes 271 with 12 skipped and 0 failures.
CodeQL cpp/large-parameter alerts 1108–1112 in core/src/feature/x86/vif_avx512.c
flagged pass-by-value of 128-byte VifPair512 and 256-byte VifTaps8 aggregates
in private stage helpers extracted in commit 6800d0f (Research-2046).

Under System V AMD64 ABI (§3.2.3) and Windows x64 ABI, aggregate types larger
than 64 bytes (eight eightbytes) cannot be passed in vector registers and are
classified as class MEMORY. Passing them by value forced callers to allocate
stack space and copy them via memory instructions when out of line. They were
never deliberately register-passed by ABI design.

Converted internal stage helpers to pass aggregates by const pointer:
  - vif_horizontal_energy_pack512(const VifPair512 *acc)
  - vif_vertical_mean8(VifPair512 *acc, const VifTaps8 *t, __m512i coeff) (Alert 1112)
  - vif_vertical_energy8(VifPair512 *acc, const VifTaps8 *a, const VifTaps8 *b, ...) (Alerts 1110, 1111)
  - vif_vertical_store8(uint32_t *dst, const VifPair512 *acc) (Alert 1109)
  - vif_vertical_store_mean8(uint32_t *dst, const VifPair512 *acc)
  - vif_vertical_energy16(VifEnergy512 *acc, const VifPair512 *weighted, ...) (Alert 1108)
  - vif_vertical_store_mean16(uint32_t *dst, const VifPair512 *acc, int r, int s)
  - vif_vertical_store_energy16(uint32_t *dst, const VifEnergy512 *acc, int r, int s)

Under -O3, GCC inlines the FORCE_INLINE helpers and folds pointer dereferences
directly without stack spills or extra instructions. Emitted machine code for
all exported functions in feature_x86_vif_avx512.c.o is byte-for-byte instruction
identical to master (differing only in assertion __LINE__ constants).

No change to public ABI (core/src/feature/x86/vif_avx512.h is untouched).

Verified:
  - test_integer_vif_avx512_stages_red_check: baseline passes, proves red-capable
    by detecting 1-bit input and intermediate plane perturbations.
  - test_integer_vif_avx512_stages: tri-way bit-exact check across scalar,
    AVX2, and AVX-512 over 3,456 combinations (8–16 bpc, 12 widths, 4 heights,
    4 scales, 2 patterns).
  - scripts/ci/check-win64-stack-alignment.py: 0 violations.
  - Sanitizers: clean under AddressSanitizer and UndefinedBehaviorSanitizer.
  - Linters: make format-check, clang-tidy, and cppcheck --inline-suppr clean.
  - Docs: Research-2078, docs/state.md, docs/rebase-notes.md, changelog fragment,
    core/src/feature/x86/AGENTS.md.
…nventory

Fold compile-time C23 and C++26 internal header smoke checks and enum/struct
ABI width assertions into existing test translation units (test_flush_context_ordering.c
and test_luminance_tools.cpp) instead of standalone smoke translation units.
Removes the internal_header_language_abi_smoke static library target from
core/test/meson.build, restoring the authoritative CPU clang-tidy translation unit
inventory from 315 back to 313 without expanding or weakening baselines.
…ements

Audit and fix workflow checkout ordering defects and add fail-closed scanner:

- In .github/workflows/ffmpeg-integration.yml: reorder actions/checkout and load-build-config.sh before setup-python and pip install in job 'ffmpeg', matching sibling job 'ffmpeg-sycl'.
- In .github/workflows/supply-chain.yml: insert actions/checkout step in job 'sbom' before downloading artifacts and preparing SBOM roots.
- In scripts/ci/check_python_dependency_locks.py: implement scan_workflow_checkout_ordering scanner and wire into check() across all tracked workflows to detect any pre-checkout consumption of repo requirements, editable packages, helper scripts, or local actions. Functions comply strictly with NASA JPL Rule 4 (<= 22 LOC, <= 9 McCabe cyclomatic complexity).
- In scripts/ci/tests/test_python_dependency_locks.py and scripts/ci/test_fail_closed_ci.py: add whole-repository and unit regression suites covering positive and negative controls.
- In scripts/ci/AGENTS.md: record checkout ordering invariant under ADR-1305.
Update the test-sycl-tidy-workflow-contract pre-commit hook files regex from test_(go|sycl)_workflow_contract to test_(go|sycl_tidy)_workflow_contract so pre-commit detects changes to scripts/ci/test_sycl_tidy_workflow_contract.py.

Strengthen test_sycl_tidy_workflow_contract.py with validation and mutation regressions ensuring the pre-commit hook entry and files selector cannot silently drift from the actual contract filename or its dependencies.
Hosted PR #1554 failed on Windows MSVC+CUDA, Windows ARM64 MSVC, and
Windows MSVC+CUDA full due to error C2065 ('nullptr': undeclared identifier)
in core/test/test_predict.c:387.

In accordance with ADR-1138 and core/test/AGENTS.md lines 491-498, C translation
units compiled by MSVC /std:clatest must spell the null pointer constant NULL
because MSVC does not implement the C23 nullptr keyword.

Replace return nullptr with return NULL at the test return site, and apply
the narrowest cited NOLINTNEXTLINE(modernize-use-nullptr) suppression citing
ADR-1138 to preserve the exact clang-tidy budget of 13 warnings for
test_predict.c and 0 for test_svm_api.c without modifying baseline files.
…closed

Enforce fail-closed Scorecard checkout validator semantics across both
PyYAML and fallback parser paths, resolving all six review blockers:

- Accept only the exact actions/checkout owner and action with a full 40-character
  hex SHA; reject spoofed owners, altered action names, unpinned tags, and short SHAs.
- Reject foreign with.repository checkouts as satisfying the local repository checkout.
- Reject conditional (if:) checkout steps as insufficient.
- Reject continue-on-error checkout steps as insufficient.
- Reject checkouts with with.path targeting a subdirectory as satisfying root-local consumers.
- Preserve folded YAML run blocks (> and >-) where -r and local paths split across physical lines in the fallback parser.
- Clear strict mypy errors around index type str | None and untyped return in fallback parser and job loader.
- Maintain strict compliance with NASA JPL Rule 4 (<= 22 LOC, <= 9 McCabe complexity per function).
- Add adversarial mutation tests for all six cases in both PyYAML and fallback parser modes.
- Document fail-closed checkout ordering invariants in scripts/ci/AGENTS.md under ADR-1305.
Resolve touched-file HISS-04 / NASA Rule 4 violations reported by
`praetorctl audit -base origin/master` in `core/src/feature/x86/vif_avx512.c`
for `vif_subsample_rd_8_vert_j` (111 LOC -> 45 LOC) and
`vif_subsample_rd_8_horiz_j` (132 LOC -> 36 LOC).

Under ADR-1289 and ADR-1298, the -touched-debt-delta-reason escape hatch
is explicitly rejected. Both outer functions maintain their ADR-0503
`static VMAF_NOINLINE_NOCLONE` register-pressure boundaries. Repetitive
unrolled operations are factored into bounded macros (`VIF_VERT_LOAD10_REF`,
`VIF_VERT_LOAD10_DIS`, `VIF_VERT_MADD5`, `VIF_HORIZ_TAP8`). Forced-inline
function helpers were proven to alter GCC SSA register allocation due to
address-taken vector pointer arguments (e.g. swapping %zmm3 and %zmm13),
whereas bounded macros preserve 100% byte-identical machine code in the hot
`.text` section (SHA-256: 80b48e27e202ca98c3a124351740fdecba1e19df53adeebbcd237889fe44374c).

Re-record `.standards-baseline.json` down from 276 to 274 infractions via
`praetorctl baseline -record`, update README governance debt block, and
correct false touched-file clean claims in Research-2098.

Verified:
  - `praetorctl audit -base origin/master`: 0 touched-file violations, pass.
  - `.text` SHA-256 exact match against base 43d reference object.
  - Public symbols exact match (`nm -g --defined-only`).
  - CodeQL replay: 0 `cpp/large-parameter` rows.
  - Focused VIF tests: 3/3 pass (`test_integer_vif_avx512_stages`, `test_vif_simd`, `test_vif_skip_scale0`).
  - Fast test suite: 145/145 pass (`meson test -C build --suite=fast`).
  - Pre-commit hooks: all pass.
…on (alerts 1275, 1276, 1239)

Close CodeQL alerts 1275, 1276, and 1239 across compat/python-vmaf/:
- In executor._run_fifo_worker, close error_sender and attach diagnostic
  context via add_note upon IPC delivery failure rather than swallowing.
- In executor._fifo_worker_failure, synthesize child traceback context upon
  error channel EOF / read failure to immediately trigger process join and
  runtime error propagation.
- In train_test_model.RegressorMixin._get_scatter_arrays, replace == None with
  elementwise identity check is None, cast to float, and zero NaNs safely.
- Audit tools/misc.py (check_scanf_match) and tools/scanf.py (isFileLike),
  adding explanatory comments and broadening stream exception handling.
- Add regression suites in python/test/executor_test.py, train_test_model_test.py,
  and python_harness_scanf_locale_bugs_test.py.
- Add Research-2080 digest, update AGENTS.md, docs/rebase-notes.md, CHANGELOG,
  and docs/state.md bug tracker.
- Annotate SCRIPT_PATH: Path and REPO_ROOT: Path in measure_quant_drop.py,
  ptq_dynamic.py, and ptq_static.py matching ADR-0681 conventions to reconcile
  silent-revert gate detection against d170ef8 without allowlisting.
- Add focused regression coverage in test_ptq_cli_contracts.py for CLI help,
  import isolation, and argv handling contracts across PTQ and quantization scripts.
@lusoris
lusoris enabled auto-merge (squash) September 26, 2026 14:13
Comment thread core/src/gpu_picture_pool.cpp Fixed
…build

Fix hosted-CI regressions across Linux and macOS runners:
1. Build vmaf-perShot-boundary-test by default (build_by_default: true)
   in core/tools/meson.build so it is built during default and install
   targets before test suites execute under --no-rebuild.
2. In core/test/meson.build, raise test_meson_secret_env_sanitization
   timeout from 30s to 120s to align with ADR-1333's probe deadline and
   prevent premature SIGTERM.
3. In core/test/test_meson_secret_env_sanitization.py, memoize entrypoint
   command parsing via lru_cache, cache file reads, and filter raw commands
   with a substring check, dropping test suite execution from ~58s to ~4s.
Comment thread compat/python-vmaf/tests/test_decorator_extended.py Fixed
Comment thread compat/python-vmaf/core/executor.py Fixed
Comment thread core/src/feature/feature_extractor.cpp Dismissed
Comment thread core/src/feature/feature_extractor.cpp Dismissed
Comment thread core/test/test_feature_collector.c Dismissed
Comment thread compat/python-vmaf/core/executor.py Dismissed
Tidy Changed failed on two real errors. fex_ctx_vector.h opened its
extern "C" block before including feature_extractor.h, which pulls in
<atomic> in C++ mode, so linting the header as C++ put templates under
C linkage. The include now sits outside the block, as it does in the 21
C++ files that already include feature_extractor.h directly.
test_metal_integer_psnr_parity.c passed an internal VmafDictionary to
vmaf_use_feature() and then freed a dictionary the call had already
consumed; it now builds a VmafFeatureDictionary and follows the
ownership contract.

Tidy Ratchet failed because four files improved without a tightened
baseline. The locally re-recorded baseline disagreed with CI on one
diagnostic only: under GCC 16 headers clang-tidy flags
assert(d->size > 0) in dict_grow_entries as a static_assert candidate.
That assert is replaced by the hard guard feature_extractor.cpp already
uses for the same doubling invariant, because under NDEBUG a zero size
would let the caller write past a zero-byte allocation. A test drives
the zero-capacity path through vmaf_dictionary_set(); it aborted on the
old assert and now gets -EINVAL. The baseline is regenerated with
make tidy-ratchet-write (620 -> 613) and now matches CI's GCC 15
measurement file for file.
Eleven harness tests failed on every Ubuntu and macOS tox lane because
golden_gate_isolation_test.py re-imported the vmaf package with
importlib.reload. reload() re-finds the spec by name, which resolves to
the python/vmaf shim; the shim installs a fresh module in sys.modules
and rewrites the original's __spec__. Later tests that patch
vmaf.run_process then miss the module their imported names are bound
to and execute the real command. The import-time path checks now run
in a child interpreter, which observes the same resolution without
shared state.

One macOS lane also failed raw_extractor_test.py because the bounded
FIFO startup wait tried each readiness permit before it checked for
child exit. AssetExtractor's producer releases its permit and returns at
once, so it could do both between those two steps and be reported as
exiting without signaling. The wait now observes exit first; since a
child releases before closing its error channel, a signalled exit is
always seen as ready. Deterministic regressions cover both
interleavings. The race predates this train, so docs/state.md records
it as T-PY-FIFO-READY-EXIT-RACE-2026-09-26 with a changelog fragment.

CodeQL py/empty-except alerts 1290 and 1291 are resolved with the
explanatory comments the query requires: the externals module is an
optional local override.
@lusoris
lusoris merged commit dd51d00 into master Sep 26, 2026
119 checks passed
@lusoris
lusoris deleted the train/pre-rc1-correctness-20260924 branch September 26, 2026 18:25
lusoris added a commit that referenced this pull request Sep 27, 2026
The nightly Full clang-tidy scan failed because tidy-ratchet.py measured
meson's generated model embeds. The nightly builds in build/ inside the
repository, and the ratchet treated every path under the repository root as
a checked-in source. It therefore measured the 18 xxd outputs
build/src/*.json.c and build/src/brisque_live.model.c, each with two
misc-use-internal-linkage warnings on `unsigned char src_*[]` and
`unsigned int src_*_len`. Run 36308945712 reported 649 warnings against a
baseline of 613 and exited 2.

PR #1561 re-recorded the cpu baseline without those files and moved only
the required Tidy Ratchet build to $RUNNER_TEMP. Before that, the nightly's
exit-4 compile failures (-flto=4, then the core/tools TUs) had hidden the
drift since 2026-09-01.

ADR-1142 exempts generated files. The ratchet now enforces that itself and
no longer depends on where the build directory lives. Every translation
unit, diagnostic and header under --build-dir is skipped, so in-tree and
out-of-tree builds measure the same checked-in sources. That also covers
make tidy-ratchet and tidy-ratchet-write, whose default build dir is the
in-tree core/build.

The arm64 baseline was the only one recorded from an in-tree build
(build-arm64). The writer re-measured it on its recorded toolchain
(aarch64-linux-gnu-gcc 16.1.0, clang-tidy 22.1.8), taking it from 764 to 615
warnings: the 36 generated-file warnings, 25 Pelorus-mirror entries the
ratchet already ignores, and 88 warnings in checked-in files cleaned since
2026-09-23. No count rose.

Also updated: the Makefile, docs/development/ci.md and scripts/ci/AGENTS.md,
which described build-dir placement inconsistently (in-tree required vs
out-of-tree required).

Verification, from an in-tree build on the cpu baseline's own toolchain
(gcc 16.2.1, clang-tidy 22.1.8):
- before: 324 TUs, 649 warnings (baseline 613), exit 2, same 18 files as CI
- after: 306 TUs, 613 warnings, TU set and per-file counts equal the
  baseline, exit 0
- arm64 after the rewrite: 284 TUs, 615 warnings (baseline 615), exit 0
- test_tidy_ratchet.py: 40 tests OK. Against the unfixed script the new
  tests fail (2 failures, 6 errors), including build/src/vmaf_v0.6.1.json.c
  appearing in the measured set.
lusoris added a commit that referenced this pull request Sep 27, 2026
The nightly ThreadSanitizer job went red on 2026-09-27 (run 36308945712,
master 8421376). TSan reported no race; one test failed:
test_meson_secret_env_sanitization, in all five probes that start Meson,
with "No module named 'mesonbuild'" raised by /home/runner/.local/bin/meson.

The test was added in dd51d00 (#1561), and this was the first nightly to
include it. Its probes run meson under a synthetic environment whose HOME is
a temporary directory. Python derives its per-user site directory from HOME,
and the nightly job installs Meson with `pip install --user` through
scripts/setup/ubuntu.sh, so the Meson console script could no longer import
its own package. The required lanes install Meson system-wide and never hit
this; the documented Ubuntu developer setup does.

The probe environment now also sets PYTHONUSERBASE to the user base the
test interpreter resolved. HOME stays synthetic, and no other host value is
copied. Two new cases put a package only in a HOME-derived user base and in
an explicit PYTHONUSERBASE outside HOME, and require a child started with
the probe environment to import it. The same child without the pinned value
serves as the failing control, which is the pre-fix environment.

Reproduced with Meson installed by `pip install --user` into a HOME-derived
user base under a standalone CPython with no system mesonbuild: the same
five failures before the fix, and 31 tests OK / 1 skip after it. A local
TSan build with CI's configuration and invocation went from 1 failure to
189 ok / 0 fail / 1 skip.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

2 participants