Skip to content

test(ai): coverage push — 2 new test files lifting online_trainer + feature_extractor from 0%/partial to 85%/94% - #578

Merged
lusoris merged 1 commit into
masterfrom
test/ai-coverage-push
Jun 3, 2026
Merged

lusoris merged 1 commit into
masterfrom
test/ai-coverage-push

Conversation

@lusoris

@lusoris lusoris commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds ai/sidecar/tests/test_online_trainer.py (30 tests) — lifts online_trainer.py from 0% → 85% coverage.
  • Adds ai/tests/test_feature_extractor_unit.py (26 tests) — lifts feature_extractor.py from partial → 94% coverage.
  • Adds changelog.d/added/ai-sidecar-online-trainer-coverage.md fragment.

Both modules were identified as the highest-LOC, lowest-covered targets in the ai/ subtree.

Type

  • test — test-only

Checklist

Bug-status hygiene

no state delta: test-only change, no bugs opened or closed.

Netflix golden-data gate

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.

Cross-backend numerical results

no cross-backend diff needed: test-only, no C source touched.

Deep-dive deliverables

  • Research digest — no digest needed: trivial test-only coverage push.
  • Decision matrix — no alternatives: only-one-way fix (write missing tests).
  • AGENTS.md invariant note — no rebase-sensitive invariants: test files only.
  • Reproducer / smoke-test command — see below.
  • CHANGELOG fragment — changelog.d/added/ai-sidecar-online-trainer-coverage.md.
  • Rebase note — no rebase impact: test files only, no C/public-header/upstream-mirror change.

Reproducer

cd ai
python -m pytest tests/test_feature_extractor_unit.py sidecar/tests/test_online_trainer.py -v
# Expected: 56 passed

Coverage delta

Module Before After
ai/sidecar/online_trainer.py 0% 85%
ai/data/feature_extractor.py ~40% (registry/resolver only) 94%

What is tested

test_online_trainer.py (30 tests)

  • _build_fallback_model — output shape, accepts different n_features
  • _load_base_model — empty path fallback, absent file fallback, PyTorch state-dict load, corrupt file fallback
  • _write_sha256_sidecar — file created, valid 64-char hex, known SHA-256 value, atomic write (no .tmp leftover)
  • OnlineTrainer.__init__ — construction, checkpoint dir creation, custom buffer capacity
  • OnlineTrainer.ingest — accumulation below batch size, buffer fill, batch triggers gradient step, step count increments, checkpoint path returned, SHA-256 sidecar present
  • OnlineTrainer.status — all keys present, checkpoint_counter increments
  • _handle_connection — valid message returns ok, job_id echoed, missing features key returns error, malformed JSON returns error, empty line skipped, multiple messages on one connection
  • run_server — starts, accepts connections, stops cleanly and unlinks socket

test_feature_extractor_unit.py (26 tests)

  • _ensure_binary — raises RuntimeError when absent, passes when file exists
  • _lookup — direct key, integer_ prefix fallback, absent returns None, direct preferred over integer_ prefix
  • _run_vmaf_json — argv composition (flags, pix_fmt, bitdepth), feature extractor flags, temp file cleanup on subprocess error
  • extract_features — happy path with 2 frames, NaN for missing metrics, integer_ fallback, empty frames, raises when binary absent, respects VMAF_BIN env
  • FeatureExtractionResult — JSON round-trip, dtype preserved
  • aggregate_clip_stats — shape is 4×n_features, correct mean, p10 < p90, NaN ignored, all-NaN propagates, empty frames returns NaN vector, raises on 1-D input, output is float32, single frame

🤖 Generated with Claude Code

…eature_extractor

Add 56 new tests across two previously untested modules:

- ai/sidecar/tests/test_online_trainer.py (30 tests):
  covers OnlineTrainer construction, ingest/batch/checkpoint lifecycle,
  _write_sha256_sidecar atomicity, _build_fallback_model / _load_base_model
  happy + error paths, _handle_connection JSON wire protocol (happy/bad-key/
  malformed/empty-line/multi-message), and run_server start+stop.
  Module coverage: online_trainer.py 0% → 85%.

- ai/tests/test_feature_extractor_unit.py (26 tests):
  covers _ensure_binary raise + pass, _lookup direct/integer_-fallback/absent,
  _run_vmaf_json argv composition/feature-flags/pix_fmt/cleanup-on-error,
  extract_features happy/NaN/integer_-fallback/empty-frames/missing-binary/
  env-binary paths, FeatureExtractionResult round-trip, and aggregate_clip_stats
  shape/mean/p10-p90/NaN-ignore/all-NaN/empty-frames/bad-ndim/float32/single-frame.
  Module coverage: feature_extractor.py partial → 94%.

semgrep skipped locally: pre-commit venv semgrep-core hits io_uring ENOMEM
(ulimit -l 8 MiB on CachyOS); system semgrep passes cleanly; CI container
runs with unlimited memlock. Same pre-existing failure as PR #481 / #457 / #484.
All other pre-commit hooks pass. Tests: 56/56 green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

# Conflicts:
#	CONTRIBUTING.md
#	README.md
#	ai/scripts/train_konvid.py
#	core/src/dnn/AGENTS.md
#	core/src/dnn/ort_backend.c
#	core/src/dnn/ort_backend_internal.h
#	core/src/feature/feature_extractor.cpp
#	docs/adr/0761-cpp23-wave8-opt-read-json-model.md
#	docs/adr/0763-cuda-adm-decouple-ldg.md
#	docs/adr/0771-simd-twin-inventory.md
#	docs/adr/0772-feature-extractor-cpp-rename.md
#	docs/adr/0777-thread-safety-audit-gpu-backends.md
#	docs/adr/0779-ebpf-fuse-bypass.md
#	docs/adr/0780-nolint-cluster-refactor.md
#	docs/adr/0793-nightly-workflow-audit.md
#	docs/adr/0802-ci-runner-image-standardization.md
#	docs/adr/0809-cli-cpp23-conversion.md
#	docs/adr/0812-renovate-go-rust-scheduling.md
#	docs/adr/0815-operator-node-distroless-dockerfiles.md
#	docs/adr/0844-float-adm-avx2-512-f2-f3.md
#	docs/adr/0852-hip-speed-extractor-wiring.md
#	docs/adr/0858-cpp23-gpu-dispatch-env.md
#	docs/adr/0865-ansnr-sunset-pre-vmaf-metric-drop.md
#	docs/adr/0879-python-dep-freshness-2026-05-30.md
#	docs/adr/0881-coverage-overrides-audit-2026-05-30.md
#	docs/adr/0888-pyright-strict-audit.md
#	docs/adr/0946-sycl-kernel-coverage-round3.md
#	docs/adr/0957-sycl-kernel-coverage-round4.md
#	docs/adr/README.md
#	docs/ai/models/learned_filter_v1.md
#	docs/backends/operator.md
#	docs/development/ebpf-fuse-bypass.md
#	docs/k8s/integration-tests.md
#	docs/rebase-notes.md
#	docs/research/0763-cuda-adm-decouple-ldg-f3-fix.md
#	docs/research/0881-coverage-overrides-audit-2026-05-30.md
#	docs/research/0946-sycl-kernel-coverage-round3-2026-05-31.md
#	docs/research/0957-sycl-kernel-coverage-round4-2026-05-31.md
#	docs/research/dependency-audit-2026-05-30.md
#	docs/research/nolint-cluster-audit-2026-05-29.md
#	docs/research/pyright-strict-audit-2026-05-30.md
#	docs/research/python-dep-freshness-2026-05-30.md
#	docs/research/python-type-annotations-audit-20260530.md
#	docs/research/research-0787-libvmaf-api-error-path-audit.md
#	docs/research/simd-twin-inventory-2026-05-29.md
#	docs/research/thread-safety-audit-backends-2026-05-29.md
#	docs/state.md
#	mkdocs.yml
#	pyproject.toml
@lusoris
lusoris force-pushed the test/ai-coverage-push branch from 8a4684e to c36a3a0 Compare June 3, 2026 22:17
@lusoris
lusoris marked this pull request as ready for review June 3, 2026 22:17
Copilot AI review requested due to automatic review settings June 3, 2026 22:17
@lusoris
lusoris merged commit 4f42702 into master Jun 3, 2026
34 of 92 checks passed
@lusoris
lusoris deleted the test/ai-coverage-push branch June 3, 2026 22:17
@lusoris
lusoris removed the request for review from Copilot June 3, 2026 22:39
@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 4, 2026
lusoris added a commit that referenced this pull request Sep 30, 2026
…ocumentation gate

The standards gate now installs praetor main 25451d8 from the remote
module proxy (PRAETOR_REF was f41e74d on master). Every praetor-managed
file is regenerated with that engine's own code: adopt (with
--verification-max-entries 200000), sync, compile-context and
devcontainer --source-root at the pin, which keeps the vmafx-dev-mcp
base image.

Baseline (ADR-1351): f41e74d re-records 185 entries on this tree with no
growth, 25451d8 records 431. All 246 new fingerprints are engine
changes, replayed on the same tree against each commit's parent:
d7a3778 adds 61 process exits from library code (Python 51, Go 5,
Rust 5), 025bbc6 adds 142 long Python functions and 36 recursive ones,
53e7594 adds 7 Go calls without a deadline. They are recorded with
--allow-increase and a reason.

Documentation gate: praetor fixed cordanaLLM/praetor#532, #533 and #534,
so the gate now passes here. .standards.yaml raises max_files to 8192
and max_file_bytes to 4 MiB, and style-excludes the generated ADR
index, the ADR row fragments and testdata fixtures, the files the
repository's own markdownlint hook already skips. The other 75
findings are fixed in the source: the 14 predictor model cards and
their template in predictor_train.py (pinned by
test_predictor_card_markdown.py), an MD013 re-enable in the SYCL
overview and a fence language in the ADR fragments README. The figure
engine under tools/figures/, the docs-figures target, a .gitattributes
block and a workflow step arrive with it.

Praetor defects, filed or tracked upstream and worked around here:
- adopt still refuses to extend the Makefile block because of computed
  targets such as $(BUILD_DIR): (#537, open). GNU Make finds no
  docs-lint or docs-figures rule outside the block, so the block is
  praetor's own DocumentationMakefileBlock() text.
- The repository-wide dist/ rule hid tools/figures/dist/; .gitignore
  re-includes it (#591).
- black, ruff and markdownlint would rewrite or flag the locked
  figure engine; their pre-commit hooks skip tools/figures/ (#578).
- Praetor requires the retired numbered workspace root to be ignored
  (#641, filed with this change). The ADR-1277 contract check accepts
  only that rule inside praetor's block, skips tools/markdownlint/ when
  scanning for references and still fails on a local directory. ADR-1351
  amends ADR-1277.

Also: repository.default_branch: master renders the ruleset for master;
REUSE.toml labels the vendored interfig sources and the player bundle;
adopt's praetorctl pre-tool hook registrations in .claude/settings.json,
.codex/hooks.json and .gemini/settings.json are left out, since audit
does not verify them and they change every agent session.
T-PRAETOR-DOCS-GATE-LIMITS-2026-09-28 is closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 30, 2026
…ocumentation gate

The standards gate now installs praetor main 25451d8 from the remote
module proxy (PRAETOR_REF was f41e74d on master). Every praetor-managed
file is regenerated with that engine's own code: adopt (with
--verification-max-entries 200000), sync, compile-context and
devcontainer --source-root at the pin, which keeps the vmafx-dev-mcp
base image.

Baseline (ADR-1351): f41e74d re-records 185 entries on this tree with no
growth, 25451d8 records 431. All 246 new fingerprints are engine
changes, replayed on the same tree against each commit's parent:
d7a3778 adds 61 process exits from library code (Python 51, Go 5,
Rust 5), 025bbc6 adds 142 long Python functions and 36 recursive ones,
53e7594 adds 7 Go calls without a deadline. They are recorded with
--allow-increase and a reason.

Documentation gate: praetor fixed cordanaLLM/praetor#532, #533 and #534,
so the gate now passes here. .standards.yaml raises max_files to 8192
and max_file_bytes to 4 MiB, and style-excludes the generated ADR
index, the ADR row fragments and testdata fixtures, the files the
repository's own markdownlint hook already skips. The other 75
findings are fixed in the source: the 14 predictor model cards and
their template in predictor_train.py (pinned by
test_predictor_card_markdown.py), an MD013 re-enable in the SYCL
overview and a fence language in the ADR fragments README. The figure
engine under tools/figures/, the docs-figures target, a .gitattributes
block and a workflow step arrive with it.

Praetor defects, filed or tracked upstream and worked around here:
- adopt still refuses to extend the Makefile block because of computed
  targets such as $(BUILD_DIR): (#537, open). GNU Make finds no
  docs-lint or docs-figures rule outside the block, so the block is
  praetor's own DocumentationMakefileBlock() text.
- The repository-wide dist/ rule hid tools/figures/dist/; .gitignore
  re-includes it (#591).
- black, ruff and markdownlint would rewrite or flag the locked
  figure engine; their pre-commit hooks skip tools/figures/ (#578).
- Praetor requires the retired numbered workspace root to be ignored
  (#641, filed with this change). The ADR-1277 contract check accepts
  only that rule inside praetor's block, skips tools/markdownlint/ when
  scanning for references and still fails on a local directory. ADR-1351
  amends ADR-1277.

Also: repository.default_branch: master renders the ruleset for master;
REUSE.toml labels the vendored interfig sources and the player bundle;
adopt's praetorctl pre-tool hook registrations in .claude/settings.json,
.codex/hooks.json and .gemini/settings.json are left out, since audit
does not verify them and they change every agent session.
T-PRAETOR-DOCS-GATE-LIMITS-2026-09-28 is closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 30, 2026
…ocumentation gate

The standards gate now installs praetor main 25451d8 from the remote
module proxy (PRAETOR_REF was f41e74d on master). Every praetor-managed
file is regenerated with that engine's own code: adopt (with
--verification-max-entries 200000), sync, compile-context and
devcontainer --source-root at the pin, which keeps the vmafx-dev-mcp
base image.

Baseline (ADR-1351): f41e74d re-records 185 entries on this tree with no
growth, 25451d8 records 431. All 246 new fingerprints are engine
changes, replayed on the same tree against each commit's parent:
d7a3778 adds 61 process exits from library code (Python 51, Go 5,
Rust 5), 025bbc6 adds 142 long Python functions and 36 recursive ones,
53e7594 adds 7 Go calls without a deadline. They are recorded with
--allow-increase and a reason.

Documentation gate: praetor fixed cordanaLLM/praetor#532, #533 and #534,
so the gate now passes here. .standards.yaml raises max_files to 8192
and max_file_bytes to 4 MiB, and style-excludes the generated ADR
index, the ADR row fragments and testdata fixtures, the files the
repository's own markdownlint hook already skips. The other 75
findings are fixed in the source: the 14 predictor model cards and
their template in predictor_train.py (pinned by
test_predictor_card_markdown.py), an MD013 re-enable in the SYCL
overview and a fence language in the ADR fragments README. The figure
engine under tools/figures/, the docs-figures target, a .gitattributes
block and a workflow step arrive with it.

Praetor defects, filed or tracked upstream and worked around here:
- adopt still refuses to extend the Makefile block because of computed
  targets such as $(BUILD_DIR): (#537, open). GNU Make finds no
  docs-lint or docs-figures rule outside the block, so the block is
  praetor's own DocumentationMakefileBlock() text.
- The repository-wide dist/ rule hid tools/figures/dist/; .gitignore
  re-includes it (#591).
- black, ruff and markdownlint would rewrite or flag the locked
  figure engine; their pre-commit hooks skip tools/figures/ (#578).
- Praetor requires the retired numbered workspace root to be ignored
  (#641, filed with this change). The ADR-1277 contract check accepts
  only that rule inside praetor's block, skips tools/markdownlint/ when
  scanning for references and still fails on a local directory. ADR-1351
  amends ADR-1277.

Also: repository.default_branch: master renders the ruleset for master;
REUSE.toml labels the vendored interfig sources and the player bundle;
adopt's praetorctl pre-tool hook registrations in .claude/settings.json,
.codex/hooks.json and .gemini/settings.json are left out, since audit
does not verify them and they change every agent session.
T-PRAETOR-DOCS-GATE-LIMITS-2026-09-28 is closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 30, 2026
…ocumentation gate

The standards gate now installs praetor main 25451d8 from the remote
module proxy (PRAETOR_REF was f41e74d on master). Every praetor-managed
file is regenerated with that engine's own code: adopt (with
--verification-max-entries 200000), sync, compile-context and
devcontainer --source-root at the pin, which keeps the vmafx-dev-mcp
base image.

Baseline (ADR-1351): f41e74d re-records 185 entries on this tree with no
growth, 25451d8 records 431. All 246 new fingerprints are engine
changes, replayed on the same tree against each commit's parent:
d7a3778 adds 61 process exits from library code (Python 51, Go 5,
Rust 5), 025bbc6 adds 142 long Python functions and 36 recursive ones,
53e7594 adds 7 Go calls without a deadline. They are recorded with
--allow-increase and a reason.

Documentation gate: praetor fixed cordanaLLM/praetor#532, #533 and #534,
so the gate now passes here. .standards.yaml raises max_files to 8192
and max_file_bytes to 4 MiB, and style-excludes the generated ADR
index, the ADR row fragments and testdata fixtures, the files the
repository's own markdownlint hook already skips. The other 75
findings are fixed in the source: the 14 predictor model cards and
their template in predictor_train.py (pinned by
test_predictor_card_markdown.py), an MD013 re-enable in the SYCL
overview and a fence language in the ADR fragments README. The figure
engine under tools/figures/, the docs-figures target, a .gitattributes
block and a workflow step arrive with it.

Praetor defects, filed or tracked upstream and worked around here:
- adopt still refuses to extend the Makefile block because of computed
  targets such as $(BUILD_DIR): (#537, open). GNU Make finds no
  docs-lint or docs-figures rule outside the block, so the block is
  praetor's own DocumentationMakefileBlock() text.
- The repository-wide dist/ rule hid tools/figures/dist/; .gitignore
  re-includes it (#591).
- black, ruff and markdownlint would rewrite or flag the locked
  figure engine; their pre-commit hooks skip tools/figures/ (#578).
- Praetor requires the retired numbered workspace root to be ignored
  (#641, filed with this change). The ADR-1277 contract check accepts
  only that rule inside praetor's block, skips tools/markdownlint/ when
  scanning for references and still fails on a local directory. ADR-1351
  amends ADR-1277.

Also: repository.default_branch: master renders the ruleset for master;
REUSE.toml labels the vendored interfig sources and the player bundle;
adopt's praetorctl pre-tool hook registrations in .claude/settings.json,
.codex/hooks.json and .gemini/settings.json are left out, since audit
does not verify them and they change every agent session.
T-PRAETOR-DOCS-GATE-LIMITS-2026-09-28 is closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant