Skip to content

fix(vmaf-tune): address report audit findings #2-#10 and document SVT-AV1-HDR knobs - #1296

Merged
lusoris merged 3 commits into
masterfrom
fix/vmaf-tune-report-audit-and-svtav1-hdr-knob-docs
Sep 6, 2026
Merged

lusoris merged 3 commits into
masterfrom
fix/vmaf-tune-report-audit-and-svtav1-hdr-knob-docs

Conversation

@lusoris

@lusoris lusoris commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Lands the remaining nine findings (#2–#10) of docs/research/vmaftune-profile-report-audit-2026-05-27.md in the vmaf-tune report renderer (PR #852 had fixed only #1 and the 480px media query) and adds the SVT-AV1-HDR knob documentation that T-SVTAV1-HDR-ADAPTER-2026-05-20 still lacked. report.py: explicit kbps / Mbps axis and tick units, failed rows (ok=False) render — instead of 0 kbps / 0.00, VideoToolbox codecs get their own palette slots instead of aliasing libx264/libx265/libsvtav1, pareto annotations are deduplicated to the lowest-bitrate frontier point per codec with a bitrate suffix, the picked-CRF scatter and failed targets appear in a deduplicated legend, and savefig(..., metadata={"Date": None}) + a pinned svg.hashsalt make HTML/SVG output byte-identical across renders. cli.py: --json-sidecar on compare and report writes <output>.json (ReportData.to_dict()) next to the HTML/Markdown; ReportData.from_dict round-trips it. Docs: a new "SVT-AV1-HDR tuning knobs" section in docs/usage/vmaf-tune.md whose ranges and defaults were taken from juliobbv-p/svt-av1-hdr README + Docs/Parameters.md at commit 0033340 (2026-09-01, verified via the GitHub API), plus the --json-sidecar row and a cross-link in the libsvtav1 adapter row. Verification: cd tools/vmaf-tune && ruff check . && pytest tests/test_report.py tests/test_compare.py -q → All checks passed! / 40 passed; concat-changelog-fragments.sh --check exit 0; assertion-density.sh PASS (256 asserts / 151 functions); pre-commit green on every touched file. Not verified: an end-to-end encode against a live SVT-AV1-HDR FFmpeg build (no such binary on the workstation); the knob table is documentation of upstream defaults, not a behaviour the fork tests.

Type

Checklist

  • Commits follow Conventional Commits.
  • make format && make lint is green locally (pre-commit on every touched file).
  • Unit tests: tools/vmaf-tune/tests/test_report.py (+7 tests, 4 new fixtures: compare v1/v2, ladder, per-shot) and tests/test_compare.py — 40 passed.
  • Docs in the same PR: docs/usage/vmaf-tune.md (SVT-AV1-HDR tuning knobs section, --json-sidecar CLI row), docs/usage/vmaf-tune-codec-adapters.md (libsvtav1 row).
  • SIMD/GPU, twins, new C sources, breaking change, ADR — n/a: Python renderer/CLI + docs only; no ADR (bug fixes + documentation, ADR-0644 / ADR-0294 already cover the design).

Bug-status hygiene (ADR-0165)

  • docs/state.md — T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 and T-SVTAV1-HDR-ADAPTER-2026-05-20 moved from Open to Recently closed, two _Updated: lines added.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: this PR closes out findings chore(docs): update mkdocs site_url to vmafx.github.io/vmafx #2-chore(deps): Update rocm/rocm-terminal Docker tag to v6.4 #10 of the existing audit docs/research/vmaftune-profile-report-audit-2026-05-27.md; no new research was required.
  • Decision matrix — no alternatives: only-one-way fix. The audit prescribes each fix; the runtime-variant decision is already recorded in ADR-0644.
  • AGENTS.md invariant note — no rebase-sensitive invariants: fork-only tools/vmaf-tune Python and docs.
  • Reproducer / smoke-test command — below.
  • CHANGELOG fragment — changelog.d/fixed/vmaftune-report-audit-findings.md, changelog.d/added/vmaftune-svtav1-hdr-knob-docs.md
  • Rebase note — docs/rebase-notes.md entry (fix/vmaf-tune-report-audit-and-svtav1-hdr-knob-docs, no rebase impact).

Reproducer

source .venv/bin/activate
cd tools/vmaf-tune && ruff check . && pytest tests/test_report.py tests/test_compare.py -q
# All checks passed!
# 40 passed
python - <<'PY'
from vmaftune.report import _bitrate_tick_label, _CODEC_COLOURS
from vmaftune.codec_adapters import known_codecs
assert _bitrate_tick_label(1500) == "1.5 Mbps" and _bitrate_tick_label(800) == "800 kbps"
assert _CODEC_COLOURS["h264_videotoolbox"] != _CODEC_COLOURS["libx264"]
assert "libsvtav1" in known_codecs() and "svtav1-hdr" not in known_codecs()
print("ok")
PY
# ok
cd ../.. && bash scripts/release/concat-changelog-fragments.sh --check && echo changelog-ok
# changelog-ok

🤖 Generated with Claude Code

@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 5, 2026
@lusoris
lusoris force-pushed the fix/vmaf-tune-report-audit-and-svtav1-hdr-knob-docs branch 3 times, most recently from 975127b to 0b30fc0 Compare September 6, 2026 03:03
lusoris and others added 3 commits September 6, 2026 06:56
…-AV1-HDR knobs

- Land profile report audit findings #2-#10 in report.py and cli.py:
  - #2: Unify bitrate axis labels and tick formatting (Mbps/kbps).
  - #3: Render em-dash for failed rows with 0.0 values in HTML/Markdown.
  - #4: Assign VideoToolbox encoders to distinct palette slots (15-17).
  - #5 & #8: Deduplicate pareto annotations to lowest-bitrate point with bitrate.
  - #6: Add --json-sidecar CLI flag and ReportData.from_dict round-trip.
  - #7: Add picked CRF label to scatter plot and deduplicate legend entries.
  - #9: Strip timestamp and pin svg.hashsalt for byte-identical rendering.
  - #10: Add failed target markers and failure annotations to sweep chart.
- Document SVT-AV1-HDR tuning knobs and libsvtav1@svt-av1-hdr runtime variant
  in docs/usage/vmaf-tune.md and docs/usage/vmaf-tune-codec-adapters.md.
- Add comprehensive regression tests in tools/vmaf-tune/tests/test_report.py.
- Update docs/state.md, docs/rebase-notes.md, and changelog fragments.
…arameters.md

The first cut of the knob table carried three defaults that contradict
juliobbv-p/svt-av1-hdr Docs/Parameters.md @ 0033340 (tune=1 not 0,
sharp-tx=1 not 0, noise-adaptive-filtering=2 not 0) and omitted twelve
documented keys. Rebuild the table from the upstream parameter reference,
state the three injection points for the -svtav1-params string and the
ADR-0294 CRF/preset window the variant inherits, and drop the
'this PR' placeholders from docs/state.md so the ADR-0165 touch gate
accepts the rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each dropped row restates one origin/master already carries; master is the
authoritative record. Verified with scripts/ci/check-state-md-rows.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/vmaf-tune-report-audit-and-svtav1-hdr-knob-docs branch from 0b30fc0 to 77b8150 Compare September 6, 2026 04:56
@lusoris
lusoris marked this pull request as ready for review September 6, 2026 04:56
@lusoris
lusoris merged commit e335857 into master Sep 6, 2026
121 of 123 checks passed
@lusoris
lusoris deleted the fix/vmaf-tune-report-audit-and-svtav1-hdr-knob-docs branch September 6, 2026 05:22
@lusoris lusoris added the type:bug Something isn't working label Sep 7, 2026
lusoris added a commit that referenced this pull request Sep 8, 2026
#1238)

Fix issue #1238 by auditing all rows in docs/state.md ## Open bugs:
- Move 4 confirmed resolved bug tracking rows to ## Recently closed:
  * T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 (PR #1340, commit b43fc44)
  * T-AI-PTQ-STATIC-QUANT-FORMAT-UNPINNED-2026-09-03 (PR #1306, commit aebb9e1)
  * T-SVTAV1-HDR-ADAPTER-2026-05-20 (PR #1296, commit e335857)
  * T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 (PR #1296, commit e335857)
- Restore 2 closed rows accidentally dropped during dedup in commit 02f9ee4:
  * T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 (PR #1294, commit 8d103e3)
  * T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 (PR #1224, commit 6c843bb)
- Clean up duplicate tombstone comments in ## Open bugs.
- Add changelog fragment changelog.d/changed/docs-1238-state-md-stale-rows.md
  and regenerate CHANGELOG.md via concat-changelog-fragments.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 8, 2026
#1238)

Fix issue #1238 by auditing all rows in docs/state.md ## Open bugs:
- Move 4 confirmed resolved bug tracking rows to ## Recently closed:
  * T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 (PR #1340, commit b43fc44)
  * T-AI-PTQ-STATIC-QUANT-FORMAT-UNPINNED-2026-09-03 (PR #1306, commit aebb9e1)
  * T-SVTAV1-HDR-ADAPTER-2026-05-20 (PR #1296, commit e335857)
  * T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 (PR #1296, commit e335857)
- Restore 2 closed rows accidentally dropped during dedup in commit 02f9ee4:
  * T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 (PR #1294, commit 8d103e3)
  * T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 (PR #1224, commit 6c843bb)
- Clean up duplicate tombstone comments in ## Open bugs.
- Add changelog fragment changelog.d/changed/docs-1238-state-md-stale-rows.md
  and regenerate CHANGELOG.md via concat-changelog-fragments.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 8, 2026
#1238)

Fix issue #1238 by auditing all rows in docs/state.md ## Open bugs:
- Move 4 confirmed resolved bug tracking rows to ## Recently closed:
  * T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 (PR #1340, commit b43fc44)
  * T-AI-PTQ-STATIC-QUANT-FORMAT-UNPINNED-2026-09-03 (PR #1306, commit aebb9e1)
  * T-SVTAV1-HDR-ADAPTER-2026-05-20 (PR #1296, commit e335857)
  * T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 (PR #1296, commit e335857)
- Restore 2 closed rows accidentally dropped during dedup in commit 02f9ee4:
  * T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 (PR #1294, commit 8d103e3)
  * T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 (PR #1224, commit 6c843bb)
- Clean up duplicate tombstone comments in ## Open bugs.
- Add changelog fragment changelog.d/changed/docs-1238-state-md-stale-rows.md
  and regenerate CHANGELOG.md via concat-changelog-fragments.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 15, 2026
#1238)

Fix issue #1238 by auditing all rows in docs/state.md ## Open bugs:
- Move 4 confirmed resolved bug tracking rows to ## Recently closed:
  * T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 (PR #1340, commit b43fc44)
  * T-AI-PTQ-STATIC-QUANT-FORMAT-UNPINNED-2026-09-03 (PR #1306, commit aebb9e1)
  * T-SVTAV1-HDR-ADAPTER-2026-05-20 (PR #1296, commit e335857)
  * T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 (PR #1296, commit e335857)
- Restore 2 closed rows accidentally dropped during dedup in commit 02f9ee4:
  * T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 (PR #1294, commit 8d103e3)
  * T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 (PR #1224, commit 6c843bb)
- Clean up duplicate tombstone comments in ## Open bugs.
- Add changelog fragment changelog.d/changed/docs-1238-state-md-stale-rows.md
  and regenerate CHANGELOG.md via concat-changelog-fragments.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 15, 2026
#1238)

Fix issue #1238 by auditing all rows in docs/state.md ## Open bugs:
- Move 4 confirmed resolved bug tracking rows to ## Recently closed:
  * T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 (PR #1340, commit b43fc44)
  * T-AI-PTQ-STATIC-QUANT-FORMAT-UNPINNED-2026-09-03 (PR #1306, commit aebb9e1)
  * T-SVTAV1-HDR-ADAPTER-2026-05-20 (PR #1296, commit e335857)
  * T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 (PR #1296, commit e335857)
- Restore 2 closed rows accidentally dropped during dedup in commit 02f9ee4:
  * T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 (PR #1294, commit 8d103e3)
  * T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 (PR #1224, commit 6c843bb)
- Clean up duplicate tombstone comments in ## Open bugs.
- Add changelog fragment changelog.d/changed/docs-1238-state-md-stale-rows.md
  and regenerate CHANGELOG.md via concat-changelog-fragments.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 15, 2026
#1238)

Fix issue #1238 by auditing all rows in docs/state.md ## Open bugs:
- Move 4 confirmed resolved bug tracking rows to ## Recently closed:
  * T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 (PR #1340, commit b43fc44)
  * T-AI-PTQ-STATIC-QUANT-FORMAT-UNPINNED-2026-09-03 (PR #1306, commit aebb9e1)
  * T-SVTAV1-HDR-ADAPTER-2026-05-20 (PR #1296, commit e335857)
  * T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 (PR #1296, commit e335857)
- Restore 2 closed rows accidentally dropped during dedup in commit 02f9ee4:
  * T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 (PR #1294, commit 8d103e3)
  * T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 (PR #1224, commit 6c843bb)
- Clean up duplicate tombstone comments in ## Open bugs.
- Add changelog fragment changelog.d/changed/docs-1238-state-md-stale-rows.md
  and regenerate CHANGELOG.md via concat-changelog-fragments.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 15, 2026
#1238)

Fix issue #1238 by auditing all rows in docs/state.md ## Open bugs:
- Move 4 confirmed resolved bug tracking rows to ## Recently closed:
  * T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 (PR #1340, commit b43fc44)
  * T-AI-PTQ-STATIC-QUANT-FORMAT-UNPINNED-2026-09-03 (PR #1306, commit aebb9e1)
  * T-SVTAV1-HDR-ADAPTER-2026-05-20 (PR #1296, commit e335857)
  * T-VMAFTUNE-PROFILE-REPORT-AUDIT-2026-05-20 (PR #1296, commit e335857)
- Restore 2 closed rows accidentally dropped during dedup in commit 02f9ee4:
  * T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 (PR #1294, commit 8d103e3)
  * T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 (PR #1224, commit 6c843bb)
- Clean up duplicate tombstone comments in ## Open bugs.
- Add changelog fragment changelog.d/changed/docs-1238-state-md-stale-rows.md
  and regenerate CHANGELOG.md via concat-changelog-fragments.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant