Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/security-scans.yml
Original file line number Diff line number Diff line change
Expand Up @@ -285,10 +285,18 @@ jobs:
"https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz" \
| tar -xz -C /usr/local/bin gitleaks
gitleaks version
- name: Gitleaks scan-scope self-test
run: python3 -B scripts/ci/tests/test_gitleaks_scan_scope.py
# Scan the history of the checked-out commit only. Without --log-opts
# gitleaks runs `git log --all`, and fetch-depth: 0 fetches every
# branch, so one branch's finding (even a force-pushed-away commit
# still in the fetch) failed every other PR. On a pull request HEAD
# is the merge commit: master's history plus the PR's commits.
- name: Run gitleaks (detect secrets)
run: |
gitleaks detect \
--source . \
--log-opts="--full-history HEAD" \
--config .gitleaks.toml \
--redact \
--report-format sarif \
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,13 @@
the new `feature_backends` array says where each extractor ran.


- **The Gitleaks check scans only the commit it checked out.** It ran
`git log --all` over a full-history checkout, so a finding on any branch in
the repository, including a commit a force-push had already replaced,
failed every other open pull request. Each run now scans the history of
its own `HEAD`: on a pull request, master plus the PR's commits.


- The oneAPI container image no longer crashes on Arc B580 (Battlemage)
GPUs. Through v1.0.0-rc.2 it shipped the Intel GPU compute runtime of
Intel's `oneapi-runtime:2025.3.1` image (version 25.18), and every
Expand Down
5 changes: 5 additions & 0 deletions changelog.d/fixed/gitleaks-scan-checked-out-history.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
- **The Gitleaks check scans only the commit it checked out.** It ran
`git log --all` over a full-history checkout, so a finding on any branch in
the repository, including a commit a force-push had already replaced,
failed every other open pull request. Each run now scans the history of
its own `HEAD`: on a pull request, master plus the PR's commits.
1 change: 1 addition & 0 deletions docs/state.md
Original file line number Diff line number Diff line change
Expand Up @@ -634,6 +634,7 @@ landed fix yet._
| **T-SYCL-MOTION-ADD-UV-SUBMIT-WAIT-2026-09-29** — `motion_sycl` waited on the device inside `submit()` with `motion_add_uv=true` | **FIXED by ADR-1371 on `fix/sycl-motion-tiny-frame-parity`.** Opened by the ADR-1363 audit on `perf/sycl-ssimulacra2-msssim-device-resident` (PR #1627), which adds this ID as an open row; whichever of the two merges second keeps only this closed row. `motion_upload_chroma` copied U and V from the picture on the primary queue and then called `vmaf_sycl_queue_wait()` (ADR-1034, Bug 2). `submit()` now packs U and V into pinned staging (`h_stage_u` / `h_stage_v`) and `motion_pre_graph` copies them on the combined queue ahead of the kernels, so the frame's only wait is in `collect()`. Output is bit-identical to the fixed-point oracle of `test_sycl_motion_add_uv_parity` (256x144 and 960x540, both GPUs). BBB 4K, 39 frames from memory, median of 4: host time per frame 0.89 to 0.55 ms on the B580 and 5.4 to 0.6 ms on the UHD 770; frame time 8.71 to 7.95 ms and 25.0 to 21.8 ms against the same kernel with the wait. |
| **T-SYCL-FLOAT-SSIM-SCALE-ONE-ONLY-2026-09-29** — `float_ssim_sycl` computed scale 1 only, so `float_ssim` at 1080p and 4K ran on the CPU | **FIXED by ADR-1370 on `feat/sycl-float-ssim-scale`.** CPU `float_ssim` decimates by `max(1, round(min(w, h) / 256))` (4 at 1080p, 8 at 4K) before SSIM; the twin had no decimation, so the ADR-1324 check refused every picture with a short side of 384 px or more and `--backend sycl --feature float_ssim` printed `float_ssim_sycl cannot run 3840x2160 8-bit pictures with these options; computing it on the CPU`. The twin now uploads raw luma and decimates on the device (`launch_decimate()`: `picture_copy()` scaling, `ssim.c`'s `1.0f / (scale * scale)` box, `iqa_filter_pixel()` window and `KBND_SYMMETRIC`, the exact CPU sum reproduced in int64 units of 2^-52). A dump-and-compare harness found the decimated planes byte-identical to `iqa_decimate()` in all 50 planes per device (Arc B580, UHD 770): BBB 4K and 1080p at auto, Netflix 576x324 at every scale from 1 to 10, 10-bit at 3, 5, 6, 12-bit at 3, 5, 10, 16-bit at 3, 7, 10, 853x481 at auto, 3, 6 and 9. Parity with `--backend cpu` (max abs over all frames, identical on both devices): Netflix 576x324 auto 3.0e-7, `scale=2` and `3` 6.0e-8; BBB 1080p auto 4.3e-5; BBB 4K auto 4.3e-5 (`enable_lcs` l / c / s 6.0e-8 / 4.2e-7 / 1.3e-6); 853x480 auto 4.5e-5. Time per 4K frame, `(t(22) - t(2)) / 20`, median of 3: B580 6.7 ms, UHD 770 12.0 ms, CPU `--threads 16` 11.0 ms, previous fallback 29.3 ms at the default thread count. The gate now refuses only a decimated plane under 11x11 or a scale above 128 (`float_ssim_geometry_supported()`); the CLI wording is unchanged because the check returns no reason. `submit()` now fails with an error instead of dereferencing NULL on the device-buffer-only `vmaf_read_pictures_sycl()` path. Guards: `test_sycl_float_ssim_parity` (+ `_large`), `test_feature_backend_twin`, `test_gpu_float_ssim_auto_scale_contract`, `test_vmaf_feature_backend_sycl`. | [ADR-1370](adr/1370-sycl-float-ssim-device-decimation.md), [Research-2130](research/2130-sycl-float-ssim-device-decimation.md) | `feat/sycl-float-ssim-scale` | 2026-09-29 | closed |
| **T-SYCL-FLOAT-SSIM-DB-DOUBLE-MEAN-2026-09-29** — `float_ssim_sycl` with `enable_db` reported tens of dB below the CPU on near-identical frames | **FIXED by ADR-1370 on `feat/sycl-float-ssim-scale`.** `iqa_ssim()` returns each frame mean as `(float)(sum / (double)(w * h))`; the twin emitted the double mean. The first four frames of the BBB 3840x2160 pair score `1 - 4.4e-10` on the device and exactly 1 on the CPU, so with `enable_db:clip_db` the twin reported 93.6 dB against the CPU's 121 dB ceiling. `float_ssim_frame_mean()` now rounds the `float_ssim` and `float_ssim_l/c/s` means to fp32; those frames report 121 dB and the largest remaining gap over the 24 frames is 0.22 dB at about 30 dB, the linear residual magnified by `4.34 / (1 - ssim)`. Linear scores move by at most 6e-8. | [ADR-1370](adr/1370-sycl-float-ssim-device-decimation.md), [ADR-1365](adr/1365-sycl-twin-cpu-option-parity.md) | `feat/sycl-float-ssim-scale` | 2026-09-29 | closed |
| **T-CI-GITLEAKS-SCANS-ALL-BRANCHES-2026-09-29** — one branch's Gitleaks finding failed every other pull request | **FIXED on `ci/gitleaks-scan-checked-out-history`.** The `Gitleaks` job (`.github/workflows/security-scans.yml`) checks out with `fetch-depth: 0`, which fetches every branch, and ran `gitleaks detect` without `--log-opts`, so gitleaks fell back to `git log --full-history --all`. On 2026-09-29 #1628 failed Gitleaks on `build-config.env:218` of commit `a74842e16`, a `generic-api-key` false positive (an apt signing-key fingerprint) in a pre-rebase commit of `fix/release-oneapi-image-runtime` that #1628 does not contain and that a force-push had already replaced; #1628's own diff and the current #1629 head scan clean. The job now passes `--log-opts="--full-history HEAD"`: on a pull request HEAD is the merge commit, so the scan covers master's history plus the PR's commits; on master and the schedule it covers master. Master's full history (4752 commits) scans clean under `.gitleaks.toml` with the new scope. `scripts/ci/tests/test_gitleaks_scan_scope.py`, run as the job's first step, pins the option, rejects `--all`, and keeps the full-history checkout and `--exit-code 1`. Branches without a pull request are no longer scanned by this job; GitHub secret scanning still covers every push. | `ci/gitleaks-scan-checked-out-history` | 2026-09-29 | fixed |
| **T-SYCL-FLOAT-MOTION-FORCE-ZERO-IGNORED-2026-09-29** — `float_motion_sycl` ignored `motion_force_zero` and emitted its debug score without `motion_fps_weight` | **FIXED by ADR-1365 on `fix/sycl-twin-option-parity`.** The twin declared `motion_force_zero` but only `flush()` read it: `collect()` emitted real `motion2` / `motion` scores where the CPU `float_motion` emits zeros (and dropped the tail `motion2`). Its debug `VMAF_feature_motion_score` was the raw SAD, where the CPU emits `motion_clip(score)` = `MIN(score * motion_fps_weight, motion_max_val)`. Now `submit()` skips the device for `motion_force_zero` and `collect()` emits zeros, and every emitted score goes through `motion_clip()`; `test_sycl_twin_option_parity` checks both against the CPU on an Arc B580 and a UHD 770. The CUDA, HIP and Metal twins honour `motion_force_zero`; their unweighted debug score is in `T-BUG048-GPU-OPTION-PARITY-REMAINDER-2026-09-26`. | [ADR-1365](adr/1365-sycl-twin-cpu-option-parity.md), [Research-2127](research/2127-sycl-twin-cpu-option-parity.md) |
| **T-DOCS-SSIM-PHANTOM-ENABLE-CHROMA-2026-09-29** — the SSIM page documented options the `ssim` extractor does not have | **FIXED on `fix/sycl-twin-option-parity`.** `docs/metrics/ssim.md` listed an `enable_chroma` option with `integer_ssim_cb` / `integer_ssim_cr` outputs and showed `--feature 'integer_ssim:enable_chroma=true'`; the CPU `integer_ssim.c` has only `enable_db` and `clip_db`, and `--feature integer_ssim=enable_chroma=true` fails with "unknown option 'enable_chroma'". The page now lists `enable_db` / `clip_db`, their backend support and working CLI examples. | [ADR-1365](adr/1365-sycl-twin-cpu-option-parity.md) |
| **T-SYCL-ADM-DECOUPLE-K-INT32-WRAP-2026-09-29** — the SYCL integer ADM decouple wrapped its Q15 ratio at scales 1-3 | **FIXED by ADR-1362 on `perf/sycl-adm-aim-device`.** `integer_adm_sycl.cpp` narrowed `(div_lookup * t) >> shift` to int32 before clamping k to [0, 32768]; the CPU clamps the int64 `tmp_k` (`adm_decouple_band_s123`), and so do the CUDA, HIP and Metal twins. Once `abs(t / o) > 2^16` the quotient passes INT32_MAX and wraps, so strong texture over a nearly flat reference decoupled wrongly. Measured on BBB 3840x2160 (50 frames, Arc B580 and UHD 770): the old twin's `integer_adm_scale2` was up to 1.40e-6 and `integer_adm_scale3` 3.3e-7 from the CPU; with the int64 clamp and the CPU's float finalisation (ADR-1362) every ADM output is bit-identical to the CPU. Planting the old narrowing into the new kernel reproduces the old twin's adm2 / scale outputs exactly and breaks aim / adm3 bit-exactness on 2-3 of 50 frames. At 576x324 two of 48 frames move by 3.7e-12. Guarded by the aim / adm3 bit-exact assertions in `test_sycl_adm_parity` and `test_sycl_adm_tiny_frames`. [Research-1362](research/1362-sycl-adm-aim-device-pass.md). | [ADR-1362](adr/1362-sycl-integer-adm-aim-device-pass.md) | `perf/sycl-adm-aim-device` | 2026-09-29 | fixed |
Expand Down
55 changes: 55 additions & 0 deletions scripts/ci/tests/test_gitleaks_scan_scope.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
# Copyright 2026 Lusoris
# SPDX-License-Identifier: EUPL-1.2
"""Pin the Gitleaks job to the history of the commit it checked out.

gitleaks falls back to `git log --all` when --log-opts is empty. With the
job's fetch-depth: 0 checkout that scans every branch in the repository, so
a finding on one branch failed every other pull request.
"""

from __future__ import annotations

import re
import unittest
from pathlib import Path

ROOT = Path(__file__).resolve().parents[3]
WORKFLOW = ROOT / ".github/workflows/security-scans.yml"


def scan_command(workflow: str) -> str:
"""Return the `gitleaks detect` command of the Gitleaks job."""
job = workflow.split("\n secret-scan:", 1)[1].split("\n dependency-review:", 1)[0]
match = re.search(r"gitleaks detect \\\n(?:\s+.*\\\n)*\s+.*", job)
if match is None:
raise AssertionError("no `gitleaks detect` command in the secret-scan job")
return match.group(0)


class GitleaksScanScopeTests(unittest.TestCase):
def setUp(self) -> None:
self.workflow = WORKFLOW.read_text(encoding="utf-8")
self.command = scan_command(self.workflow)

def test_scans_checked_out_history_only(self) -> None:
self.assertIn('--log-opts="--full-history HEAD"', self.command)

def test_never_scans_all_refs(self) -> None:
self.assertNotRegex(self.command, r"--all\b")
self.assertNotRegex(self.command, r"--log-opts=\"?\s*\"?\s*\\")

def test_keeps_the_full_history_checkout(self) -> None:
job = self.workflow.split("\n secret-scan:", 1)[1]
self.assertIn("fetch-depth: 0", job.split("gitleaks detect", 1)[0])

def test_fails_on_findings(self) -> None:
self.assertIn("--exit-code 1", self.command)

def test_parser_rejects_a_job_without_the_command(self) -> None:
with self.assertRaises(AssertionError):
scan_command("\n secret-scan:\n steps: []\n dependency-review:\n")


if __name__ == "__main__":
unittest.main()
Loading