Skip to content

fix(test): run the GPU pool UAF test without MALLOC_PERTURB_ so it cannot fill the host's memory - #2547

Merged
lusoris merged 1 commit into
masterfrom
fix/gpu-pool-uaf-test-perturb
Oct 7, 2026
Merged

lusoris merged 1 commit into
masterfrom
fix/gpu-pool-uaf-test-perturb

Conversation

@lusoris

@lusoris lusoris commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

test_gpu_picture_pool_uaf (suite fast) can take all of a host's RAM and swap. It now runs with MALLOC_PERTURB_=0, and its two cases are unchanged.

The test asks vmaf_gpu_picture_pool_init() for 0x7FFFFFFF pictures, about 192 GB, to reach the allocation-failure cleanup. Meson sets a random MALLOC_PERTURB_ for every test, so glibc fills each allocation it returns. On a host with vm.overcommit_memory = 1 the request succeeds and the fill writes every byte of it. On the RC workstation (60 GB RAM + 60 GB swap) the merge train's stack gate timed out on it after 91 s (local gate failed on the stack of 5 (fast tests fail ...: test_gpu_picture_pool_uaf TIMEOUT 90.9s), 00:47 on 2026-10-08). A local gate run timed out after 32 s; both runs fill swap while they last.

Meson keeps an explicit MALLOC_PERTURB_=0 and does not set its own value (mesonbuild/mtest.py: "Setting MALLOC_PERTURB_="0" will completely disable this feature"). The allocation is then not touched. The failure path and its two cases still run.

Type

  • fix — bug fix
  • test — test-only

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally (the commit hooks pass).
  • Unit tests: the test binary with MALLOC_PERTURB_=88 in a 3 GiB memory scope (systemd-run --scope -p MemoryMax=3G) is OOM-killed (137) after 3 s; with MALLOC_PERTURB_=0 both cases pass at once. meson introspect --tests on this branch shows the test's environment with MALLOC_PERTURB_: 0.
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff. — not applicable.
  • If I touched a feature extractor with SIMD/GPU twins, I updated every twin. — not applicable.
  • New .c / .h file with a license header. — not applicable.
  • Breaking change. — not breaking.
  • ADR added. — not applicable: a test environment fix.

Bug-status hygiene (ADR-0165)

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below — not applicable.

Cross-backend numerical results

Not applicable: no score path changed.

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: trivial test environment fix.
  • Decision matrix — no alternatives: only-one-way fix (the test must not write the oversized allocation; the cleanup path it tests is unchanged).
  • AGENTS.md invariant note — no rebase-sensitive invariants.
  • Reproducer / smoke-test command — below.
  • CHANGELOG fragment — no changelog: test-only change, no user-visible delta.
  • Rebase note — no rebase impact: fork-only test registration.

Reproducer

cat /proc/sys/vm/overcommit_memory      # 1
MALLOC_PERTURB_=88 systemd-run --user --scope -p MemoryMax=3G -p MemorySwapMax=1G build/test/test_gpu_picture_pool_uaf   # 137 after 3 s
MALLOC_PERTURB_=0  build/test/test_gpu_picture_pool_uaf   # 2 tests run, 2 passed

@github-actions github-actions Bot added the type:bug Something isn't working label Oct 7, 2026
lusoris added a commit that referenced this pull request Oct 7, 2026
…nnot fill the host's memory (#2547)

* fix(test): run the GPU pool UAF test without MALLOC_PERTURB_ so it cannot fill the host's memory

test_gpu_picture_pool_uaf asks the pool for about 192 GB to reach the allocation-failure cleanup. Meson sets a random MALLOC_PERTURB_ for each test, so glibc writes every byte it hands out; on a host with vm.overcommit_memory = 1 the request succeeds and the fill takes all RAM and swap. The test now runs with MALLOC_PERTURB_=0; its two cases are unchanged.

* docs(state): cite PR #2547 in the GPU pool test row
@lusoris
lusoris force-pushed the fix/gpu-pool-uaf-test-perturb branch from 1e3fade to 6fed045 Compare October 7, 2026 23:36
@lusoris
lusoris merged commit 6fed045 into master Oct 7, 2026
22 of 42 checks passed
@lusoris
lusoris deleted the fix/gpu-pool-uaf-test-perturb branch October 7, 2026 23:40
lusoris added a commit that referenced this pull request Oct 8, 2026
…2548 landed twice

Signed-off-by: Lusoris <lusoris@proton.me>

#2548 landed the merge train's squash of its first revision, which still carried its own MALLOC_PERTURB_=0 entry for test_gpu_picture_pool_uaf and a state row for the same bug; #2547 had already landed both. core/test/meson.build now lists the entry once (#2547's), docs/state.md keeps T-GPU-POOL-UAF-TEST-FILLS-HOST-MEMORY-2026-10-08 only, and the #2548 changelog and rebase-note fragments no longer claim that fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant