Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions changelog.d/added/cuda-compileiq-pilot-filter1d-v2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# CompileIQ pilot v2 — abandoned, investigation complete (ADR-0742)

Investigated automated CUDA kernel auto-tuning via `compileiq` (PyPI) against
`core/src/feature/cuda/integer_vif/filter1d.cu` using the `vmaf-dev-mcp:cuda13.3`
container. The `compileiq` package at v0.0.0a0 is an empty placeholder with no
functional CLI; the pilot was abandoned immediately after confirming this.
Decision recorded in ADR-0742; research digest at
`docs/research/research-0742-cuda-compileiq-filter1d-pilot-v2.md`.
Recommended next step for `filter1d.cu` tuning: `/profile-hotpath cuda vif`
with `ncu` or a grid-search wrapper over nvcc block-size parameters.
Closes PR #66 (CompileIQ pilot v1).
9 changes: 9 additions & 0 deletions core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,3 +457,12 @@ the corrected methodology.
— `enable_avx512=true` with `enable_asm=false` issues a warning (no-op, not an error);
— `enable_hipcc=true` with `enable_hip=false` issues a warning (no-op, not an error).
The checks run at configuration time (before `subdir()` calls) to catch misconfigurations early. The principle: every option that depends on another must `error()` on the bad combo, never silently no-op. See [`src/meson.build` lines 100–111, 74–76, 142–144](src/meson.build).

- **No ACF file for `filter1d.cu`** (ADR-0742): The CompileIQ auto-tuning
pilots (PR #66, v2 2026-05-28) were abandoned because the `compileiq`
package on PyPI (v0.0.0a0) is an empty placeholder. No
`core/src/feature/cuda/integer_vif/filter1d.acf` file exists. If a future
CUDA toolkit bump is paired with a real CompileIQ release, the ACF must be
re-tuned on the new toolkit before committing — ACF files encode
CUDA-version-specific register/occupancy decisions. Use
`/profile-hotpath cuda vif` with `ncu` for immediate kernel-tuning guidance.
68 changes: 68 additions & 0 deletions docs/adr/0742-cuda-compileiq-filter1d-pilot-v2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# ADR-0742: Abandon CompileIQ Kernel Auto-Tuning Pilot — Tool Not Available on PyPI

- **Status**: Accepted
- **Date**: 2026-05-28
- **Deciders**: lusoris
- **Tags**: cuda, performance, tooling, kernel-tuning, fork-local

## Context

The fork has an ongoing interest in automated CUDA kernel parameter search for
`core/src/feature/cuda/integer_vif/filter1d.cu` — the hottest kernel in the
integer VIF extractor. A pilot using a tool called "CompileIQ" was proposed and
attempted twice.

Pilot v1 (PR #66) was abandoned because the `vmaf-dev-mcp` container at the
time ran Python 3.14, and the pilot brief assumed CompileIQ required Python
`<3.14`.

Pilot v2 (this ADR) was initiated with the new `vmaf-dev-mcp:cuda13.3` image
(Ubuntu 26.04 LTS, CUDA 13.3) and a plan to install Python 3.13 via a sub-venv
to work around the version constraint.

Investigation during v2 revealed that the `compileiq` package on PyPI
(`pypi.org/project/compileiq`) is version `0.0.0a0` — an empty placeholder
with no module contents, no CLI entry point, and no functional code. The
package description reads `"Empty package compileiq."` The Python version issue
from v1 was a false lead; the actual blocker is that the tool does not exist as
a published package on PyPI.

## Decision

Abandon the CompileIQ auto-tuning pilots. Do not pursue further attempts until
evidence of a functional, published CompileIQ release is available from a
primary source (the tool's authors or a peer-reviewed reference). Use
`/profile-hotpath` with `ncu` for immediate profiling needs; design a grid
search wrapper using `subprocess.run` + meson reconfigure if systematic
block-size tuning is required.

## Alternatives Considered

| Option | Pros | Cons | Why Not Chosen |
|---|---|---|---|
| Retry with Python 3.13 via deadsnakes PPA | Python 3.13 is available for Ubuntu 26.04 via PPA | Still installs the same empty `compileiq 0.0.0a0` package — Python version is irrelevant | Does not fix the root cause |
| Use KTT (Kernel Tuning Toolkit) | Active open-source C++ auto-tuner; handles CUDA block sizes + register pressure + unroll | Requires adding a C++ dependency and writing a KTT host harness for filter1d kernels; ~3–5 days effort | Viable future option; not the immediate priority |
| Grid search via `subprocess.run` + meson reconfigure | Zero new dependencies; fully fork-controlled | Manual implementation required; ~1–2 days | Best next option if systematic tuning is desired |
| `/profile-hotpath` + `ncu` | Available today; identifies register pressure / occupancy / bank conflicts without recompiling | Not an auto-tuner; requires human interpretation | Recommended next step for `filter1d.cu` |
| Accept current hand-tuned parameters | No effort required | May leave performance on the table | Acceptable for now; filter1d already has smem tiling (win #1 + #4) |

## Consequences

- **Positive**: No dead code, no ACF files, no conditional build plumbing
committed for a non-functional tool. Pilot cost capped at investigation time.
- **Negative**: Auto-tuning of `filter1d.cu` block parameters remains a manual
or future task.
- **Neutral / Follow-ups**:
- `/profile-hotpath cuda vif` is the recommended next action for
`filter1d.cu`.
- If CompileIQ is ever published as a functional package, the pilot can be
reopened; the objective function design from the brief is still valid.
- PR #66 is closed by the DRAFT PR that accompanies this ADR.

## References

- `req`: "CompileIQ pilot v2 on `core/src/feature/cuda/integer_vif/filter1d.cu`"
(user request, 2026-05-28 session).
- Research digest: [docs/research/research-0742-cuda-compileiq-filter1d-pilot-v2.md](../research/research-0742-cuda-compileiq-filter1d-pilot-v2.md)
- PR #66 (pilot v1, abandoned)
- `pypi.org/project/compileiq` — inspected 2026-05-28; v0.0.0a0, empty package
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -550,3 +550,4 @@ ADRs may exist there for local session continuity, but the tracked
| [ADR-0712](0712-ide-config-multilang-refresh.md) | IDE config audit and refresh for multi-language post-rebrand VMAFX: clangd, gopls, rust-analyzer, and Python LSP wired to the post-rebrand directory layout. | Accepted | 2026-05-28 | ide, clangd, gopls, rust-analyzer, build, fork-local |
| [ADR-0714](0714-vmafx-operator-skeleton.md) | vmafx-operator kubebuilder skeleton + CRDs: `VmafxJob`, `VmafxNode`, `VmafxModelTraining` in API group `vmafx.dev/v1`; Stage 1 stub reconcilers; Helm chart integration; envtest suite. | Accepted | go, k8s, operator, crd, controller-runtime, phase4b, fork-local |
| [ADR-0717](0717-vmafx-node-ffmpeg-latest.md) | vmafx-node ffmpeg version policy: pin to latest stable tagged release (n8.2); multi-stage Dockerfile with shared ffmpeg-builder-cpu stage; four variants (cpu/cuda/rocm/sycl); startup encoder-inventory probe. | Accepted | 2026-05-28 | node, ffmpeg, docker, phase4b, fork-local |
| [ADR-0742](0742-cuda-compileiq-filter1d-pilot-v2.md) | CompileIQ CUDA kernel auto-tuning pilot v2 — abandoned: `compileiq` PyPI package is an empty placeholder (v0.0.0a0, no CLI). Pilot v1 (PR #66) and v2 both abandoned; recommended next step is `/profile-hotpath cuda vif` with `ncu`. | Accepted | 2026-05-28 | cuda, performance, tooling, kernel-tuning, fork-local |
10 changes: 10 additions & 0 deletions docs/rebase-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -39917,3 +39917,13 @@ Fork-local files:
`docs/adr/0714-vmafx-operator-skeleton.md`,
`docs/development/operator.md`,
`changelog.d/added/vmafx-operator-skeleton.md`,

---

## ADR-0742 — CompileIQ kernel auto-tuning pilot v2 (2026-05-28)

No rebase impact: this PR adds only documentation and ADR files (research
digest, ADR, changelog fragment). No C/CUDA source files were modified.
`core/src/feature/cuda/integer_vif/filter1d.cu` is unchanged. No
`-Denable_compileiq` build option was added. Upstream syncs have no
interaction with these doc-only files.
114 changes: 114 additions & 0 deletions docs/research/research-0742-cuda-compileiq-filter1d-pilot-v2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Research Digest 0742 — CompileIQ Pilot v2: filter1d.cu Auto-Tuning (Abandoned)

**Date**: 2026-05-28
**Branch**: `research/cuda-compileiq-filter1d-pilot-v2-20260528`
**Outcome**: Abandoned — `compileiq` PyPI package is an empty placeholder (v0.0.0a0)
**Supersedes**: PR #66 (CompileIQ pilot v1, abandoned for Python 3.14 incompatibility)

---

## Summary

A second attempt was made to run CompileIQ automated kernel-tuning against
`core/src/feature/cuda/integer_vif/filter1d.cu` using the newly-built
`vmaf-dev-mcp:cuda13.3` container image (CUDA 13.3, Ubuntu 26.04 LTS).

The attempt was abandoned immediately after confirming that the `compileiq`
package available on PyPI (version `0.0.0a0`) is an empty placeholder with no
functional CLI. There is no `run` subcommand, no `--apply-controls` flag, and
no module contents beyond `__version__ = "0.0.0a0"`.

---

## Environment Probed

| Component | Result |
|---|---|
| Container image | `vmaf-dev-mcp:cuda13.3` (50.9 GB, Ubuntu 26.04 LTS) |
| CUDA toolkit | 13.3, release 13.3 V13.3.33 (confirmed via `nvcc --version`) |
| Python in container | 3.14.4 (system default; Ubuntu 26.04 ships Python 3.14 only) |
| Python 3.13 availability | Available via `ppa:deadsnakes/ppa` for Ubuntu 26.04 |
| Python 3.12 availability | Not available (no apt package, not in deadsnakes for 26.04) |
| `compileiq` on PyPI | v0.0.0a0 — empty package (`Empty package compileiq.`) |

---

## Attempted Steps

**Step 1: Branch setup** — succeeded. Branch
`research/cuda-compileiq-filter1d-pilot-v2-20260528` created from
`origin/master`.

**Step 2: Smoke test (entrypoint bypass)** — `nvcc --version` confirmed CUDA
13.3. Container entrypoint has SYCL/HIP GPU probes that each wait up to 300 s
before proceeding; bypassed with `--entrypoint bash`.

**Step 3: Python 3.13 install** — `python3.13` is not in Ubuntu 26.04's default
apt repos. The `python3.13-venv` package is also absent. The deadsnakes PPA
(`ppa:deadsnakes/ppa`) does provide `python3.13` for Ubuntu 26.04.

**Step 4: compileiq on Python 3.14** — Attempted direct install without
Python 3.13 sub-venv. `pip install compileiq` succeeded (exit 0) but installed
version `0.0.0a0`, which contains only an empty `__init__.py` with
`__version__ = "0.0.0a0"`. No `compileiq` console-script entry point is
registered. The `ls /tmp/civ/bin/` output showed a `𝜋thon` symlink but no
`compileiq` binary.

**Conclusion**: The `compileiq` package name on PyPI is a placeholder — either
a name reservation, an abandoned project, or the tool was never published under
this name. The tool referenced in the pilot brief does not exist at
`pypi.org/project/compileiq`.

---

## Root Cause

The pilot v1 (PR #66) was abandoned because the container had Python 3.14 and
the deep-research assumed `compileiq` required `<3.14`. Pilot v2 was initiated
with the hypothesis that Python 3.13 in a sub-venv would unblock it. This v2
investigation revealed the actual blocker: the `compileiq` tool itself does not
exist on PyPI as a functional package. The Python version issue in v1 was a
false lead.

---

## Alternatives for CUDA Auto-Tuning

The following real tools exist for GPU kernel parameter search:

| Tool | Status | Notes |
|---|---|---|
| [KTT (Kernel Tuning Toolkit)](https://github.com/HiPerComp/KTT) | Active open source | C++ API, search block sizes / registers / unroll factors |
| [CLTune](https://github.com/CNugteren/CLTune) | Archived | CUDA + OpenCL, Python bindings, last commit 2019 |
| [NVIDIA `ncu --set full`](https://docs.nvidia.com/nsight-compute/) | Available in CUDA 13.3 | Profiler, not auto-tuner; guides manual tuning |
| [OpenTuner](https://opentuner.org/) | Mature | Framework for arbitrary parameter search; can wrap nvcc |
| Custom grid search via `subprocess.run(nvcc ...)` | Always available | Low-overhead; used by the `/profile-hotpath` skill |
| cuTuner / AutoTVM CUDA backend | Research prototypes | Not production-ready for our kernel surface |

The `/profile-hotpath` skill (ADR-0199) with `ncu` is the recommended next step
for `filter1d.cu` if auto-tuning is still desired. A grid search over
`BLOCKX`, `BLOCKY`, and `val_per_thread` can be implemented as a thin
Python wrapper using `subprocess.run` + meson reconfigure; estimated
implementation effort: 1–2 days.

---

## Disposition

- ADR-0742: Decision recorded as "abandon CompileIQ; tool not available on
PyPI."
- No code changes committed.
- No ACF file produced.
- PR #66 closure is requested in the accompanying DRAFT PR.
- Future CUDA kernel auto-tuning should use `/profile-hotpath` + `ncu` or
a grid-search wrapper over nvcc block-size parameters.

---

## References

- PR #66 (pilot v1, abandoned 2026-xx-xx)
- `pypi.org/project/compileiq` — v0.0.0a0, installed size ~3 KB, no entry
points
- ADR-0742 (this decision)
- CUDA 13.3 toolkit: `/opt/cuda/bin/nvcc --version` → `release 13.3, V13.3.33`
Loading