Repository navigation
build(node): generate the eBPF object at build time with a pinned clang instead of committing it (ADR-1622) - #2063
Merged
Merged
Conversation
…bundles as .sigstore.json (#2061) * fix(ci): attribute the Metal math headers to Netflix and sign tester bundles as .sigstore.json Licence Provenance listed seven metal_*_math.h headers (#1921) whose EUPL-1.2 tag disagreed with the Netflix code they reproduce; five now carry the dual tag and two are recorded as reproducing none. Scorecard Signed-Releases did not count the tester bundles' .bundle signatures (it counts .sigstore.json), which with the committed eBPF object took the aggregate to 8.38 against the 8.5 floor; the tester workflows now write .sigstore.json. * fix(ci): prepend suite venv to PATH in run_affected_suites and isolate hook test python run_affected_suites.py now prepends the suite venv's bin/ directory to PATH and sets VIRTUAL_ENV so subshells and hook tests execute inside the isolated suite environment, and test_install.py prioritises sys.executable on its PATH.
* fix(ci): start the release workflows for version tags only Tester prereleases (tester-*, tester-windows-*) fire the release event and started the production publish, operator-node and supply-chain workflows, which failed at "Validate tag" and turned master red. Guard each workflow's first job (and the always() summary jobs) on a v* tag; dependents skip. Add a contract test over every on: release workflow.
…a licence-gated controller image (ADR-1589) (#2033) * feat(helm)!: deploy vmafx-controller as its own workload and publish a licence-gated controller image (ADR-1589) The chart deployed no controller and no workflow built one; a controller could only run as the server workload with a self-built image and without its gRPC port on a Service. controller.enabled now renders a one-replica Recreate Deployment (SQLite queue on a ReadWriteOnce claim) with an http and a grpc Service port. auth.* is rendered into the controller only, and auth.enabled and controller.enabled require each other. The nodes and the operator are pointed at the controller, node.controllerToken and operator.controllerToken mount their tokens from Secrets, and the NetworkPolicies open the flows between them. docker/Dockerfile.controller follows Dockerfile.go-server stage for stage (the old file linked Debian's libvmaf-dev and had no licence stages), the licence record production-controller-image reuses the go-server components, and docker-publish-operator-node.yml builds, signs, SBOMs, smoke-tests and publishes ghcr.io/vmafx/vmafx-controller with its -source image. The new GHCR package needs its visibility checked after the first publish. The controller gains --version (T-CONTROLLER-NO-VERSION-FLAG-2026-10-04). e2e-k8s.yml bounds its four tool downloads with --max-time and reports failed diagnostics instead of discarding them; the eleven findings leave the HISS baseline (413 -> 402; T-E2E-K8S-UNBOUNDED-DOWNLOADS-2026-10-04). T-HELM-NO-CONTROLLER-WORKLOAD-2026-10-04 closed. Migration: auth.* now needs controller.enabled. A release that ran a controller through image.repository fails to render; set controller.enabled (and controller.image for a custom image) and copy the old queue database to the <release>-controller-data claim if its jobs are needed.
* chore(deps): Update dependency onnxruntime to >=1.30.0 * chore(deps): refresh the lock fingerprints for the vmaf-tune extras bump
* chore(deps): Update dependency black to v26.10.0 * chore(deps): pin black 26.10.0 in the rc1-tester extras and refresh both black locks The Makefile pin and the pre-commit hook moved to 26.10.0 in #2044; the dev-linters lock and the rc1-tester extras were the two places left on 26.5.1. black 26.10.0 leaves the tree unchanged under make format-check.
…ng instead of committing it (ADR-1622) (#2063) * build(node): generate the eBPF object at build time with a pinned clang instead of committing it (ADR-1622) OpenSSF Scorecard's Binary-Artifacts check flags the committed cmd/vmafx-node/bpf/rclonebypass_bpfel.o, which keeps the Scorecard aggregate under its 8.5 floor. Per the maintainer's decision of 2026-10-05, neither the object nor its bpf2go binding is committed any more. scripts/dev/gen-node-bpf.sh generates both and is called from Go CI, the API-compatibility gate, docker/Dockerfile.node, dev/Containerfile, the Go Makefile targets and the go vet hook. build-config.env pins clang 19.1.7 and the object's sha256; --require-pin refuses another clang or digest, and a missing tool fails naming it. clang 19.1.7 builds the same bytes on Debian 13 (amd64, arm64) and Ubuntu 26.04. Supersedes the committed-object part of ADR-1539. Migration: building cmd/vmafx-node now needs clang with the BPF target, llvm-strip and the libbpf headers; run `make node-bpf` first.
lusoris
force-pushed
the
build/bpf-object-at-build-time
branch
from
October 5, 2026 06:16
100bf28 to
a60b7a9
Compare
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The node's eBPF object is now generated at build time with a pinned clang instead of being committed, which removes the ELF that Scorecard's
Binary-Artifactscheck flags (the maintainer decided this on 2026-10-05).scripts/dev/gen-node-bpf.sh(make node-bpf) is the single generation step; Go CI,docker/Dockerfile.node,dev/Containerfileand thego-build/go-testtargets call it.build-config.envpins clang 19.1.7 and the object's sha256;--require-pinrefuses anything else, and a missing tool fails naming it, with no fallback and no download.Stacked on #2061 (it carries the
T-SCORECARD-COMMITTED-BPF-OBJECT-2026-10-05row this PR closes).The bpf2go binding stays committed source, with its
go:embedrewritten toembeddedObject(). The lockedGo API Compatibilitygate runsgo list -export ./...with no generation step, so the package has to compile without the object. A node built without it refusesVMAFX_EBPF_BYPASS=1and names the generator.Type
build/ci— tooling / infraChecklist
make format && make lintis green locally (commit hooks passed).go test ./cmd/vmafx-node/...andscripts/dev/tests/test_gen_node_bpf.pypass.docs/adr/_index_fragments/1622-bpf-object-generated-at-build-time.md, slug appended to_order.txt.Bug-status hygiene
docs/state.md:T-SCORECARD-COMMITTED-BPF-OBJECT-2026-10-05moved from Open to Recently closed with the evidence.Netflix golden-data gate
assertAlmostEqual(...)score in the Netflix golden Python tests.Deep-dive deliverables
## Alternatives considered.AGENTS.mdinvariant note —cmd/vmafx-node/AGENTS.mditem 15.changelog.d/changed/node-ebpf-object-generated-at-build-time.md.docs/rebase-notes.md, "The node's eBPF object is generated at build time".Reproducer
Measured:
a8079aa4e539dc30e5f275a424f1831d911fb0ef72e579c5c8527b8de2415ca0on Debian 13 amd64, Debian 13 arm64 (QEMU) and Ubuntu 26.04 amd64; clang 23.1.1 reproduces the previously committed object byte for byte.docker build --target go-builderofDockerfile.nodefrom a tree without the object:gen-node-bpf --require-pinpasses inside the stage andgo buildproduces/out/vmafx-node.golang:1.27-trixiecontainer,TestEmbeddedObjectMatchesPinnedDigest,TestEmbeddedObjectMatchesMirrorsandTestEmbeddedObjectLicencepass; without the object the package builds and those tests skip namingmake node-bpf.PATHlacking clang,make node-bpfexits 2:clang not found ... install clang 19.1.7 ... apt-get install clang-19 llvm-19 libbpf-dev.go test ./cmd/vmafx-node/...passes against the in-tree CPU libvmaf (VMAF_BINset to the CPUvmaf).Known follow-ups
Binary-Artifactsreading 10 on the next Scorecard run on master.dev/Containerfilewas edited (clang-19 neighbours,llvm-19,libbpf-dev, the generation step) but its full image was not built here; the same script, packages and digest were exercised in the Debian and Ubuntu containers.core/test/fuzz/**/*.binare the only other.binfiles in the tree; they are ASCII text anddata, not executables.Breaking changes / migration
Producing the node's eBPF object needs clang with the BPF target,
llvm-stripand the libbpf headers: runmake node-bpf(seedocs/development/node-ebpf-build.md). Container builds need nothing on the host.