Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/libvmaf-build-matrix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1105,14 +1105,22 @@ jobs:
if: matrix.backend == 'cuda'
shell: cmd
run: |
<<<<<<< HEAD
ninja -v -C core\build install
=======
ninja -v -C libvmaf\build install
>>>>>>> 24bb5daf89 (docs: post-merge-train sweep — VMAFx + core/ path refs, ADR index, state.md)

- name: Build libvmaf (SYCL)
if: matrix.backend == 'sycl'
shell: cmd
run: |
call "C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
<<<<<<< HEAD
ninja -v -C core\build install
=======
ninja -v -C libvmaf\build install
>>>>>>> 24bb5daf89 (docs: post-merge-train sweep — VMAFx + core/ path refs, ADR index, state.md)

# No test step — windows-2025 has no GPU. The whole point of
# this job is exercising the MSVC + CUDA / MSVC + oneAPI link
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/security-scans.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,11 +133,14 @@ jobs:
languages: python
queries: security-and-quality
config-file: ./.github/codeql-config.yml
<<<<<<< HEAD
# Python analysis does not require a build step; the explicit no-op
# below suppresses CodeQL's C++-oriented autobuild.sh which would
# fail trying to build the old libvmaf/ path (post-rename to core/).
- name: No-op build (Python analysis only)
run: echo "Python-only CodeQL scan — no C++ build required"
=======
>>>>>>> 24bb5daf89 (docs: post-merge-train sweep — VMAFx + core/ path refs, ADR index, state.md)
- uses: github/codeql-action/analyze@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v4
with:
category: "/language:python"
Expand Down
20 changes: 2 additions & 18 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -364,7 +364,7 @@ linked AGENTS.md before resolving conflicts.
`libvmaf_mcp.h`, audit-first `-ENOSYS` stubs in
`core/src/mcp/mcp.c`, `enable_mcp` + 3 transport sub-flags. T5-2b
(cJSON + mongoose + transport bodies) is open. See
[core/AGENTS.md §Rebase-sensitive invariants](core/AGENTS.md).
[libvmaf/AGENTS.md §Rebase-sensitive invariants](libvmaf/AGENTS.md).
- **HIP scaffold (T7-10, ADR-0212 placeholder, PR #200)** —
audit-first AMD HIP backend scaffold mirroring Vulkan T5-1 /
ADR-0175. Public `libvmaf_hip.h`, stub kernels, `enable_hip` meson
Expand All @@ -380,7 +380,7 @@ linked AGENTS.md before resolving conflicts.
requires (1) `FEATURE_METRICS` entry, (2) `FEATURE_TOLERANCE` entry
if it relaxes places=4, (3) row in
`docs/development/cross-backend-gate.md`. See
[core/AGENTS.md](core/AGENTS.md).
[libvmaf/AGENTS.md](libvmaf/AGENTS.md).
- **FastDVDnet temporal pre-filter (T6-7, ADR-0215 placeholder,
PR #203)** — 5-frame window pre-filter feeding ssim/ms_ssim.
- **psnr chroma Vulkan (T3-15(b), ADR-0216 placeholder, PR #204)**
Expand Down Expand Up @@ -431,22 +431,6 @@ linked AGENTS.md before resolving conflicts.
`host_id`, `backend_results`, `mcp_results`) are an internal format;
update `docs/development/dev-mcp.md` if the schema changes. This
directory does not affect the libvmaf C build or any CI gate.
- **VMAFX repo-layout rename (`libvmaf/` → `core/`, PR #1571, ADR-0700)**:
The C library and build root was renamed from `libvmaf/` to `core/` as
part of the VMAFX rebrand (ADR-0686). `libvmaf.so`, `libvmaf.pc`, and
`<libvmaf/…>` install-path headers are unchanged. Any in-flight branch
that still references paths under `libvmaf/` must be rebased; the
recommended recipe is:
`git format-patch <base>..<tip> --stdout | sed 's|libvmaf/|core/|g' | git am --3way`
The full recipe lives in `docs/rebase-notes.md` §`refactor/meta/vmafx-repo-layout`.
Do not introduce new references to `libvmaf/` in any new file.
- **vmafx-server HTTP transport (`[http]` optional dep, PR #1583, ADR-0701)**:
`mcp-server/vmaf-mcp/pyproject.toml` gained an `[http]` optional
dependency group (`aiohttp`, `prometheus-client`). Any rebase that
modifies `pyproject.toml` must preserve this group. The `--transport http`
flag in `server.py::main()` dispatches to `http_transport.py`; do not
reorder the transport dispatch block. See
[mcp-server/AGENTS.md](mcp-server/AGENTS.md).

## 14. Interaction style — prefer structured popup questions

Expand Down
62 changes: 36 additions & 26 deletions cmd/vmafx-node/executor_test.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
// SPDX-License-Identifier: BSD-3-Clause-Plus-Patent
// Copyright 2026 Lusoris
//
// cmd/vmafx-node/executor_test.go — unit tests for the job executor covering
// the binary-delegation path (ScoringJob via "false" binary).
// cmd/vmafx-node/executor_test.go — unit tests for the job executor.
//
// ADR-0719: vmafx-node rclone integration.
package main
Expand All @@ -13,15 +12,26 @@ import (
"path/filepath"
"testing"

controllerv1 "github.com/VMAFx/vmafx/gen/go/controller"
"github.com/VMAFx/vmafx/pkg/libvmaf"
"github.com/VMAFx/vmafx/pkg/storage"
)

// TestExecutor_ScoringJobFailsWithBadBinary verifies that Execute returns a
// non-nil error when the underlying vmaf binary exits non-zero.
// We construct a Scorer pointing at "false" (a POSIX built-in that always
// exits 1) so the unit test never requires a real vmaf installation.
func TestExecutor_ScoringJobFailsWithBadBinary(t *testing.T) {
// mockStorage is a Storage that always returns the path it was given.
type mockStorage struct {
mode storage.Mode
}

func (m *mockStorage) Mode() storage.Mode { return m.mode }

func (m *mockStorage) Prepare(_ context.Context, sourceURI string) (string, func(), error) {
return sourceURI, func() {}, nil
}

// TestExecutor_LocalPaths verifies that the executor calls Prepare on both
// reference and distorted URIs and forwards the resolved paths to vmaf.
// The actual vmaf invocation is not run in the unit test; we verify only that
// Prepare is called and that cleanup is deferred correctly by checking that no
// goroutines are leaked.
func TestExecutor_LocalPaths(t *testing.T) {
t.Parallel()

dir := t.TempDir()
Expand All @@ -34,26 +44,26 @@ func TestExecutor_ScoringJobFailsWithBadBinary(t *testing.T) {
}
}

// Build a Scorer that delegates to "false" — always exits 1.
scorer, err := libvmaf.New("false", "")
if err != nil {
// "false" must exist on the PATH in CI; skip rather than fail.
t.Skipf("could not locate 'false' binary: %v", err)
}

exec := NewExecutor(scorer, nil, "cpu", nil)
store := &mockStorage{mode: storage.ModeHTTPServe}
exec := NewExecutor(store, "false", nil) // "false" → exits 1 immediately, never actually runs vmaf

job := &controllerv1.Job{
Id: "test-001",
Scoring: &controllerv1.ScoringParams{
Reference: refPath,
Distorted: disPath,
Model: "vmaf_v0.6.1",
},
// We do NOT expect a successful score here since "false" is not vmaf.
// We just verify that Prepare is called for both URIs without panics or
// data races, and that cleanup() defers are handled.
job := ScoringJob{
JobID: "test-001",
ReferenceURI: refPath,
DistortedURI: disPath,
Width: 576,
Height: 324,
PixelFormat: "yuv420p",
ModelPath: "/models/vmaf_v0.6.1.json",
}

result := exec.Execute(context.Background(), job)
if result.Error == nil {
ctx := context.Background()
_, err := exec.Execute(ctx, job)
// We expect an error because "false" exits 1.
if err == nil {
t.Error("expected error from 'false' vmaf binary, got nil")
}
}
171 changes: 166 additions & 5 deletions cmd/vmafx-node/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,9 @@
//
// cmd/vmafx-node/main_test.go — lifecycle tests for vmafx-node.
//
// Tests exercise the executor dispatch path via the mock controller server.
// Config-loading and node-registration tests were removed in the go-workspace
// audit (2026-05-29): the controller-registration path was replaced by the
// ffmpeg-probe / gRPC server model in the current main.go; those symbols
// (loadConfig, node, register, executeAndReport) no longer exist.
// Tests exercise the config loading, GPU detection wiring, and the
// executor dispatch path. The full lifecycle (Register → PullWork →
// Execute → ReportResult) is tested with a mock controller server.
//
// ADR-0713: vmafx-node Go worker binary.

Expand All @@ -17,12 +15,16 @@ import (
"context"
"log/slog"
"net"
"os"
"testing"
"time"

"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"

controllerv1 "github.com/VMAFx/vmafx/gen/go/controller"
"github.com/VMAFx/vmafx/pkg/ai"
"github.com/VMAFx/vmafx/pkg/gpu"
)

// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -108,6 +110,61 @@ func startMockController(t *testing.T, mc *mockController) (*grpc.Server, string
return srv, lis.Addr().String()
}

// ---------------------------------------------------------------------------
// Config tests
// ---------------------------------------------------------------------------

func TestLoadConfig_MissingAddr(t *testing.T) {
// t.Setenv is incompatible with t.Parallel()
t.Setenv("VMAFX_CONTROLLER_ADDR", "")
_, err := loadConfig()
if err == nil {
t.Fatal("expected error when VMAFX_CONTROLLER_ADDR is not set")
}
}

func TestLoadConfig_Defaults(t *testing.T) {
// t.Setenv is incompatible with t.Parallel()
t.Setenv("VMAFX_CONTROLLER_ADDR", "localhost:8080")
t.Setenv("VMAFX_NODE_ID", "test-node")
cfg, err := loadConfig()
if err != nil {
t.Fatalf("loadConfig: %v", err)
}
if cfg.controllerAddr != "localhost:8080" {
t.Errorf("controllerAddr = %q, want %q", cfg.controllerAddr, "localhost:8080")
}
if cfg.nodeID != "test-node" {
t.Errorf("nodeID = %q, want %q", cfg.nodeID, "test-node")
}
if cfg.logLevel != slog.LevelInfo {
t.Errorf("logLevel = %v, want info", cfg.logLevel)
}
}

func TestLoadConfig_LogLevelDebug(t *testing.T) {
// t.Setenv is incompatible with t.Parallel()
t.Setenv("VMAFX_CONTROLLER_ADDR", "localhost:8080")
t.Setenv("VMAFX_LOG_LEVEL", "debug")
cfg, err := loadConfig()
if err != nil {
t.Fatalf("loadConfig: %v", err)
}
if cfg.logLevel != slog.LevelDebug {
t.Errorf("logLevel = %v, want debug", cfg.logLevel)
}
}

func TestLoadConfig_InvalidLogLevel(t *testing.T) {
// t.Setenv is incompatible with t.Parallel()
t.Setenv("VMAFX_CONTROLLER_ADDR", "localhost:8080")
t.Setenv("VMAFX_LOG_LEVEL", "TURBO")
_, err := loadConfig()
if err == nil {
t.Fatal("expected error for invalid VMAFX_LOG_LEVEL")
}
}

// ---------------------------------------------------------------------------
// Executor tests
// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -160,3 +217,107 @@ func TestExecutor_AIJobUnsupportedStage1(t *testing.T) {
t.Fatal("expected error — Stage 1 AI jobs have no input transport")
}
}

// ---------------------------------------------------------------------------
// Lifecycle integration test with mock controller
// ---------------------------------------------------------------------------

// TestNodeLifecycle_RegisterHeartbeatPullReport runs the core lifecycle:
// register → heartbeat (at least one) → pull one SCORING job →
// report result (error expected since no vmaf binary in test env).
func TestNodeLifecycle_RegisterHeartbeatPullReport(t *testing.T) {
if os.Getenv("VMAFX_INTEGRATION") == "" {
t.Skip("set VMAFX_INTEGRATION=1 to run lifecycle integration test")
}
t.Parallel()

mc := newMockController()
mc.jobToReturn = &controllerv1.Job{
Id: "integration-job-1",
Scoring: &controllerv1.ScoringParams{
Reference: "/nonexistent/ref.yuv",
Distorted: "/nonexistent/dis.yuv",
Model: "vmaf_v0.6.1",
},
}

_, addr := startMockController(t, mc)

// Set env for the node.
t.Setenv("VMAFX_CONTROLLER_ADDR", addr)
t.Setenv("VMAFX_NODE_ID", "test-node")
t.Setenv("VMAFX_LOG_LEVEL", "error")

cfg, err := loadConfig()
if err != nil {
t.Fatalf("loadConfig: %v", err)
}

log := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: slog.LevelError}))
n := &node{cfg: cfg, log: log}
n.gpuCap = gpu.Detect()
if n.cfg.backend == "" {
n.cfg.backend = n.gpuCap.PrimaryBackend()
}

conn, dialErr := grpc.NewClient(addr, grpc.WithTransportCredentials(insecure.NewCredentials()))
if dialErr != nil {
t.Fatalf("dial: %v", dialErr)
}
t.Cleanup(func() { _ = conn.Close() })

n.controllerCC = conn
n.client = controllerv1.NewVmafxControllerClient(conn)
n.executor = NewExecutor(nil, ai.NewRegistry(t.TempDir()), "cpu", log)

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

// Register.
if err := n.register(ctx); err != nil {
t.Fatalf("register: %v", err)
}
// RegisterNode was called.
select {
case <-mc.registered:
case <-time.After(2 * time.Second):
t.Fatal("RegisterNode not called within 2s")
}

// Pull one job.
pullCtx, pullCancel := context.WithTimeout(ctx, 5*time.Second)
resp, pullErr := n.client.PullWork(pullCtx, &controllerv1.PullWorkRequest{
NodeId: n.cfg.nodeID,
SessionToken: n.sessionToken,
Capability: &controllerv1.NodeCapability{
GpuVendor: string(n.gpuCap.Vendor),
Backends: n.gpuCap.Backends,
Concurrency: 1,
},
})
pullCancel()
if pullErr != nil {
t.Fatalf("PullWork: %v", pullErr)
}

job := resp.GetJob()
if job == nil {
t.Fatal("expected a job from PullWork")
}

// Execute (will fail since /nonexistent/ref.yuv does not exist — that is expected).
n.executeAndReport(ctx, job)

// ReportResult must have been called.
select {
case reported := <-mc.reportedResult:
if reported.GetJobId() != "integration-job-1" {
t.Errorf("reported job_id = %q, want %q", reported.GetJobId(), "integration-job-1")
}
if !reported.GetFinal() {
t.Error("expected final=true in ReportResult")
}
case <-time.After(3 * time.Second):
t.Fatal("ReportResult not called within 3s")
}
}
2 changes: 1 addition & 1 deletion cmd/vmafx-tune/cmd/compare_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ func TestStubSubcommands(t *testing.T) {
t.Fatalf("build vmafx-tune-go: %v\n%s", buildErr, buildOut)
}

for _, sub := range []string{"tune-per-shot", "fast", "corpus"} {
for _, sub := range []string{"ladder", "tune-per-shot", "fast", "corpus"} {
sub := sub
t.Run(sub, func(t *testing.T) {
t.Parallel()
Expand Down
Loading
Loading