Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/go-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,16 @@ jobs:
- name: go vet
run: go vet ./...

- name: Install gosec
run: go install github.com/securego/gosec/v2/cmd/gosec@v2.21.4

- name: gosec (exclude generated)
run: |
# Generated protobuf code (gen/) carries G103 (unsafe.Slice) and
# G115 (int -> uint32 cgo casts) by design; -exclude-generated
# filters those. Source files must remain at 0 findings.
gosec -exclude-generated -quiet ./...

- name: go test (skip operator envtest — requires kubebuilder/etcd)
run: |
# Skip cmd/vmafx-operator: needs /usr/local/kubebuilder/bin/etcd
Expand Down
13,774 changes: 2,819 additions & 10,955 deletions CHANGELOG.md

Large diffs are not rendered by default.

15 changes: 13 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,25 @@ cythonize-deps: $(VENV_PIP)
# Fork-specific targets (lusoris). The upstream targets above are preserved as-is.
# ============================================================================

.PHONY: lint lint-c lint-py lint-sh lint-md format format-check sec sbom \
.PHONY: lint lint-c lint-py lint-sh lint-md lint-go format format-check sec sbom \
test-netflix-golden test-sanitizers test-fast install-hooks hooks-install help \
coverage coverage-html coverage-check assertion-density pr-check

# Top-level lint — runs every analyzer we own. Uses the meson compile_commands.json.
lint: lint-c lint-py lint-sh lint-md docs-fragments-check
lint: lint-c lint-py lint-sh lint-md lint-go docs-fragments-check
@echo "=== all lints passed ==="

# Go security scan (gosec). Skips generated files by default; surfaces every
# G* finding outside the gen/ tree. Source of truth for the gate added by
# the gosec-findings-fix sweep — keep the touched-file rule honest.
lint-go:
@command -v gosec >/dev/null || { \
echo "gosec not found — install via 'go install github.com/securego/gosec/v2/cmd/gosec@latest'; skipping"; \
exit 0; \
}
@echo "--- gosec (exclude-generated) ---"
@gosec -exclude-generated -quiet ./...

# Fragment-tree drift check (ADR-0221). Verifies CHANGELOG.md and
# docs/adr/README.md are in sync with their per-PR fragment trees.
docs-fragments-check:
Expand Down
38 changes: 38 additions & 0 deletions changelog.d/security/gosec-findings-fix-sweep.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
- **gosec sweep across the Go surface — fix all findings + add CI gate
(ADR-0983).** Ran `gosec ./...` against `cmd/vmafx-{controller,node,
server,operator,mcp,tune}/...`, `pkg/...`, and `gen/go/...`. Of 38
raw findings, six G115 (cgo int→uint32 casts) and four G103
(`unsafe.Slice` in protoc-generated `pb.go`) live in code we do not
hand-author and are gated via `-exclude-generated`. The remaining
26 fork-original findings are now resolved:

- **G304 (CWE-22): `cmd/vmafx-mcp/impl.go::describeModel` accepted
a caller-supplied `name` and joined it onto the repo root before
`os.Stat`-ing the result, so `{"name": "../../../etc/passwd"}`
would have reached `/etc/passwd`.** Routed the candidate through
`libvmaf.ValidatePath` so the lookup is bounded to
`libvmaf.AllowedRoots()`. New regression test
`cmd/vmafx-mcp/impl_gosec_test.go::TestDescribeModelRejectsTraversal`.
- **G306 / G301: `cmd/vmafx-tune/cmd/compare.go::writeOutput`
tightened from 0o644 / 0o755 to 0o600 / 0o750.** Comparison
reports include dataset path strings; restricting to the owner
matches the broader fork-write policy.
- **G104: `runVmafScore`'s `outFile.Close()` and `os.Remove(outPath)`
return values are now checked.** Close failure short-circuits the
request with an error; remove failure logs to stderr (best-effort
cleanup).
- **G204 / G304: 22 false-positive suppressions previously written
as `//nolint:gosec` were rewritten as `// #nosec G<rule> -- ...`
with citations.** gosec does not parse golangci-lint nolint
directives, so the old comments were dead text. Each new
suppression names the rule and the validating helper (constant
binary name, `libvmaf.ValidatePath`-filtered path, `os.CreateTemp`
output).
- **G202: `cmd/vmafx-controller/queue/queue.go::ListAll` SQL
concatenation flagged but verified safe** — only `repeatCommaQ`
output (pure `,?,?,...` placeholders) is concatenated; status
values bind through `placeholders...`. Suppressed with a citation.

CI gate added in `.github/workflows/go-ci.yml` (gosec step after
`go vet`, before `go test`) plus a `make lint-go` Makefile target.
`gosec -exclude-generated ./...` is the new zero-finding contract.
4 changes: 4 additions & 0 deletions cmd/vmafx-controller/queue/queue.go
Original file line number Diff line number Diff line change
Expand Up @@ -508,6 +508,10 @@ func (q *SQLiteQueue) ListAll(_ context.Context, statuses []string) ([]*Job, err
for i, s := range statuses {
placeholders[i] = s
}
// #nosec G202 -- The concatenated fragment is repeatCommaQ output, a
// pure ",?,?,..." placeholder string of length len(statuses)-1; no
// user data enters the SQL text. Status values bind through
// `placeholders...` as parameterised arguments.
query := "SELECT id, status, scoring, COALESCE(assigned_node,''), COALESCE(score,0), COALESCE(features,'{}'), COALESCE(error,''), created_at, updated_at FROM jobs WHERE status IN (?" + repeatCommaQ(len(statuses)-1) + ") ORDER BY created_at ASC"
rows, err = q.db.Query(query, placeholders...)
}
Expand Down
17 changes: 17 additions & 0 deletions cmd/vmafx-mcp/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,20 @@ path (default) and the direct cgo path introduced by ADR-0931 (opt-in via
accepts the four MCP-level model forms (`version=NAME`, `path=ABS`,
bare stem, abs/rel path). New forms require a coordinated update to
the Python server's resolver (`mcp-server/vmaf-mcp/src/vmaf_mcp/`).

7. **gosec G304 / G204 contract** (every `os.ReadFile` / `os.Open` /
`exec.Command*` in `impl.go`): any path or command variable consumed
by these calls MUST either (a) round-trip through
`libvmaf.ValidatePath` (caller-supplied paths), (b) originate from
`os.CreateTemp` / `os.MkdirTemp` (locally-generated temp paths), or
(c) come from `libvmaf.FindBinary` / `findVmafTune` / an
`exec.LookPath` of a fixed binary name. Adding a new subprocess call
site or file read without one of these gates means the new path is
directly attacker-influenced; either add the validation or annotate
`// #nosec G204` / `// #nosec G304` with a citation that names the
protecting helper. The CI gate (`gosec -exclude-generated` in
`go-ci.yml`) blocks the merge until one of the two is true.
`cmd/vmafx-mcp/impl_gosec_test.go::TestDescribeModelRejectsTraversal`
pins the `describeModel` allowlist; equivalent regressions for new
tools belong next to it. See
[ADR-0983](../../docs/adr/0983-gosec-findings-fix-sweep.md).
69 changes: 58 additions & 11 deletions cmd/vmafx-mcp/impl.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
Expand Down Expand Up @@ -183,8 +184,15 @@ func runVmafScore(ref, dis string, width, height int, pixfmt string, bitdepth in
return nil, fmt.Errorf("failed to create temp output file: %w", err)
}
outPath := outFile.Name()
outFile.Close()
defer os.Remove(outPath)
if closeErr := outFile.Close(); closeErr != nil {
return nil, fmt.Errorf("close temp output file: %w", closeErr)
}
defer func() {
if rmErr := os.Remove(outPath); rmErr != nil && !errors.Is(rmErr, os.ErrNotExist) {
// Best-effort cleanup; surface via stderr but don't fail the call.
fmt.Fprintf(os.Stderr, "runVmafScore: remove temp %s: %v\n", outPath, rmErr)
}
}()

argv := []string{
"-r", ref,
Expand All @@ -205,13 +213,18 @@ func runVmafScore(ref, dis string, width, height int, pixfmt string, bitdepth in
}
}

// #nosec G204 -- vmafBin resolved via libvmaf.FindBinary (env-overridable to
// a fixed allowlist of paths) and `ref`/`dis` are already libvmaf.ValidatePath-
// filtered in the calling handlers (handleVmafScore + handleVmafScoreEncoded).
cmd := exec.Command(vmafBin, argv...)
var stderr bytes.Buffer
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("vmaf exited %v: %s", err, stderr.String())
}

// #nosec G304 -- outPath is the value returned by os.CreateTemp above, not
// a caller-supplied path; it cannot escape /tmp.
data, err := os.ReadFile(outPath)
if err != nil {
return nil, fmt.Errorf("failed to read vmaf output: %w", err)
Expand Down Expand Up @@ -394,6 +407,8 @@ func handleRunBenchmark(ctx context.Context, _ map[string]any) (any, error) {
"VMAF_ROOT="+vmafRoot,
"VMAF_BIN="+libvmaf.FindBinary(),
)
// #nosec G204 -- script path is constructed from libvmaf.RepoRoot() and a
// fixed relative path (testdata/bench_all.sh); not caller-controlled.
cmd := exec.CommandContext(ctx, "bash", script)
cmd.Env = env
out, err := cmd.CombinedOutput()
Expand Down Expand Up @@ -495,6 +510,11 @@ rmse = float(np.sqrt(((pred-y)**2).mean()))
print(json.dumps({"model":model_path,"features":features_path,"split":split,"n":len(x),
"plcc":plcc,"srocc":srocc,"rmse":rmse,"columns":cols}))
`
// #nosec G204 -- "python3" is a fixed binary name, script is a constant
// string literal above; modelPath and featuresPath are passed through
// libvmaf.ValidatePath by the caller (handleEvalModelOnSplit), split is
// constrained to {train,val,test,all} server-side, inputName is a JSON
// schema-validated identifier.
cmd := exec.Command("python3", "-c", script, modelPath, featuresPath, split, inputName)
out, err := cmd.CombinedOutput()
if err != nil {
Expand Down Expand Up @@ -527,22 +547,22 @@ func handleCompareModels(ctx context.Context, args map[string]any) (any, error)
inputName := strArg(args, "input_name", "features")

var ranked []map[string]any
var errors []map[string]any
var modelErrors []map[string]any
for _, m := range rawModels {
mStr, _ := m.(string)
mp, err := libvmaf.ValidatePath(mStr)
if err != nil {
errors = append(errors, map[string]any{"model": mStr, "error": err.Error()})
modelErrors = append(modelErrors, map[string]any{"model": mStr, "error": err.Error()})
continue
}
r, err := delegateToPythonEval(mp, featuresPath, split, inputName)
if err != nil {
errors = append(errors, map[string]any{"model": mStr, "error": err.Error()})
modelErrors = append(modelErrors, map[string]any{"model": mStr, "error": err.Error()})
continue
}
if rm, ok := r.(map[string]any); ok {
if errMsg, hasErr := rm["error"]; hasErr {
errors = append(errors, map[string]any{"model": mStr, "error": errMsg})
modelErrors = append(modelErrors, map[string]any{"model": mStr, "error": errMsg})
continue
}
ranked = append(ranked, rm)
Expand All @@ -553,7 +573,7 @@ func handleCompareModels(ctx context.Context, args map[string]any) (any, error)
pj, _ := ranked[j]["plcc"].(float64)
return pi > pj
})
return map[string]any{"ranked": ranked, "errors": errors}, nil
return map[string]any{"ranked": ranked, "errors": modelErrors}, nil
}

// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -684,6 +704,9 @@ func extractFramePNG(ctx context.Context, yuv, outPNG string, width, height int,
"-frames:v", "1",
"-y", outPNG,
}
// #nosec G204 -- "ffmpeg" is a fixed binary name; argv values originate
// from libvmaf.ValidatePath-filtered YUV paths and integer geometry
// fields validated by extractFramePNG's caller (handleDescribeWorstFrames).
cmd := exec.CommandContext(ctx, "ffmpeg", argv...)
out, err := cmd.CombinedOutput()
if err != nil {
Expand Down Expand Up @@ -750,6 +773,9 @@ func handleProbeBackend(_ context.Context, args map[string]any) (any, error) {
}

t0 := time.Now()
// #nosec G204 -- vmafBin resolved via libvmaf.FindBinary (env-overridable
// to allowlist); argv values are tmpDir paths from os.MkdirTemp + literal
// flag arguments, no caller-controlled input.
cmd := exec.Command(vmafBin, argv...)
var stderr bytes.Buffer
cmd.Stderr = &stderr
Expand All @@ -767,6 +793,8 @@ func handleProbeBackend(_ context.Context, args map[string]any) (any, error) {
}, nil
}

// #nosec G304 -- outJSON is an os.MkdirTemp-derived path joined with a
// literal filename ("out.json"); not caller-controlled.
data, err := os.ReadFile(outJSON)
if err != nil {
return map[string]any{
Expand Down Expand Up @@ -839,6 +867,7 @@ func handleVmafVersion(_ context.Context, _ map[string]any) (any, error) {
var versionStr *string
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
// #nosec G204 -- vmafBin resolved via libvmaf.FindBinary; "--version" is literal.
out, err := exec.CommandContext(ctx, vmafBin, "--version").CombinedOutput()
if err == nil {
blob := string(out)
Expand Down Expand Up @@ -929,6 +958,8 @@ func ffprobeGeometry(path string) (width, height int, pixfmt string, bitdepth in
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// #nosec G204 -- "ffprobe" is a fixed binary; `path` is the libvmaf.ValidatePath
// output already filtered to AllowedRoots by handleVmafScoreEncoded.
out, e := exec.CommandContext(ctx, "ffprobe", args...).Output()
if e != nil {
return 0, 0, "", 0, fmt.Errorf("ffprobe: %w", e)
Expand Down Expand Up @@ -983,6 +1014,10 @@ func decodeToYUV(ctx context.Context, src, dst, pixFmt string) error {
"-pix_fmt", pixFmt,
"-y", dst,
}
// #nosec G204 -- "ffmpeg" is a fixed binary; `src` originates from
// libvmaf.ValidatePath via handleVmafScoreEncoded, `dst` is an
// os.MkdirTemp output path, pixFmt comes from a fixed lookup table
// (toFFmpegPixfmt).
cmd := exec.CommandContext(ctx, "ffmpeg", argv...)
out, err := cmd.CombinedOutput()
if err != nil {
Expand Down Expand Up @@ -1096,16 +1131,18 @@ func describeModel(nameOrPath string) (map[string]any, error) {
root := libvmaf.RepoRoot()
modelsDir := filepath.Join(root, "model")

// Step 1: try as a direct path.
// Step 1: try as a direct path. Validate via libvmaf.ValidatePath so the
// caller cannot reach files outside the allowlisted roots (e.g. via "../"
// traversal joined onto root).
candidate := nameOrPath
if !filepath.IsAbs(candidate) {
candidate = filepath.Join(root, candidate)
}
candidate, _ = filepath.Abs(candidate)
if fi, err := os.Stat(candidate); err == nil && !fi.IsDir() {
ext := strings.ToLower(filepath.Ext(candidate))
if validated, vErr := libvmaf.ValidatePath(candidate); vErr == nil {
ext := strings.ToLower(filepath.Ext(validated))
if modelExtensions[ext] {
return describeModelFile(candidate, root)
return describeModelFile(validated, root)
}
}

Expand Down Expand Up @@ -1159,6 +1196,9 @@ func describeModelFile(path, root string) (map[string]any, error) {
"feature_names": nil,
}
if ext == ".json" {
// #nosec G304 -- describeModelFile is only reachable from describeModel
// which validates the path via libvmaf.ValidatePath OR walks the
// in-repo model/ dir (filepath.WalkDir).
data, err := os.ReadFile(path)
if err == nil {
var payload map[string]any
Expand Down Expand Up @@ -1208,6 +1248,9 @@ func handleRunCompare(ctx context.Context, args map[string]any) (any, error) {
argv = append(argv, "--no-parallel")
}

// #nosec G204 -- vmaftune is resolved via findVmafTune (fixed candidate
// list); argv values are flag-prefixed strings derived from JSON-Schema
// validated tool arguments.
out, err := exec.CommandContext(ctx, vmaftune, argv...).Output()
if err != nil {
return nil, fmt.Errorf("vmaf-tune compare failed: %w", err)
Expand Down Expand Up @@ -1246,6 +1289,8 @@ func handleRunLadder(ctx context.Context, args map[string]any) (any, error) {
argv = append(argv, "--framerate", strconv.FormatFloat(floatArg(args, "framerate", 0), 'f', -1, 64))
}

// #nosec G204 -- vmaftune resolved via findVmafTune; argv values are
// schema-validated tool arguments prefixed by literal flags.
out, err := exec.CommandContext(ctx, vmaftune, argv...).Output()
if err != nil {
return nil, fmt.Errorf("vmaf-tune ladder failed: %w", err)
Expand Down Expand Up @@ -1290,6 +1335,8 @@ func handleRunTunePerShot(ctx context.Context, args map[string]any) (any, error)
argv = append(argv, "--scene-threshold", strconv.FormatFloat(floatArg(args, "scene_threshold", 0), 'f', -1, 64))
}

// #nosec G204 -- vmaftune resolved via findVmafTune; argv values are
// schema-validated tool arguments prefixed by literal flags.
out, err := exec.CommandContext(ctx, vmaftune, argv...).Output()
if err != nil {
return nil, fmt.Errorf("vmaf-tune tune-per-shot failed: %w", err)
Expand Down
58 changes: 58 additions & 0 deletions cmd/vmafx-mcp/impl_gosec_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
// Copyright 2026 Lusoris. All rights reserved.
// Use of this source code is governed by the BSD-3-Clause-Plus-Patent
// license that can be found in the LICENSE file.
//
// impl_gosec_test.go — regression tests for the gosec-G304 hardening of
// describeModel (path-traversal under root). The pre-fix code unconditionally
// joined the caller-supplied `name` argument onto the repo root and called
// os.Stat / os.ReadFile, allowing a "../../../etc/passwd"-style request to
// reach files outside the model allowlist. The fix routes the join through
// libvmaf.ValidatePath so the lookup is bounded to AllowedRoots().

package main

import (
"strings"
"testing"
)

// TestDescribeModelRejectsTraversal exercises the post-fix invariant: a
// caller-supplied path that resolves outside libvmaf.AllowedRoots() must NOT
// be silently opened. The pre-fix implementation would Stat /etc/passwd via
// "../../../etc/passwd"; the fix makes the candidate lookup fall through to
// the in-repo model/ walker which returns a "not found" error instead.
func TestDescribeModelRejectsTraversal(t *testing.T) {
cases := []string{
"../../../etc/passwd",
"../../../../etc/shadow",
"/etc/passwd",
"/proc/self/environ",
}
for _, name := range cases {
t.Run(name, func(t *testing.T) {
_, err := describeModel(name)
if err == nil {
t.Fatalf("describeModel(%q) returned nil error; expected rejection", name)
}
// The error should come from the fallback "not found" or the
// allowlist gate, not from a successful sneak-open.
msg := err.Error()
if !strings.Contains(msg, "not found") && !strings.Contains(msg, "allowlist") {
t.Fatalf("describeModel(%q) unexpected error %q; expected not-found/allowlist gate", name, msg)
}
})
}
}

// TestDescribeModelRejectsEmptyName confirms the explicit empty-name guard in
// handleDescribeModel still holds — defence in depth around the ValidatePath
// hardening.
func TestDescribeModelRejectsEmptyName(t *testing.T) {
_, err := handleDescribeModel(t.Context(), map[string]any{"name": ""})
if err == nil {
t.Fatalf("handleDescribeModel({name:''}) returned nil error; expected rejection")
}
if !strings.Contains(err.Error(), "required") {
t.Fatalf("unexpected error %q; expected 'name is required'", err.Error())
}
}
Loading
Loading