fix(compaction): clean exact job temp directory

[efegokdemir]

Summary

• replace lossy partition-prefix temp cleanup with exact JobID-owned directory cleanup
• prevent one completed compaction job from deleting another concurrent job's working directory
• reuse the existing JobID validation before removing the temp path
• preserve parent-side cleanup for subprocess crash/OOM cases
• leave startup orphan cleanup behavior unchanged

Regression coverage

• reproduces the colliding-name shape from Compaction temp-dir cleanup can os.RemoveAll a concurrently running job's working directory #749
• proves two distinct jobs would have shared the old underscore-based cleanup prefix
• verifies cleanup for job A removes only job A's directory
• verifies job B's directory and sentinel remain intact
• covers unsafe/invalid JobID protection

Validation

• go test ./internal/compaction/...
• go test -race ./internal/compaction/...
• go vet ./internal/compaction/...
• go test ./...
• go build ./...
• git diff --check
• go build -tags=duckdb_arrow ./...
• go vet -tags=duckdb_arrow ./...
• go test -tags=duckdb_arrow -race ./...

All pass locally.

Closes #749

[xe-nvdk]

Thanks — this is the right fix for #749, and the test reproduces the issue's exact collision (a + a/a_a_cpu/… and a_a + a_a/cpu/… both folding to a_a_a_a_cpu_2026_09_12_14_) before proving job A's cleanup leaves job B's directory and sentinel alone. I traced the part the tests cannot: the subprocess really does create its working directory at filepath.Join(TempDirectory, JobID) (job.go:309), with the DuckDB spill directory beneath it (subprocess.go:175), so removing that one path cleans everything the job made, on success and on crash alike. validateJobID already refuses separators, .., dot-prefixed and empty IDs, and the parent-generated ID ({db}_{partition}_{unixnano}_b{batch} with / in a pseudo-database folded to .) always passes it. Build, gofmt, vet and the compaction suite under -race are clean, and CI is green.

One small change before merge, because this is a deletion path:

The path is now computed in three places that have to agree. job.go:309 builds it to create the directory, subprocess.go:175 builds it for the spill dir, and cleanupSubprocessTempDir builds it to delete. Today they are the same expression; the whole fix depends on that staying true, and nothing pins it. Add one helper and use it at all three sites:

// jobTempDir is the working directory a job owns under the compaction temp
// root. Creation, the spill directory and parent-side cleanup all go through
// it so cleanup can never drift from what the subprocess actually made.
func jobTempDir(tempDirectory, jobID string) string {
	return filepath.Join(tempDirectory, jobID)
}

and a test asserting that the directory Job.Run creates for a given TempDirectory/JobID is the one cleanupSubprocessTempDir removes (create via the helper, clean via the function, assert gone). That is the same lesson as #752: a property that three copies happen to share is not a property until one function owns it.

Two notes, not blocking:

• The old sweep also happened to remove leftover directories of the same partition from earlier runs whose parent-side cleanup had failed (RemoveAll returning an error, say EBUSY). Those now wait for CleanupOrphanedTempDirs at the next boot. That is acceptable, and the issue's other hazard (a daily prefix matching every hourly directory of that day) is gone with the prefix itself; both are worth one sentence in the release note.
• main moved after your base, so the note conflicts positionally; rebase when you push the change above and keep your entry at the top of ## Bug fixes.

[xe-nvdk]

The change I asked for is in and it is the right shape: jobTempDir is the single owner of the path, and all three sites go through it — creation (job.go), the DuckDB spill directory (subprocess.go) and parent-side cleanup (manager.go), plus the two log lines. TestCleanupSubprocessTempDirRemovesJobTempDir pins the coupling (create through the helper, remove through the function), the collision test still proves job A's cleanup leaves job B's directory and sentinel alone, and TestCleanupSubprocessTempDirRejectsUnsafeJobID keeps a bad ID from ever resolving to the temp root.

Verified on the head: build, vet and the compaction suite under -race are clean (the two gofmt hits in this package are pre-existing files you did not touch), CI green. The release note says what an operator will notice, including that a failed parent cleanup now waits for the startup sweep. I pushed a merge of main onto the branch to resolve the positional release-notes conflict (#785 landed first); your entry stays at the top of ## Bug fixes, nothing else changed.

Nice fix, and thanks for reproducing the exact prefix collision from the issue in the test.