feat(cli): add 'nix derivation source-origins' subcommand - #2
Merged
Conversation
Maps inputSrc store paths back to original filesystem source paths. During evaluation, nix copies source paths into the store via copyPathToStore(). This commit: 1. Adds a reverse mapping (storeToSrc) to EvalState that is populated alongside the existing srcToStore cache. 2. Exposes getSourceOrigin(StorePath) and getSourceOrigins() accessors on EvalState to query this mapping after evaluation. 3. Adds a new 'nix derivation source-origins' command that evaluates installables and, for each resulting derivation, prints a JSON mapping of each inputSrc store path to its original source path. This enables build-system tooling to determine which working-tree directories contributed to a derivation's build inputs - useful for monorepo CI to know which parts of the repo are affected by a change. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Add sourceStoreToOriginalPath map to EvalState for tracking store path → original filesystem path mappings - Set originalRootPath on accessors in git.cc (git root) and path.cc (flake dir) input schemes so mountInput() can record mappings - Implement recordPathOrigin() with 3-strategy resolution for builtins.path/cleanSourceWith sources that bypass storeToSrc - Modify addPath() to call recordPathOrigin() in both cached and new store path branches - Auto-disable eval cache in source-origins command - Use git+file:// scheme for correct monorepo root resolution Tested on diverse monorepo flakes: rust (atlas-ops, atlas, queue-urls), python (atlas), typescript (clank-stank, exa-deploy), go (readability_parser), and flakes/graph. All correctly resolve local source paths. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…e-level precision When a store path was created by cleanSourceWith with a broad source root (e.g. src = ../../../.), the sourcePath would report the monorepo root, making every file change trigger a match. Now we enumerate the actual store path contents (BFS) and report individual files as sourceFiles[]. The store path IS the filtered result — its contents are exactly what passed the cleanSourceWith filter. Each file is mapped back to its original source location using the sourcePath as the base. This gives file-level precision for all cleanSourceWith/builtins.path cases, making dependency graph filtering dramatically more precise. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…:// examples, and implementation details Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
source-origins now traces inputDrvs → inputSrcs through the entire build closure, not just the top-level derivation. This lets nix-dep-graph discover dependencies like uv path deps that are inputSrcs of derivations deep in the transitive closure. The old non-recursive behavior is available via --no-recursive. Fixes GIGA-2286. Co-Authored-By: Michael Fine <mfine15@gmail.com>
When recursing through the full derivation closure, most transitive derivations have no source origin mapping (they come from substituters or previous builds). The JSON value() call was throwing because the key exists with a null type, not a string. Use is_string() check instead. Co-Authored-By: Michael Fine <mfine15@gmail.com>
EvalState performs concurrent evaluation, so the sourceStoreToOriginalPath cache (written from mountInput and recordPathOrigin, read from getOriginalPath) must use the same concurrent container that the sibling caches (srcToStore, storeToSrc, etc.) use. Previously it was a plain std::map, which races with concurrent eval. Strategy 3 in recordPathOrigin iterated the map and inserted into it in the same loop; snapshot the entries via cvisit_all first, then insert after, to avoid deadlocking on the concurrent map's shared-lock.
* git.cc: propagate `originalRootPath` to the outer `MountedSourceAccessor` for git repos with submodules. Without this, the wrapper accessor returned to callers has no `originalRootPath`, and `mountInput` fails to register the source-to-filesystem mapping for those flakes. * eval.cc: drop the `snapshot.size() == 1` shortcut in Strategy 3 of `recordPathOrigin`. The shortcut attributed any root-path `SourcePath` to the sole registered source regardless of whether they are actually related, which could produce incorrect provenance when only one source has been registered so far. Always require a mount-identity match via `storeFS`.
directory_iterator with is_directory() follows symlinks, which hangs indefinitely on circular symlinks like 'link -> .' and can enumerate files outside the store path via symlinks to external directories. Treat symlinks as leaf entries instead.
In this codebase the srcToStore cache lives in fetchers::Settings (used inside fetchToStore()) rather than EvalState, so the doc's "reverse of srcToStore" wording no longer matches the code. storeToSrc is populated directly in copyPathToStore(). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🛡️ Guardian Review
|
Mechanical reformat of the ported series to satisfy this repo's pre-commit clang-format hook (nix develop -c ./maintainers/format.sh). No functional changes. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the
nix derivation source-originscommand (9 commits, authorship and messages preserved) from exa-labs/nix onto this repo's Determinate 3.21.2 base.The command evaluates installables and, for each derivation in the build closure (recursive by default,
--no-recursiveto disable), prints a JSON mapping of everyinputSrcstore path back to the original filesystem path that was copied into the store during evaluation. When a source is a filtered directory (cleanSourceWith/builtins.path),sourceFilesenumerates the store path contents for file-level precision. This lets build-graph tooling discover which working-tree paths feed each derivation.Implementation: a
storeToSrcreverse map populated incopyPathToStore(), asourceStoreToOriginalPathmap populated inmountInput()(git/path input schemes tag accessors withoriginalRootPath), andrecordPathOrigin()called fromaddPath()for filtered sources. Both maps useboost::concurrent_flat_mapfor parallel eval safety.Port notes (3.18.0 → 3.21.2 adaptations):
EvalState::srcToStoreno longer exists here (the source→store cache moved intofetchers::Settings/fetchToStore()), sostoreToSrcis recorded directly incopyPathToStore()after fetching; a follow-up commit drops the stalesrcToStorecross-reference from the command docs.Test plan
nix build .#defaulton aarch64-darwin: all components compile; the only functional-test failures (remote-store,bash-profile,impure-env) also fail identically on plainmainin the same local sandbox (unresolvable build-user name), i.e. pre-existing and environment-specificbuiltins.pathwith a filter:sourcePathresolves to the original filesystem paths andsourceFileslists exactly the files that passed the filter;--no-recursive,--pretty, and--helpworkGenerated with Devin