forked from cargo2nix/cargo2nix
-
Notifications
You must be signed in to change notification settings - Fork 0
Feature/crq 016 nixify #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jmikedupont2
wants to merge
113
commits into
release-0.12
Choose a base branch
from
feature/CRQ-016-nixify
base: release-0.12
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ch for newest failed)
…ue to resolution issues
Refactored the `cargo` override in `flake.nix` to use a clearly named `extraCargoBuildFlags` variable. This allows for safely passing additional arguments to the `cargo build` command in a structured and configurable way.
Updated the time-rs submodule to its latest commit, which includes changes to flake.nix, time-macros/Cargo.toml, and a new Cargo.nix file.
Corrected the syntax for defining `extraCargoBuildFlags` within the `cargo` override in `flake.nix`. This resolves the build error caused by incorrect attribute set definition.
This commit introduces the `repo_manager` tool, designed to automate the forking and patching of Rust crates as Git submodules. Key changes include: - Added `tools/repo_manager` with its source code and configuration. - Updated `Cargo.lock` and `Cargo.toml` to reflect new dependencies and tool. - Modified `Makefile`, `flake-phase1.nix`, `flake.lock`, and `flake.nix` to integrate the new tool and build process. - Added `tasks/implement_fork_and_patch_submodule.toml` to outline the task for implementing the submodule logic. - Added `generate_gh_fork_commands.sh` to generate commands for forking repositories.
…te main.rs The `execute_actions_plan` and `update_cargo_config` functions have been moved into their respective files: - `src/execute-actions-plan.rs` - `src/update-cargo-config.rs` The `src/main.rs` file has been updated to use these new modules, resolving compilation errors related to duplicate function definitions. Changes to `Cargo.toml` and `Cargo.lock` reflect necessary dependency updates for the successful build.
This commit introduces a granular cache invalidation mechanism for `cargo-repo-sync`
by modeling the project as a Git tree structure with nested hashes.
Key changes include:
- **`RollupLock` Enhancement:** The `RollupLock` struct has been extended to store
granular hashes for `Cargo.toml` files, Rust source files, crates, submodules,
and an overall project root hash. This replaces the single `last_snapshot_time`
as the primary staleness indicator.
- **Git Tree Traversal for Hashing:** The `create_snapshot` function has been
rewritten to strictly adhere to the Git object model. It now uses a recursive
helper function (`traverse_git_tree`) that leverages `git2-rs` to:
- Calculate content hashes (Git Oids) for individual files (`Cargo.toml`, `.rs`).
- Compute composite hashes for directories (trees) based on their children.
- Determine composite hashes for submodules, incorporating their HEAD commit,
tree hash, and working directory status.
- Aggregate these into composite crate hashes and a final project root hash.
- **Staleness Check Refinement:** The `run_submodule_status` function now
utilizes the `last_snapshot_time` (with a 1-year threshold) as an initial
check to skip the expensive `create_snapshot` process if the overall cache
is deemed fresh. Further granular checks will be implemented in subsequent
phases using the newly populated granular hashes.
- **Dependency Update:** Added `sha2` crate for robust hashing.
This lays the groundwork for generating reusable, granular Nix expressions
per crate and managing cargo overrides for submodules based on precise
change detection.
Adds Python scripts to: - Parse cargo depgraph output and calculate crate layers (process_depgraph.py). - Merge layer information with non-vendored module usage counts (merge_depgraph_data.py). - Analyze Layer 0 modules to find the most used (analyze_layer0_usage.py).
This commit introduces the new `GenerateWorkspaces` command to the CLI,
which analyzes crate dependencies and generates a clustered workspace
structure.
Key changes include:
- Added `GenerateWorkspaces` command to `cli/args.rs`.
- Implemented the core logic for `run_generate_workspaces_command` in
`cli/commands/generate_workspaces.rs`, including:
- Parsing '.dot' and 'tree.txt' files for dependency analysis.
- Grouping crates by topological layers.
- Generating 'Cargo.toml' files for each layer's workspace,
including '[workspace]', '[workspace.dependencies]', and
'[patch.crates-io]' sections.
- Fixed various compilation errors across multiple files, including:
- Missing 'use' statements.
- Type mismatches in 'cargo_config_generator.rs'.
- Corrected 'or_insert_auto' to 'or_insert' usage in 'toml_edit'.
- Resolved 'non-exhaustive patterns' in 'main.rs'.
- Made 'run_process_tt_txt_command' public and created its skeleton.
Note: A persistent error regarding "binding modifiers" in 'cli/args.rs'
remains, likely due to 'clap' compatibility with Rust 2024 edition. This
issue cannot be resolved by direct code modification.
204aee5 to
84f5050
Compare
…architecture - Decoupled nix-generator-lib from cargo-edit-tool to break cyclic dependency. - Introduced cargo-edit-lib to define GitAdapter and CargoMetadataProvider traits. - Modified cargo-edit-tool to implement these new traits. - Updated cargo_repo_sync_lib_test to use the new trait-based approach and fixed compilation errors. - Made cargo_metadata optional in cargo-edit-tool via feature flag.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
User description
Pull Requests
Many files are generated. When you draft your changes, please separate out
commits that affect:
Keeping these changes isolated in specific commits makes it much easier to pull
in your changes in parallel with other features. Maintainers may harvest your
changes. We only guarantee to preserve authorship in the git log (when we
remember to do so).
Creating pull requests
for breaking changes
changes applied with authorship preserved (if we remember)
PR Type
Enhancement, Configuration changes
Description
Comprehensive Nixification of cargo2nix project: Introduced multiple flake configurations (
flake.nix,flake-phase1.nix,flake-phase2.nix,full-flake.nix,flake-cargo2nix.nix) for reproducible Nix-based development and buildsGit submodule infrastructure: Added 13 Git submodules pointing to
meta-introspectorforks onfeature/CRQ-016-nixifybranch for Rust ecosystem projects and dependenciesBuild automation: Created comprehensive Makefile with recursive submodule management, Nix build orchestration, and automated commit/push workflows
Nix code formatting standardization: Reformatted multiple Nix files across overlays and examples with consistent indentation, leading comma style, and improved readability
Dependency updates: Updated
Cargo.tomlwith Git-based patch dependencies, upgradedcargo-platformandtomlversions, and added[patch.crates-io]section for dependency overridesBug fixes: Added handling for
CfgExpr::TrueandCfgExpr::Falsevariants in platform configuration; improved error messaging for missing version attributesDocumentation and task definitions: Added conceptual mapping between elliptic curves and build systems, comprehensive task definitions for
cargo-git-manageimplementation, and build failure diagnosticsUtility scripts: Created shell scripts for submodule processing, Nix build execution, cargo2nix executable discovery, and package introspection
Test and example configurations: Added simple example project with flake configuration, test flakes for Rust toolchain and time-macros validation, and template files for reproducible builds
Diagram Walkthrough
File Walkthrough
11 files
Cargo.nix
Nix code formatting and style improvementsexamples/3-cross-compiling/Cargo.nix
leading commas
indentation
overrides.nix
Nix formatting and parameter restructuringoverlay/overrides.nix
consistent indentation
perlreference to useinheritsyntaxmkcrate.nix
Nix parameter and formatting standardizationoverlay/mkcrate.nix
=instead of=with differentspacing
user-facing.nix
Nix function parameter and logic formattingoverlay/make-package-set/user-facing.nix
Cargo.nix
Nix code formatting and structure improvementsexamples/1-hello-world/Cargo.nix
run-tests.nix
Nix test runner formatting improvementsoverlay/run-tests.nix
workspace-shell.nix
Workspace shell formatting and dependency updatesoverlay/workspace-shell.nix
noBuild.workspacemapping to remove empty function callopenssl.devto native build inputsRUST_SRC_PATHvariable (extra closing brace)flake.nix
Cross-compiling example flake formattingexamples/3-cross-compiling/flake.nix
in recblock with improved indentationflake.nix
Bigger project example flake formattingexamples/2-bigger-project/flake.nix
in recblock with improved indentationflake.nix
Hello-world example flake formattingexamples/1-hello-world/flake.nix
in recblock with improved indentationflake.nix
Independent packaging example flake formattingexamples/4-independent-packaging/flake.nix
in recblock with improved indentation16 files
full-flake.nix
Complete flake configuration for cargo2nix developmentfull-flake.nix
overlays
heapless,allocator-api2,time-macros,time,cargo, andopenssl-sysflake.nix
Flake simplification and input consolidationflake.nix
branch
flake-phase2.nix
Phase 2 flake for self-built cargoflake-phase2.nix
flake-phase1.nix
Phase 1 flake for pre-built cargoflake-phase1.nix
flake-cargo2nix.nix
Dedicated flake for cargo2nix binary buildflake-cargo2nix.nix
flake.nix
Simple example flake configurationexamples/simple/flake.nix
flake.nix
Rust toolchain test flaketest_flakes/rust-toolchain-test/flake.nix
flake.nix
Time-macros test flake configurationtest_flakes/time-macros-test/flake.nix
temp_shell.nix
Temporary shell environment configurationtemp_shell.nix
ci.yml
CI workflow nixpkgs and cargo2nix input updates.github/workflows/ci.yml
branch
Makefile
Recursive Makefile for Submodule and Nix Build ManagementMakefile
cargo2nix,nix-build, andnix-evaloperationsupdate-submodules-and-buildtarget supporting configurable recursiondepth
submodule-build-and-pushtarget that injectscargo2nixmetadatainto
Cargo.tomland executes build pipelineand validation
Makefile
Submodule Build Pipeline and Configuration Managementsubmodules/Makefile
standardized build pipeline
.cargo/config.toml, runcargo vendor, and executecargo2nixfor each submodulecargosubmodule with Nix channel andversion customization
check-cargo-configtarget for validating configurationconsistency across submodules
Makefile.submodule
Submodule Template Makefile with cargo2nix IntegrationMakefile.submodule
cargo2nixintegrationCargo.tomlforcargo2nix_pathconfiguration
Nix store
.gitmodules
Git Submodule Configuration for Nixify Feature Branch.gitmodules
meta-introspectorforks onfeature/CRQ-016-nixifybranchrust-base64,config-rs,ron-rs,time-rscargo,rust-ini,ordered-multimap-rs,heapless,gitoxide,faster-hex,dlv-list-rsallocator-api2,dashmap,hashbrownflake.nix.template
Nix Flake Template for Reproducible Build Environmentflake.nix.template
nixpkgs,flake-utils,cargo2nix, andrust-overlay2025-09-16withrustBuilder.makePackageSetstatixandopenssl_1_1packagesMakefile.template
Submodule Makefile Template with Nix IntegrationMakefile.template
targets
nix-buildtarget usingnix developwithCARGO2NIX_ROOToverride
nix-flake-buildtarget for direct Nix flake builds1 files
main.rs
Error handling and API compatibility improvementssrc/main.rs
guidance
Packages::AlltoPackages::All(vec![])for API compatibility1 files
platform.rs
Platform configuration expression handlingsrc/platform.rs
CfgExpr::TrueandCfgExpr::FalsevariantsBoolExprvalues for complete patternmatching
9 files
main.rs
Simple example main programexamples/simple/src/main.rs
generation
randcrate dependencyrun.sh
Submodule processing and build automation scriptsubmodules/run.sh
submodule
automorphism.sh
System boot visualization scriptautomorphism.sh
find_cargo2nix_executables.sh
Cargo2nix executable discovery utilityfind_cargo2nix_executables.sh
commit.sh
Submodule commit and push automationsubmodules/commit.sh
build_nix.sh
Nix build execution scriptbuild_nix.sh
build_and_log.sh
Cargo build logging scriptbuild_and_log.sh
getallpkgs.sh
Package discovery utility scriptgetallpkgs.sh
nixpkgs
Cargo.toml
Simple example Cargo manifestexamples/simple/Cargo.toml
randdependency for demonstration24 files
automorphism.md
Elliptic Curve Conceptual Mapping and Automorphism Documentationautomorphism.md
stylized text representations
project structures
curve data model
using cryptographic concepts
elliptic_curve_concept_mapping.md
Elliptic Curves and Build Systems Conceptual Frameworkdocs/elliptic_curve_concept_mapping.md
Cargo.tomlfiles and ellipticcurves (identity and state curves)
structure
consistency checks
(Make, Autotools, Cargo)
README.md
Documentation for cargo-git-manage Tool and Updated InstructionsREADME.md
cargo-git-managetool with key featuresand usage examples
cargo2nixinvocation to referencemeta-introspectorfork withspecific release branch
cargo-git-managewith development shellintegration
makePackagSettomakePackageSetreboot.md
Build Failure Diagnosis and Recovery Instructionsreboot.md
noBrokenSymlinkserror incrate-cargo-0.86.0derivationfeature 'edition2024'requirement innix-files-comparatorCargo.tomledition2024featureand regenerating
Cargo.nixautomorphism.latex
LaTeX Mathematical Formulation of Build System Automorphismsautomorphism.latex
classification conjecture
(G, R,Φ)automorphic orbit
automorphic forms
architect_unified_nix_runtime.toml
Unified Nix Runtime Architecture Task Definitiontasks/architect_unified_nix_runtime.toml
cargovendor,cargo update,cargo2nix, andcargo-git-managedependency resolution, Nix-native operations, and bootstrap
integration
gitoxideand file systemmonitoring
implement_git_state_tracking.toml
Git State Tracking Implementation Tasktasks/implement_git_state_tracking.toml
cargo-git-managehas_uncommitted_changes,has_unpushed_commits,is_remote_ahead,get_head_commit_hashgit2crateintegrate_minizinc_with_rust.toml
MiniZinc Constraint Solver Integration Tasktasks/integrate_minizinc_with_rust.toml
in
cargo-git-managesolutions
define_minizinc_data_structures.toml
MiniZinc Data Structure Definition Tasktasks/define_minizinc_data_structures.toml
inputs and outputs
.dznformat and deserialization fromMiniZinc output
design
implement_nix_build_command.toml
Nix Build Command Implementation Tasktasks/implement_nix_build_command.toml
NixBuildCommandimplementingCargoCommandtrait
needs_executionlogic based onCargo.nixchanges and buildoutput status
executemethod to performnix buildoperationnixify_minizinc_workflow.toml
MiniZinc Nix Derivation Packaging Tasktasks/nixify_minizinc_workflow.toml
solvers
derivations
implement_old_version_reference.toml
Old Version Reference Bootstrap Implementation Tasktasks/implement_old_version_reference.toml
bootstrap
implement_nix_build_command.tomlimplement_submodule_add_core_logic.toml
Submodule Add Command Implementation Tasktasks/implement_submodule_add_core_logic.toml
cargo git-manage submodule addcorelogic
git2crate for programmatic submodule addition.gitmodulesconfiguration and error handling requirementsimplement_git_add_commit_push_command.toml
Git Add-Commit-Push Command Implementation Tasktasks/implement_git_add_commit_push_command.toml
GitAddCommitPushCommandimplementingCargoCommandtraitneeds_executionlogic based on uncommitted changes andunpushed commits
executemethod forgit add,git commit, andgit pushoperations
integrate_new_commands_into_main.toml
New Commands Integration into Main Function Tasktasks/integrate_new_commands_into_main.toml
cargo-git-managemainfunction
GitPullbeforeCargoUpdate,NixBuildbeforeGitAddCommitPushrefine_cargo_update_needs_execution.toml
Cargo Update Command Refinement Tasktasks/refine_cargo_update_needs_execution.toml
CargoUpdateCommand::needs_executionlogicCargo.tomlvsCargo.locktimestampcomparison
implement_git_pull_command.toml
Git Pull Command Implementation Tasktasks/implement_git_pull_command.toml
GitPullCommandimplementingCargoCommandtrait
needs_executionlogic based on remote HEAD comparisonexecutemethod forgit pulloperationimplement_acl_enforcement.toml
ACL Enforcement Implementation Tasktasks/implement_acl_enforcement.toml
operations
operation implementations
refine_cargo2nix_needs_execution.toml
Cargo2nix Command Refinement Tasktasks/refine_cargo2nix_needs_execution.toml
Cargo2NixCommand::needs_executionlogicCargo.toml,Cargo.lock, and.rsfilechanges
Cargo.nixstaleness detectionrefine_cargo_vendor_needs_execution.toml
Cargo Vendor Command Refinement Tasktasks/refine_cargo_vendor_needs_execution.toml
CargoVendorCommand::needs_executionlogicCargo.lockchanges and vendordirectory staleness
implement_submodule_add.toml
Submodule Add Logic Implementation Tasktasks/implement_submodule_add.toml
git2crate.gitmodulesconfiguration and change stagingdefine_cli_structure.toml
CLI Structure Definition Tasktasks/define_cli_structure.toml
cargo git-manageCLI structure usingclapsubmoduleandbranchoperationsimplement_submodule_update.toml
Submodule Update Logic Implementation Tasktasks/implement_submodule_update.toml
cargo git-manage submodule updatelogicimplement_submodule_remove.toml
Submodule Remove Logic Implementation Tasktasks/implement_submodule_remove.toml
cargo git-manage submodule removelogic.gitmodulesremoval,git rm, and cleanup operations1 files
Cargo.toml
Dependency Updates and Git-Based Patch DependenciesCargo.toml
cargodependency from pinned version to wildcard (*)cargo-platformfrom0.2.0to0.3.1andtomlfrom0.8.2to0.9.8meta-introspectorforks on
feature/CRQ-016-nixifybranch[patch.crates-io]section with overrides fortime,base64,ron,config, andhashbrown[workspace]section (commented out members)1 files
time-macros-unused-imports.patch
Time-Macros Unused Imports Patchpatches/time-macros-unused-imports.patch
time-macroscrate addressing unused importswarning
#[allow(unused_imports)]attribute tosrc/offset.rsstd::iter::Peekableimport28 files
Summary by CodeRabbit
New Features
Documentation
Chores