Skip to content

Commit 05633a8

Browse files
committed
merge
2 parents 8d547fa + 0517d40 commit 05633a8

File tree

3 files changed

+63
-28
lines changed

3 files changed

+63
-28
lines changed

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
target
1+
nix-crates-index/

README.md

Lines changed: 57 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,44 @@
11
# nixcrates
2-
a drop-in replacement for `cargo` which yet uses crates.io index.
2+
A drop-in replacement for `cargo` which yet uses crates.io index.
33

4-
**note:** this is an early draft don't expect much.
4+
**Warning**: crates.io-index uses 'semantic versioning' (see http://semver.org/) and we don't support this here. 'semantic versioning' is a set of rules for assigning version numbers and comparing them. Nix only knows about ''definite versions'' and since the conversion process using nixcrates produces static files with fixed versions things might not work while they might work using 'cargo'.
5+
Having only ''definite versions'' implies no dependency calculation, for instance by using SAT to solve them, has prooven to be a great relief in many situations. However, it would require a **cargo2nix** tool to have that.
56

6-
**warning**: crates.io-index uses 'semantic versioning' (see http://semver.org/) and we don't support this here. 'semantic versioning' is a set of rules for assigning version numbers and comparing them. Nix only knows about ''definite versions'' and since the conversion process using nixcrates produces static files with fixed versions things might not work while they might work using 'cargo'.
7-
having only ''definite versions'' implies no dependency calculation, for instance by using SAT to solve them, has prooven to be a great relief in many situations. however, it would require a **cargo2nix** tool to have that.
7+
**Warning**: This repo has 42mib+ because it still contains a checkin of the nixified nix-crate-index which later was layed of into its own repository on `https://github.com/nixcloud/nix-crates-index.git`. Someone has to do a 'git rebase' and remove it...
88

9-
# usage
9+
# Usage
1010

11-
creating a initial crates.io-index conversion:
11+
We describe an initial procedure to keep old revisions around.
12+
13+
## Iterative update
14+
15+
If you are using our index from `https://github.com/nixcloud/nix-crates-index.git` then you need to do this:
16+
17+
1. update the crates.io-index repository:
18+
19+
cd crates.io-index
20+
git pull
21+
22+
2. use `nixcrates` to update it
23+
24+
cd nix-crates-index/
25+
# remove all files (nixcrates would not do that for us!)
26+
rm -Rf *
27+
nixcrates ../crates.io-index/ ./
28+
git add *
29+
30+
git rev-parse master
31+
fc336466a7ede6f14ce427deb5a64c5d02d14be0
32+
33+
git commit -m 'fc3364: new revision added'
34+
35+
3. afterwards try to build your packages and eventually update `nix-crates-index/all-carg-packages.nix` to reflect dependency changes
36+
37+
nix-build -A allTargets
38+
39+
## Initialization
40+
41+
Creating a initial crates.io-index conversion:
1242

1343
nix-shell -p cargoc crates
1444
git clone https://github.com/rust-lang/crates.io-index
@@ -17,37 +47,35 @@ creating a initial crates.io-index conversion:
1747
git clone https://github.com/nixcloud/nix-crates-index.git
1848
cargo run ../crates.io-index/ ./nix-crates-index/
1949

20-
this will then create a 1:1 directory structure and nixify all crates.io JSON files.
50+
This will then create a 1:1 directory structure and nixify all crates.io JSON files.
51+
52+
**Warning**: Normally this should not be done since you want iterative updates.
2153

22-
warning: this repo contains a modified version of a converted crates.io-index
54+
**Warning**: This way we need to bootstrap `nixcrate` with `cargo` and rustStable from `NixOS`/`nixpkgs`.
2355

24-
# build examples
56+
# Build examples
2557

26-
you need rustc nightly from most recent nipxkgs master!
58+
Currently we use `rustc nightly` from most recent `nipxkgs master`!
2759

28-
example targets:
60+
Example targets:
2961

3062
nix-build default.nix -A nixcrates
3163
nix-build default.nix -A allTargets
64+
nix-build default.nix -A allCrates.rustache
3265

33-
warning: there are horrible hack in this project just to make this minimal set of packages work. we were under a lot of time-pressure and low in funding. on judgement day, hopefully god didn't see it!
34-
35-
# todos
36-
37-
* write better documentation
38-
66+
**Warning**: there are horrible hack in this project just to make this minimal set of packages work. we were under a lot of time-pressure and low in funding. on judgement day, hopefully god didn't see it!
3967

40-
# how it works
68+
# How it works
4169
We are parsing the json files of the crates.io repository and translating them into nix files. Considering dependencies we are currently only supporting versening via `~` and `^` as well as single versions. If anything else is used we are switching to the newest version. In case of `^` and `~` we are using the highest allowed version.
4270

43-
## how do versions look like in the created
71+
## How do versions look like in the created
4472
`{package_name}` is pointing on the newest version.
4573
`all__{package_name}` contains a set of all versions.
4674

4775
There are also several shortcuts to address the newest subversion. For every missing semver version number simply the newest is assumed.
4876
For example `all__{package_name}.{package_name}_0_1` points the the version of `{package_name}` with `0.1.X` where `X` is the highest number found.
4977

50-
## what failes and why?
78+
## What failes and why?
5179

5280
### Missing dependencies
5381
For some reason sometimes the crates.io json file is not listing all the dependencies. If this occours the missing dependencie has to be added by hand. However eddeting an autogenerated file is a pain.
@@ -65,3 +93,12 @@ Lets assume:
6593

6694
Now our repo would compile `dep_a` with `dep_c_0_2_0` and `dep_b` with `dep_c_0_1_2`. This is a problem as soon as `{dep_a}` as well as `{dep_b}` are exposing types from `{dep_c}` in their interfaces.
6795

96+
### Not supported stuff
97+
98+
The `flate2-example` uses `miniz-sys` which uses `bundled c code` that is something we don't support yet. To see the error:
99+
100+
nix-build -I nixpkgs=/home/joachim/Desktop/projects/nixos/nixpkgs -A allCrates.miniz-sys
101+
102+
### Dependencies
103+
104+
Say a crate you want does not build since a dependency is not set correctly, then you can use `lib.recursiveUpdate` in `nix-crates-index/all-carg-packages.nix` to change/extend them. This holds also true for `pkgs` from nixpkgs!

default.nix

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -23,15 +23,13 @@ rec {
2323
buildInputs = with allCrates; crateDeps ++ deps;
2424
buildPhase = ''
2525
${symlinkCalc buildInputs}
26-
du -a
27-
# ${rustc}/bin/rustc $src/main.rs --crate-type "bin" ${depsStringCalc deps} --emit=dep-info,link --crate-name nix_crates -L dependency=mylibs
26+
# du -a
2827
${rustc}/bin/rustc $src/main.rs --crate-type "bin" --emit=dep-info,link --crate-name nixcrates -L dependency=mylibs ${depsStringCalc deps}
2928
'';
3029
installPhase = ''
3130
mkdir -p $out/bin
3231
cp nixcrates $out/bin
3332
'';
34-
3533
};
3634

3735
getopts-example = stdenv.mkDerivation rec {
@@ -60,9 +58,9 @@ rec {
6058

6159
buildPhase = ''
6260
${symlinkCalc buildInputs}
63-
du -a mylibs
64-
ls -lathr mylibs
65-
echo ${depsString}
61+
# du -a mylibs
62+
# ls -lathr mylibs
63+
# echo ${depsString}
6664
# [pid 14162] execve("/nix/store/fff3jbf9vbqhmf6qjrmzhliq516x7yrf-rustc-1.11.0/bin/rustc", ["rustc", "src/main.rs", "--crate-name", "hello_flate2", "--crate-type", "bin", "-g", "--out-dir", "/home/joachim/Desktop/projects/fractalide/fetchUrl/hello_flate2/target/debug", "--emit=dep-info,link", "-L", "dependency=/home/joachim/Desktop/projects/fractalide/fetchUrl/hello_flate2/target/debug", "-L", "dependency=/home/joachim/Desktop/projects/fractalide/fetchUrl/hello_flate2/target/debug/deps", "--extern", "flate2=/home/joachim/Desktop/projects/fractalide/fetchUrl/hello_flate2/target/debug/deps/libflate2-d719035eaa7c6a88.rlib", "-L", "native=/home/joachim/Desktop/projects/fractalide/fetchUrl/hello_flate2/target/debug/build/miniz-sys-60c8d67696f63a43/out"], [/* 105 vars */]) = 0
6765
6866
${rustc}/bin/rustc $src/main.rs --crate-type "bin" --emit=dep-info,link --crate-name main -L dependency=mylibs ${depsString} -L native= #flate2=${allCrates.flate2_0_2_14}/libflate2.rlib
@@ -86,7 +84,7 @@ rec {
8684
# du -a
8785
# /run/current-system/sw/bin/ldd ./main
8886
./main
89-
du -a
87+
# du -a
9088
if [ -f foo.tar ]; then
9189
echo -e "---------\nSUCCESS: tar-example was executed successfully! \n--------"
9290
else

0 commit comments

Comments
 (0)