Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Auto merge of #31123 - alexcrichton:who-doesnt-want-two-build-systems…
…, r=brson This series of commits adds the initial implementation of a new build system for the compiler and standard library based on Cargo. The high-level architecture now looks like: 1. The `./configure` script is run with `--enable-rustbuild` and other standard configuration options. 2. A `Makefile` is generate which proxies commands to the new build system. 3. The new build system has a Python script entry point which manages downloading both a Rust and Cargo nightly. This initial script also manages building the build system itself (which is written in Rust). 4. The build system, written in rust and called `bootstrap`, architects how to call `cargo` and manages building all native libraries and such. One might reasonably ask "why rewrite the build system?", which is a good question! The Rust project has used Makefiles for as long as I can remember at least, and while ugly and difficult to use are undeniably robust as they contain years worth of tweaking and tuning for working on as many platforms in as many situation as possible. The rationale behind this PR, however is: * The makefiles are impenetrable to all but a few people on this planet. This means that contributions to the build system are almost nonexistent, and furthermore if a build system change is needed it's incredibly difficult to figure out how to do so. This hindrance prevents us from doing some "perhaps fancier" things we may wish to do in make. * Our build system, while portable, is unfortunately not infinitely portable everywhere. For example the recently-introduced MSVC target is quite unlikely to have `make` installed by default (e.g. it requires building inside of an MSYS2 shell currently). Conversely, the portability of make comes at a cost of crazy and weird hacks to work around all sorts of versions of software everywhere, especially when it comes to the configure script and makefiles. By rewriting this logic in one of the most robust platforms there is, Rust, we get to assuage all of these worries for free! * There's a standard tool to build Rust crates, Cargo, but the standard library and compiler don't use it. This means that they cannot benefit easily from the crates.io ecosystem, nor can the ecosystem benefit from a standard way to build this repository itself. Moving to Cargo should help assuage both of these needs. This has the added benefit of making the compiler more approachable for newbies as working on the compiler will just happen to be working on a large Cargo project, all the same standard tools and tricks will apply. * There's a huge amount of portability information in the main distribution, for example around cross compiling, compiling on new OSes, etc. Pushing this logic into standard crates (like `gcc`) enables the community to immediately benefit from new build logic. Despite these benefits, it's going to be a long road to actually replace our current build system. This PR is just the beginning and doesn't implement the full suite of functionality as the current one, but there are many more to follow! The current implementation strategy hopes to look like: 1. Land a second build system in-tree that can be itereated on an and contributed to. This will not be used just yet in terms of gating new commits to the repo. 2. Over time, bring the second build system to feature parity with the old build system, start setting up CI for both build systems. 3. At some point in the future, switch the default to the new build system, but keep the old one around. 4. At some further point in the future, delete the entire old build system. --- Alright, so with all that out of the way, here's some more info on this PR itself. The inital build system here is contained in the `src/bootstrap` directory and just adds the necessary minimum bits to bootstrap the compiler itself. There is currently no support for building documentation, running tests, or installing, but the implemented support is: * Compiling LLVM with `cmake` instead of `./configure` + `make`. The LLVM project is removing their autotools build system, so we'd have to make this transition eventually anyway. * Compiling compiler-rt with `cmake` as well (for the same rationale as above). * Adding `Cargo.toml` to map out the dependency graph to all crates, and also adding `build.rs` files where appropriate. For example `alloc_jemalloc` has a script to build jemalloc, `flate` has a script to build `miniz.c`, `std` will build `libbacktrace`, etc. * Orchestrating all the calls to `cargo` to build the standard distribution, following the normal bootstrapping process. This also tracks dependencies between steps to ensure cross-compilation targets happen as well. * Configuration is intended to eventually be done through a `config.toml` file, so support is implemented for this. The most likely vector of configuration for now, however, is likely through `config.mk` (what `./configure` emits), so the build system currently parses this information. There's still quite a few steps left to do, and I'll open up some follow-up issues (as well as a tracking issue) for this migration, but hopefully this is a great start to get going! This PR is currently tested on all the Windows/Linux/OSX triples for x86\_64 and x86, but more portability is always welcome! --- Future functionality left to implement * [ ] Re-verify that multi-host builds work * [ ] Verify android build works * [ ] Verify iOS build work (mostly compiler-rt) * [ ] Verify sha256 and ideally gpg of downloaded nightly compiler and nightly rustc * [ ] Implement testing -- this is a huge bullet point with lots of sub-bullets * [ ] Build and generate documentation (plus the various tools we have in-tree) * [ ] Move various src/etc scripts into Rust -- not sure how this interacts with `make` build system * [ ] Implement `make install` - like testing this is also quite massive * [x] Deduplicate version information with makefiles
- Loading branch information
