-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Include Clang in llvm-tools #3847
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
# Summary | ||
[summary]: #summary | ||
|
||
Include a version of `clang` and `clang++` compiled against Rust LLVM in the `llvm-tools` component in nightly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is "on nightly" intended to be load bearing here?
In general this RFC I think needs to more clearly document the user expectations around breakage and whether we're ok making it common to reference this tool. Today, we mostly rely on slow moving distros I think to mitigate the less stable interfaces clang (and in general C / C++ compilers expose, to my knowledge), but if we're shipping it ourselves that pushes towards Rust being the one blamed for breakage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what the Rust way to manage stabilization for this is. The problem is that if it were in stable, breaking changes from clang interfaces might count as breaking changes that trickle into Rust.
I included the "in nightly" to limit the scope of this RFC to avoid having to deal with this question, as such an unlikely type of breakage wouldn't be a problem on nightly.
|
||
Builds of `clang` and `clang++` should be added to the `llvm-tools` component to enable version matching when working with base LLVM tools. | ||
|
||
# Drawbacks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we expect distros (e.g., Debian, Fedora) to ship a Rust clang if we start doing this? Or will Rust users only get this experience from a rustup install?
Do we expect that to mean that e.g. ecosystem crates are encouraged to not worry as much about C/C++ compiler version compatibility since many Rust users get very recent compilers through this mechanism?
I think they don't ship llvm tools today in general.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking about it as an alternate release pattern to avoid dealing with distro-specific variations; we would only allow installation via rustup, not by distro. We already have this infrastructure in place for the other LLVM tools.
Most crates could continue using system-installed compilers, as C has a stable ABI, and fine-tuning with LLVM isn't necessary for most applications.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the stability should be "we provide clang and it will have llvm of a matching version as rust has, that's it". Otherwise it will be perma-nightly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the stability should be "we provide clang and it will have llvm of a matching version as rust has, that's it". Otherwise it will be perma-nightly.
Sure, I don't think there's a reasonable way we can commit to something more stable than that. But I think it's worth having a section in the RFC on how we teach users that this is not subject to the normal relatively strict stability guarantees we have, and in particular e.g. -Werror and similar are probably a bad idea when compiling with this clang. Is that sufficiently discoverable? Does it help that it's in a separate component (perhaps we shouldn't repeat the mistake of skipping -preview in the name?)? Is there some warning we should embed on first use or in rustup add?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having it in nightly-only seems like it would be sufficient to avoid stability guarantees. I don't think there are many people opting into nightly-only features and expecting them to never break
Just dropping a link to the tracking issue for llvm tools: rust-lang/rust#85658 |
|
||
It may also drive more people to use manual compilation processes, which may cause fragmentation or be at odds with the Rust vision. | ||
|
||
# Rationale and alternatives |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It could be a separate component e.g. llvm-tool-clang
to keep the size of llvm-tools
down if you don't need it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be best to keep it in llvm-tools
to avoid increasing the number of components. I.e. there shouldn't be a separate component for each llvm tool.
On the other hand, clang
and clang++
are bigger than the other tools, which might warrant another component
I mention this in the "Unresolved Questions" section because I'm not sure which choice is better.
# Summary | ||
[summary]: #summary | ||
|
||
Include a version of `clang` and `clang++` compiled against Rust LLVM in the `llvm-tools` component in nightly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question: does this mean we would need to be building clang
and clang++
in our dist builders?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not entirely sure, but it would probably depend on if the dist builder includes the llvm-tools
component. If so, then yes, it would require building clang
and clang++
# Summary | ||
[summary]: #summary | ||
|
||
Include a version of `clang` and `clang++` compiled against Rust LLVM in the `llvm-tools` component in nightly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discussion: hm, this means that some maintenance bandwidth would need to be used for keeping clang
and clang++
building, or try to fix it, even if it's a nightly-only dist component right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but it shouldn't be much, because the other llvm tools are already being built.
Allowing user-access to the LLVM pipeline allows for many user-built features, such as cross-language inlining. However, LLVM version mismatching between tools can lead to frustrating problems. Including `clang` and `clang++` in `llvm-tools` allows users to use only the tools that Rust ships with, ensuring consistent versioning. | ||
|
||
In future versions of Rust, including a compiler with Rustup could also improve ergonomics for FFI crates, as it could avoid depending on system compilers. See how [Zig's implementation](https://actually.fyi/posts/zig-makes-rust-cross-compilation-just-work/) led to easy cross-compiles in Rust to Macos. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see any explicit mention of sysroots, and mentioning Zig-like capabilities as future work suggests it's not in scope for this RFC. In other words, the propsal for now is to ship only the compiler binary, not all the other files needed to actually compile C and C++ code, and rely on those to already be installed by other means?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not very familiar with cross-compiling, which is why I listed it as future work.
My understanding of the compiler binary is that it depends on the other llvm tools, and wouldn't introduce new dependencies for users, at least when not cross-compiling.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even without the complications of cross-compiling, the compiler binary and other tools aren't enough to usefully compile C and C++ code. At minimum you headers and static/dynamic libraries for the C standard library and other runtime facilities that are expected on that platform (e.g., on Linux with glibc that's libc and crt0 for C programs, often also libm and pthreads and probably more I'm forgetting). People would still need to have those installed to get any use out of the clang, lld, etc. binaries.
In practice they'll probably have the libraries installed anyway because on most host platforms rustc requires that a basic C toolchain is installed for linking Rust programs (e.g., on Unix platforms cc
is used for linking, on windows-msvc you need to have some MSVC thing installed).
|
||
This will increase compile times and require more storage on devices with the `llvm-tools` component installed. | ||
|
||
It may also drive more people to use manual compilation processes, which may cause fragmentation or be at odds with the Rust vision. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does this mean?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Compile times and storage requirements would be increased for the llvm-tools
component because clang
and clang++
would need to be compiled and stored.
Manual compilation processes means compiling with tools other than just cargo
- development in this area could make users write build scripts using the new clang
, rather than using an existing crate like cc
.
It could be at odds with the Rust vision, because Rust wasn't intended to come with a C/C++ compiler as an optional component, and it would mean including a compiler that isn't part of cargo
.
Summary
Include a version of
clang
andclang++
compiled against Rust LLVM in thellvm-tools
component in nightly.Motivation
Allowing user-access to the LLVM pipeline allows for many user-built features, such as cross-language inlining. However, LLVM version mismatching between tools can lead to frustrating problems. Including
clang
andclang++
inllvm-tools
allows users to use only the tools that Rust ships with, ensuring consistent versioning.In future versions of Rust, including a compiler with Rustup could also improve ergonomics for FFI crates, as it could avoid depending on system compilers. See how Zig's implementation led to easy cross-compiles in rust to Macos.
Background
clang
andclang++
are LLVM-based C and C++ compilers mentioned in official documentation:Unfortunately, this example does not always work, because it calls system
clang
, which may use a different version of LLVM than Rust. Additionally, even at the same version, there is a potential for problems from mixing base LLVM tools with the Rust fork of LLVM.Rustup has the ability to install a component called
llvm-tools
, which exposes the llvm tools used by Rust, includingllvm-link
andllc
- notably, it does not contain a build ofclang
orclang++
.Conclusion
Builds of
clang
andclang++
should be added to thellvm-tools
component to enable version matching when working with base LLVM tools.Drawbacks
This will increase compile times and require more storage on devices with the
llvm-tools
component installed.It may also drive more people to use manual compilation processes, which may cause fragmentation or be at odds with the Rust vision.
Rationale and alternatives
Users can opt for system
clang
andclang++
when building projects with LLVM, however there is no guarantee that users will have an appropriate version of the system tools, or that the Rust fork of LLVM won't contain any breaking changes.Prior art
This may help in the goal Expose experimental LLVM features for GPU offloading, as raw LLVM access is particularly useful for GPU compilation libraries.
This was mentioned in Shipping clang as a Rustup component
See also the issues for
llvm-dis
,llc
andopt
Unresolved questions
Should
clang
andclang++
be part of thellvm-tools
component or added as their own component?Rendered