Repository navigation
Unstable fingerprints tracking issue #84970
Description
For users hitting this bug: please read this blog post which discusses what you can do. An excerpt is below:
The Internal Compiler Error asks you to report a bug, and if you can do so, we still want that information. We want to know about the cases that are failing.
But regardless of whether or not you file a bug, the problem here can be resolved by either:
- deleting your incremental compilation cache (e.g. by running
cargo cleanfor the whole project orcargo clean -p crate-causing-the-error), or - force incremental compilation to be disabled, by setting
CARGO_INCREMENTAL=0in your environment or settingbuild.incrementaltofalsein your$HOME/.cargo/config.tomlfile (which might also be called$HOME/.cargo/config) by adding this:
build.incremental = 'false'
We recommend that users of 1.52.0 disable incremental compilation, to avoid running into this problem.
We do not recommend that users of 1.52.0 downgrade to an earlier version of Rust in response to this problem. There is at least one instance of a silent miscompilation caused by incremental compilation that was not caught until we added the fingerprint checking.
I believe the (new) ICEs as a result of improved error detection in the incremental code will land in 1.53 onto stable, which is in 6 weeks. We are unlikely to fix all of the fallout (i.e. all bugs, at least 25 of which are currently open, though some are duplicates) by that time and be willing to backport the fixes. I suspect we are also unlikely to want to disable the new assertions; they are catching known unsoundness. I am opening this issue to discuss possible alternatives, which are hopefully more user friendly than the current ICE.
I think one direct improvement is to adjust the panic/assert on encountering this to be a proper compile error, which can tell the user that they should (for example) delete the incremental directory and invoke the compiler again. This is already a significant improvement over the current message, I suspect. We can still ask that they file a bug report.
We should also consider if it's worth considering applying performance-hurting mitigations; it seems definitely true we can drop incremental support on beta/stable until these bugs are fixed, but perhaps a smaller hammer is also viable.
Table of ICEs:
| Issue | Query name | Has MCVE | PR fixing issue | fixed in 1.5x? | PR adding regression test | Done |
|---|---|---|---|---|---|---|
| #83259 | predicates_of |
✅ | #84233 | 1.53 | #84233 | ✅ |
proc_macro_decls_static |
N/A | |||||
| #83126 | extern_mod_stmt_cnum |
mcve-83126 | #83153 | 1.52 | None yet | ❌ |
| #85197 | optimized_mir |
mcve-85197 | #85211 | 1.54 | #85211 | ✅ |
| #84225 | exported_symbols |
? | #84226 | 1.53 | None yet | ❌ |
| #85019 | item_children |
? | #83901 | 1.54 | None yet | ❌ |
| #83538 | evaluate_obligation (EvaluatedToOk and EvaluatedToOkModuloRegions) |
✅ | #85186 | 1.54 | #85186 | ✅ |
| #84963 | evaluate_obligation (OverflowError) |
✅ | None yet | ? | None yet | ❌ |
native_libraries |
? | #85702 | 1.54 | #85702 | ✅ | |
| #85360 | evaluate_obligation | ✅ | #85868 | 1.56 | #91065 | ✅ |
cc @Aaron1011 (driver of fixing the bugs, I believe)
cc @pnkfelix @wesleywiser (T-compiler leads)
cc @rust-lang/release for awareness
Activity
Mark-Simulacrum commentedon May 6, 2021
Nominating for T-compiler as I believe we should surface this during a meeting - likely for some small discussion, but mostly for awareness, and to see if we can get some help for possible ideas. I think this is a P-critical bug - not in the sense that we must have a fix for the underlying cause, but in the sense that this issue needs to be addressed in time for the next stable release.
I think it is likely worth applying the same fix to nightly as well, but perhaps in a channel-gated way if it has (for example) simply performance implications. This is a bit unclear.
Aaron1011 commentedon May 6, 2021
I think emitting a compiler error would be a great idea. However, I think it should still encourage users to open an issue (emphasizing that this is an internal compiler bug), so that we can see if any new issues pop up.
Aaron1011 commentedon May 6, 2021
OUTDATED: See the new table in #84970 (comment)
Here's the current state of the crashes involving different queries:
item_children- this is caused by not hashing attributes. PR Integrate attributes as part of the crate hash #83901 fixed thispredicate_of- this was fixed by Add TRACKED_NO_CRATE_HASH and use it for--remap-path-prefix#84233. However, we could still use a regression test before closing the issue Incremental compilation ICE withpredicates_of#84341evaluate_obligation(EvaluatedToOk/EvaluatedToOkModuloRegions) - this has an open PR UseEvaluatedToOkModuloRegionswhen we get a coinductive match #83913, but that PR has a large performance impact. I've been working on minimizing thesyncrash, which might yield more insight into how to avoid the performance problem.evaluate_obligation(OverflowError) - this is issue Incremental compilation ICE with from compiler-state-dependent overflow errors #84963. This is completely different from the otherevaluate_obligationcrash, and I haven't had a chance to investigate it yet. Hopefully, the fix will be much simpler.optimized_mir- this is issue ICE found unstable fingerprints for optimized_mir #84960 and thread 'rustc' panicked at 'found unstable fingerprints for optimized_mir #85064As far as I know, these are all of the currently existing crashes. If anyone sees a crash involving a query other than one of these, please add it to this list.
jyn514 commentedon May 6, 2021
lqd commentedon May 6, 2021
jonas-schievink commentedon May 6, 2021
I think this would actually most likely be fine – these ICE-causing bugs have existed for years, and it took very long for someone to run into a miscompilation.
Mark-Simulacrum commentedon May 6, 2021
Well, a known miscompilation - certainly the amount of ICEs we've seen suggests plenty of wrong results, right? Maybe in practice those don't cause problems, I don't know, but I'd rather give an error, especially if we can also e.g. poison the incremental cache automatically and make it just a matter of rerunning the compiler. In theory we could even teach Cargo to do that, though maybe that's going too far.
Mark-Simulacrum commentedon May 6, 2021
We discussed this during T-compiler meeting today, and decided on these actions:
109 remaining items
tengyifei commentedon Mar 6, 2024
Another example: #122116
skaphi commentedon Apr 8, 2025
Hitting this issue on a personal project, right after retroactivly deriving PartialEq and Eq on a bunch of structs and enums.
RalfJung commentedon Jun 11, 2025
I just got this when building the compiler itself after running
git pull(arriving at 1c04750) andcargo update -p rustc_apfloat:Kinrany commentedon Jul 21, 2025
One more.
cargo cleanfixed the issue on first attempt.rustc version
cargo version
the error
marcusbuffett commentedon Jul 21, 2025
tracel-ai/burn#3154 includes steps with a very reliable way to trigger this issue, for anyone that wants to take a crack at this
oknyshuk commentedon Aug 26, 2025
Can confirm, I'm using Burn + wgpu and consistently encounter this bug a couple times every day.
RalfJung commentedon Sep 13, 2025
There seems to be a fairly high concentration of incremental errors that mention
shallow_lint_levels_on. Not sure if that's helpful to track down the problem...carols10cents commentedon Sep 26, 2025
Ahhh my old friend, unstable fingerprints.
Just got this using rustc 1.90.0 (1159e78 2025-09-14) running on aarch64-apple-darwin building crates.io.
Details
Noratrieb commentedon Oct 22, 2025
I'm going to close this issue, not because the issues are all fixed, but because the tracking issue format is no longer useful for these problems (and newer compilers will no longer link to this issue #147830).
If you hit an error that links to here, we do want to hear about it, just in a new issue! We can't do much without a reproduction, but it's still useful to know that the errors are being hit even without one.
That said, to actually fix the problems (and to be clear, this error has many different causes that have to be fixed individually) we need a reproduction. This generally means source code + a diff, where compiling the source code first and then applying a diff will cause the bug. It doesn't have to be minimal (though that helps), any reliably reproduction will do. Using
RUSTFLAGS=-Zincremental-verify-ichtends to make reproductions more reliable.So if you hit this, please try your best to remember the edits you did and craft a reproduction from it. If you can't that's fine, but it's invaluable to have reproductions. And then open a new issue where we can look at it.
Update lang/rust to version 1.52.1.
Update lang/rust to version 1.52.1.