Skip to content

Compiler takes 30 minutes + large (32GB+) amounts of ram to compile code with large amount of arguments + function-within-function #122944

Open

Description

I do not have a simple reproducible example since this bug goes over my head, perhaps someone else can find the bug but it seems pretty bad:

This file takes over 30 minutes of time to compile in rustc 1.77.0 (rustc 1.77.0 (aedd173 2024-03-17)) and rustc nightly (rustc 1.78.0-nightly (2bf78d1 2024-02-18)): https://github.com/Anti-Raid/splashtail/blob/74a33ddebe045d2a9fb4acc2d65ef97f75314e84/botv2/src/modules/moderation/cmd.rs [this specific commit triggered the issue of 30 minutes though this was always exponentially slow from the very start]

Don't know why it is, but this seemingly innocent file takes up a full 25-30 minutes of compile time and hogs up over 32GB of ram to the point that I needed to add 128GB of swap in order for Linux's Out Of Memory killer to not SIGKILL rust. In CI, this just leads to a SIGKILL quickly within 20 minutes of compiling. Actual compilation seems to go through without much of an error. Removing moderation from the module tree reduces compile times from 30 minutes to 5-6 minutes and RAM usage returns to normal

I expected to see this happen: Compile times within 5-6 minutes (which is normal right now, don't know if the 5 minute time comes from the same root cause as this bug though but it could be) and normal RAM usage (<32GB ram)

Instead, this happened:

warning: struct ConfigOption is never constructed
--> src/silverpelt/config_opt.rs:159:12
|
159 | pub struct ConfigOption {
| ^^^^^^^^^^^^
|
= note: #[warn(dead_code)] on by default

warning: botv2 (bin "botv2") generated 1 warning
Finished release [optimized] target(s) in 30m 27s

[Ignore the warning, thats unrelated]

Linux kernel kills rust with a SIGKILL 9 randomly and Out Of Memory killer activates

Meta

Tested and reproduced across both Apple M1 and a Dedicated Server (Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz) on both rust stable and nightly using both LLVM and Cranelift as codegen backends. Switching linkers to clang+mold/ld.lld from defaults did not help. Rust versions are given below:

rustc 1.77.0 (aedd173 2024-03-17) and rustc 1.78.0-nightly (2bf78d1 2024-02-18)

No backtrace is available, the compiler does not crash, attempts to compile the code using --timing for timing information don't seem to even complete in any reasonable timespan. Issue occurs on both debug and release builds so this is not an issue of release build optimizations. Typical options to speed up compile times such as opt-level = 1 do not work in reducing neither the compile times nor the memory usage. Splitting up the code into multiple crates does not work as long as this file is in the module tree and hence compiled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    A-macrosArea: All kinds of macros (custom derive, macro_rules!, proc macros, ..)C-bugCategory: This is a bug.E-needs-mcveCall for participation: This issue has a repro, but needs a Minimal Complete and Verifiable ExampleI-compilememIssue: Problems and improvements with respect to memory usage during compilation.I-slowIssue: Problems and improvements with respect to performance of generated code.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions