"memory allocation of .. bytes failed" appears when using `cargo run test` to run specific test name

[chentodd]

Problem

cargo run test shows "memory allocation of .. bytes failed" when I run specific test case:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.20s
     Running `target/debug/cargo test source_replacement`
memory allocation of 63063237072 bytes failed
Aborted (core dumped)

I expect there are no issues when I use cargo run test to run the tests.

Steps

1. Run rustup update to update needed tools to build cargo
2. Clone cargo repository
3. cd into cargo repository
4. Change to master branch
5. Run cargo check --workspace
6. Run cargo build
7. Run cargo run test source_replacement

here is the test video that contains step.5 to step.7 for the reference: 20241021_cargo_test_video.zip

Possible Solution(s)

No response

Notes

The error message in this issue is similar to #7543

Version

cargo 1.84.0 (edd36eb 2024-10-21)
release: 1.84.0
commit-hash: edd36eb
commit-date: 2024-10-21
host: x86_64-unknown-linux-gnu
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.5.0 (sys:0.4.74+curl-8.9.0 system ssl:OpenSSL/3.0.13)
os: Ubuntu 24.4.0 (noble) [64-bit]

additional hardware information of test machine:
Memory: 16 GB

[epage]

I wasn't able to reproduce. Unsure what the difference is.

[weihanglo]

The RAM size is critical.

additional hardware information of test machine:
Memory: 16 GB

I reproduced it on a machine with 4 GiB RAM.

(24-10-24 17:12:24) <0> [~/cargo]  
%  cargo -vV
cargo 1.82.0 (8f40fc59f 2024-08-21)
release: 1.82.0
commit-hash: 8f40fc59fb0c8df91c97405785197f3c630304ea
commit-date: 2024-08-21
host: x86_64-unknown-linux-gnu
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.9.0-DEV (sys:0.4.74+curl-8.9.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Amazon Linux AMI 2.0.0 [64-bit]

(24-10-24 17:12:30) <0> [~/cargo]  
%  free -h
              total        used        free      shared  buff/cache   available
Mem:           3.6G        520M        1.6G        1.7M        1.5G        2.8G
Swap:            0B          0B          0B

(24-10-24 17:12:35) <0> [~/cargo]  
%  cargo run -F all-static -- test -F all-static source_replacement
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.23s
     Running `target/debug/cargo test -F all-static source_replacement`
memory allocation of 63063237072 bytes failed
zsh: abort      cargo run -F all-static -- test -F all-static source_replacement

Testing other combinations:

• ✅ cargo test -F all-static source_replacement
• ✅ cargo build -F all-static source_replacement
• ✅ ./target/debug/cargo build -F all-static
• ✅ ./target/release/cargo build -F all-static
• ❌ ./target/debug/cargo test -F all-static
• ❌ ./target/release/cargo test -F all-static
• ✅ cp ./target/debug/cargo dargo && cargo clean && ./dargo test -F all-static
• ✅ cp ./target/release/cargo rargo && cargo clean && ./rargo test -F all-static
• ✅ cp ./target/debug/cargo dargo && ./dargo build -F all-static
• ✅ cp ./target/debug/cargo dargo && ./dargo build -F all-static
• ❌ cp ./target/debug/cargo dargo && ./dargo test -F all-static
• ❌ cp ./target/debug/cargo dargo && ./dargo test -F all-static

So it looks like the requirements are

• With existing build cache
• Using the newly built cargo binary
• Running cargo test

to trigger a core dump.

I may investigate it more when having time. At least we need to be sure that it is really an edge case for cargo itself.

[weihanglo]

Did a cargo t against rust-lang/crates.io with both debug/release built cargo from master. It works alright with 4GiB RAM (c5.large AWS EC2 instance). So, probably this is probably a Cargo-specific issue.

Labeled as low priority though feel free to help debug it!

[AlexTMjugador]

I got hit by this bug when running cargo check on this commit (cargo 1.84.0-nightly (cf53cc54b 2024-10-18), rustc 1.84.0-nightly (c1db4dc24 2024-10-25)). My box is also x86_64-unknown-linux-gnu, but it has 32 GiB of RAM, so throwing more RAM at the problem does not help either. I didn't have to set RUST_BACKTRACE to any value.

Edit: doing a cargo clean fixed it. Looks like doing a rustup update while having an older Rust version build cache is troublesome, is this a supported use case or does it happen to mostly work in practice in spite of it not being supported? For what it's worth, it's not the first time I do a rustup update with an already existing build cache.

[linyihai]

Edit: doing a cargo clean fixed it. Looks like doing a rustup update while having an older Rust version build cache is troublesome, is this a supported use case or does it happen to mostly work in practice in spite of it not being supported? For what it's worth, it's not the first time I do a rustup update with an already existing build cache.

I encountered this problem recently. And I did cargo clean and rustup update to workaround it.

[weihanglo]

so throwing more RAM at the problem does not help either

It does help if you have more than 63063237072 bytes available in your RAM 😆.
I do have a machine with 128 GiB RAM and can't reproduce this issue on it.

---

Since there is a non Cargo project affected, I removed the P-low label.