Linux memory usage regression

[alexheretic]

I noticed RLS can consume quite a lot of memory lately. Upstream: rust-lang/rust#56980

Idle memory usage

Memory usage running rls --cli. Reading taken after running once until built, quitting and re-running again and waiting for build complete.

Project	Linux nightly-2018-12-14	Linux stable 1.31
cargo init --bin	80 MiB	25 MiB
rust-random/rand	500 MiB	45 MiB
rust-lang/regex	1.5 GiB	81 MiB

So current stable gives a decent idea of where it should be. I tested beta channel out of interest and had results similar to current nightly.

Status

Tests use regex d4b9419 on Arch Linux

2019-03-10

Release	Idle memory usage
1.31.1	62 MB
1.32.0	1.6 GB
1.33.0	800 MB

Update 2019-07-04

Release	Idle memory usage
1.34.2	740 MB
1.35.0	1.3 GB
1.36.0	1.0 GB

Update 2019-08-16

Release	Idle memory usage
1.31.1	83 MB
1.37.0	2.4 GB

[aloucks]

Does stable include the removal of jemalloc?

Memory consumption on windows still seems to be reasonable:

80 MiB after building rust-lang/regex:

$ rustc --version
rustc 1.32.0-nightly (4a45578bc 2018-12-07)

$ rls --version
rls-preview 1.31.6 (bfa1371 2018-12-07)

[alexheretic]

Thanks for testing Windows, so perhaps it's not affected. I did try selecting system & jemalloc allocators but they both had similarly high usage for me on Linux.

[Xanewok]

I believe we build Rust with jemalloc (only libstd?), so the distribution artifacts (incl. RLS) should use jemalloc.

Since system allocator is being selected since recently, is it possible we use both allocators (system for locally-built RLS space and jemalloc for the dynamically linked rustc-land), leading to increased memory usage?

[alexheretic]

I bisected the memory regression based on nightly toolchain releases

Testing as above with the regex crate:

• rustup run nightly-2018-11-03 rls --cli -> 100 MiB
• nightly-2018-11-04, nightly-2018-11-05 rls is not available
• rustup run nightly-2018-11-06 rls --cli -> 1.5 GiB

[alexheretic]

I can also deduce this regression is caused upstream if I can reproduce it using the same rls source + different compilers.

For example using current rls code (173be77),

plus a small fix for older compilers

diff --git src/build/rustc.rs src/build/rustc.rs
index be3c52c..9ebc925 100644
--- src/build/rustc.rs
+++ src/build/rustc.rs
@@ -301,6 +301,7 @@ impl<'a> CompilerCalls<'a> for RlsRustcCalls {
                 edition: match state.session.edition() {
                     RustcEdition::Edition2015 => Edition::Edition2015,
                     RustcEdition::Edition2018 => Edition::Edition2018,
+                    _ => panic!(),
                 },
             };
             let files = fetch_input_files(state.session);

Again testing as above with the regex crate:

• rustup run nightly-2018-11-03 cargo run --manifest-path ../rls/Cargo.toml --no-default-features --release -- --cli -> 100 MiB
• rustup run nightly-2018-11-06 cargo run --manifest-path ../rls/Cargo.toml --no-default-features --release -- --cli -> 1.6 GiB

[alexheretic]

With 1.32 coming out I thought I'd retest rls memory usage to provide some more recent numbers.

Project	Linux stable 1.31 / nightly-2018-11-03	Linux stable 1.32
rust-lang/regex	80 MiB	1.4 GiB

So this regression has not been addressed and will affect the next stable release.

[mati865]

Since switch to system as default allocator tools don't use jemalloc anymore: rust-lang/rust#56980 (comment)

I can reproduce huge memory usage that doesn't go down even with RLS built on the very up to date Linux distro (latest available GLIBC, kernel, toolchain, etc).
This is pretty worrisome because it means there is leak somewhere or there is bug in memory handling. It was masked by jemalloc but that doesn't mean there was no issue.

At this point underlying issue should be identified and fixed.
I'll try adding similar hack as in rust-lang/rust#57287 but it shouldn't be considered as fix.

EDIT: Is there a way to reproduce large memory usage by manually running a rls command instead of using IDE? It's not easy to debug it when it's run from IDE.

[jens1o]

Yeah @mati865, you can run rls --cli in a workspace.

[mati865]

@jens1o well I was hoping for something like rls arg1 arg2 arg3 ... and boom 600 MiB memory consumed 😄

[alexheretic]

I've tried forcing use of jemalloc & adding the stuff in rust-lang/rust#57287 to rls main (though I don't understand it). These don't seem to help the memory usage regression though, or maybe I'm doing something wrong.

[mati865]

I don't have good news. After switching RLS to use jemmalloc it's peak memory usage increased a lot but later it drops to numbers similar to system allocator. Clippy and Cargo were still using system allocator during testing.

Valgrind and Heapstack didn't give any meaningful results. Maybe somebody more skilled could find out what's wrong, I'm out of ideas.

[programmerjake]

10GB used for me after initial build completed on https://salsa.debian.org/Kazan-team/kazan
rls-preview 1.33.0 (9f3c906 2019-01-20) installed via rustup
Linux x86_64

Note that building kazan builds LLVM as part of the process, so it can take a really long time and lots of memory.