OOM installing rustc-dev with 1GB of memory

[bjorn3]

Problem

I tried installing a rust toolchain in a VM I assigned 1GB of ram. (100MB used by the minimal debian install) When installing rustc-dev it got an OOM however.

Steps

1. Create a VM
2. Assign 1GB of ram
3. Install a minimal debian system in the VM. When running it only uses 100MB of ram.
4. Attempt to install a rust toolchain with the rustc-dev component in the VM.
5. OOM

Possible Solution(s)

I thought rustup 1.24 enabled installation on systems with little ram. Is this a regression? Would it be possible to stream the component rather than attempt to load it all at once in memory?

Notes

For now I will be increasing the assigned amount of memory, but IMO it should be possible to install on systems with this little ram, hence this issue.

Rustup version

rustup 1.25.1 (bb60b1e89 2022-07-12)

Installed toolchains

Default host: x86_64-unknown-linux-gnu
rustup home:  /home/bjorn/.rustup

no active toolchain

I attempted to install nightly-2022-12-13-x86_64-unknown-linux-gnu.

[bjorn3]

Assigned 2GB and now rustup maxed at 1.55GB.

[kinnison]

Sounds like the general overheads on that platform are higher than we expect. If you reduce RUSTUP_UNPACK_RAM does that help? You can set it to something tiny (e.g. 10 bytes) to get rustup to tell you the minimum it thinks it needs for your system. You can also set it to something huge (10s of gigabytes) to see if Rustup is mis-detecting the RAM available in your system.

Currently we estimate 200MB for rustup, IO logic, etc. and then we use detected max ram (https://docs.rs/effective-limits/latest/effective_limits/fn.memory_limit.html) to limit our buffer use to leave at least that estimate free.

[guest271314]

@kinnison I am running into this while running a live OS using only RAM. Starting with 1GB disk space installing Rust using https://sh.rustup.rs with minimal profile then trying to run cargo run in a project, e.g. https://github.com/denoland/roll-your-own-javascript-runtime, winds up using all available disk space. Poking around this comment claims Rust installing uses 1.6GB. Any solution for this?

[guest271314]

https://itsfoss.com/install-rust-cargo-ubuntu-linux/?ht-comment-id=8619618

Hi, When I install rust like this in my container: - Using apt just provides a too old cargo version - Using curl just grow my home or docker image by 1.6GB (which is not acceptable neither). How can I have a decently small cargo installation ?

[bjorn3]

This issue is about RAM not disk usage. A standard rust toolchain without extra components installed easily fits in 1GB. If you use crates from crates.io you might surpass it though.

[guest271314]

A standard rust toolchain without extra components installed easily fits in 1GB.

I am running a live OS using only RAM. It would be very helpful to publish the steps needed to be taken to install Rust using less than 1GB.

[guest271314]

@bjorn3 The comment I posted from a random report in the wild is not a one-off. From 2019 https://unix.stackexchange.com/posts/551260:

Since you’re using Debian, you could use the Rust packages instead:

sudo apt install cargo

This will reduce the requirements significantly compared to the 1.2GiB taken by .rustup.

If you know the steps necessary for a minimal Rust installation - where disk space and/or RAM are limited - kindly share or direct me to where I can try those steps.

[bjorn3]

During installation you get twice the disk space usage compared to the final size (the downloaded tarballs + the unpacked tarballs) plus a third time for ram usage. If all of these have to fit in ram due to using a live system, exceeding 1GB is guaranteed. Rustup doesn't support streaming directly from the network to an unpacked installation. It always downloads everything before unpacking and takes care to allow rolling back changes if something goes wrong.

[guest271314]

If all of these have to fit in ram due to using a live system, exceeding 1GB is guaranteed.

So it is not possible to install Rust with less than 1GB of disk space, correct?

[bjorn3]

You might slightly exceed it if you use a real disk (maybe using the minimal profile works to fit in 1GB), but if the filesystem is stored in ram, you definitively need more than 1GB of ram to fit both the filesystem and the rustup installer process.

[rbtcollins]

I'm marking the side-discussion off-topic. Please make a new bug when your situation is not exactly the one a related bug seems to have.

[rbtcollins]

@bjorn3 ok, so 1GB does not work now. Have you tried @kinnison 's suggestions - in particular setting RUSTUP_UNPACK_RAM very low?

I don't recall any changes to the memory buffer management stuff recently, so not anticipating regressions. Note that the cached component dist files are not memory regulated - but we do only only one in memory cache at a time, and its pull-parsed, so I wouldn't expect a huge footprint.

Perhaps running under memgrind or massif or some such would help too.

[saethlin]

I am currently trying to install rustup on a t4g.nano instance running Ubuntu 22.04 and the unpacking is running out of memory.

info: profile set to 'minimal'
info: setting default host triple to aarch64-unknown-linux-gnu
info: syncing channel updates for 'stable-aarch64-unknown-linux-gnu'
info: latest update on 2023-10-05, rust version 1.73.0 (cc66ad468 2023-10-03)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
 76.5 MiB /  76.5 MiB (100 %)  72.4 MiB/s in  1s ETA:  0s
info: installing component 'cargo'
warning: Ignoring RUSTUP_UNPACK_RAM (10) less than minimum of 33554432.
info: installing component 'rust-std'
warning: Ignoring RUSTUP_UNPACK_RAM (10) less than minimum of 33554432.
Killed

free -h looks like this:

               total        used        free      shared  buff/cache   available
Mem:           419Mi       126Mi       252Mi       0.0Ki        40Mi       274Mi
Swap:             0B          0B          0B