fail to allocate memory when run `rustup update`

[wongsyrone]

Problem
Fail to allocate memory when I run rustup update on Windows 10 x64 using cmd.exe

C:\Users\home>rustup update
info: syncing channel updates for 'stable-x86_64-pc-windows-msvc'
info: syncing channel updates for 'nightly-x86_64-pc-windows-gnu'
info: latest update on 2021-05-07, rust version 1.54.0-nightly (676ee1472 2021-05-06)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
 16.2 MiB /  16.2 MiB (100 %)   8.0 MiB/s in  2s ETA:  0s
info: downloading component 'rust-mingw'
info: downloading component 'rust-std'
 23.6 MiB /  23.6 MiB (100 %)   8.0 MiB/s in  2s ETA:  0s
info: downloading component 'rustc'
 79.8 MiB /  79.8 MiB (100 %)  14.0 MiB/s in  7s ETA:  0s
info: downloading component 'rustfmt'
info: removing previous version of component 'cargo'
info: removing previous version of component 'clippy'
info: removing previous version of component 'rust-docs'
info: removing previous version of component 'rust-mingw'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'rustc'
warning: during uninstall component rustc was not found
info: removing previous version of component 'rustfmt'
warning: during uninstall component rustfmt was not found
info: installing component 'cargo'
info: using up to 500.0 MiB of RAM to unpack components
info: installing component 'clippy'
info: installing component 'rust-docs'
 16.2 MiB /  16.2 MiB (100 %)   3.1 MiB/s in  7s ETA:  0s
info: installing component 'rust-mingw'
info: installing component 'rust-std'
 23.6 MiB /  23.6 MiB (100 %)  16.1 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 79.8 MiB /  79.8 MiB (100 %)  14.4 MiB/s in 19s ETA:  0s
info: installing component 'rustfmt'
info: syncing channel updates for 'nightly-x86_64-pc-windows-msvc'
info: latest update on 2021-05-07, rust version 1.54.0-nightly (676ee1472 2021-05-06)
info: downloading component 'rust-src'
info: downloading component 'rust-analysis'
info: downloading component 'rust-analyzer-preview'
info: downloading component 'rustfmt'
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
 16.2 MiB /  16.2 MiB (100 %)  14.4 MiB/s in  2s ETA:  0s
info: downloading component 'rust-std'
 22.9 MiB /  22.9 MiB (100 %)  13.6 MiB/s in  2s ETA:  0s
info: downloading component 'rustc'
 59.6 MiB /  59.6 MiB (100 %)   9.6 MiB/s in  8s ETA:  0s
info: removing previous version of component 'rust-src'
info: removing previous version of component 'rust-analysis'
info: removing previous version of component 'rust-analyzer-preview'
info: removing previous version of component 'rustfmt'
info: removing previous version of component 'cargo'
info: removing previous version of component 'clippy'
info: removing previous version of component 'rust-docs'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'rustc'
info: installing component 'rust-src'
info: installing component 'rust-analysis'
info: installing component 'rust-analyzer-preview'
info: installing component 'rustfmt'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-docs'
 16.2 MiB /  16.2 MiB (100 %)   3.0 MiB/s in  7s ETA:  0s
info: installing component 'rust-std'
 22.9 MiB /  22.9 MiB (100 %)   8.0 MiB/s in 11s ETA:  0s
info: installing component 'rustc'
memory allocation of 16777216 bytes failed

C:\Users\home>

Steps

1. run rustup update

Possible Solution(s)

setting the environment variable RUSTUP_UNPACK_RAM to 21474836480 does NOT solve the problem

Notes

Output of rustup --version:

C:\Users\home>rustup --version
rustup 1.24.1 (a01bd6b0d 2021-04-27)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.52.0 (88f19c6da 2021-05-03)`

Output of rustup show:

C:\Users\home>rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  D:\.rustup_home

installed toolchains
--------------------

stable-x86_64-pc-windows-msvc (default)
nightly-x86_64-pc-windows-gnu
nightly-x86_64-pc-windows-msvc

installed targets for active toolchain
--------------------------------------

x86_64-pc-windows-gnu
x86_64-pc-windows-msvc

active toolchain
----------------

stable-x86_64-pc-windows-msvc (default)
rustc 1.52.0 (88f19c6da 2021-05-03)

[wongsyrone]

setting the environment variable RUSTUP_UNPACK_RAM to 21474836480 does NOT solve the problem.

How can I get my toolchain back as I cannot install it anymore?

[wongsyrone]

I changed the value, it looks like working now.

C:\Users\home>set | findstr /I rustup
RUSTUP_HOME=D:\.rustup_home
RUSTUP_UNPACK_RAM=20000000

[kinnison]

We think there's likely a problem with memory fragmentation on Windows' allocator. We are working on a fix to this in #2756 and I hope we will resolve this fairly soon.

[gruber123]

In the nighltly toolchain It fails on rustc unpack allocation cannot allocate 16 mb of memory [crash],

[kinnison]

It fails on rustc unpack allocation cannot allocate 16 mb of memory [crash],

@gruber123 what platform are you using?

[gruber123]

It fails on rustc unpack allocation cannot allocate 16 mb of memory [crash],

@gruber123 what platform are you using?

windows 7 x64

[kinnison]

@gruber123 Cool, could you download and install https://github.com/rust-lang/rustup/suites/2685811520/artifacts/59213037 (you must be logged into github to download it) and then try again to do the install of nightly? (You may need to remove the incomplete/broken version first)

[gruber123]

Thanks, I've already installed it after closing some programs, but failing message for next updates is frustrating.
With first nigthly release after stable release (v1.52) was appeared I've tried to update nightly release (before updating to stable) and failed all the time no matter how many or less programs (memory) are opened (same memory fail on rustc), after some failure I've made full update rustup update and this time it works, today rustup update nightly same story.

[kinnison]

Can you please confirm - did you install the rustup-init which I linked to above? If so, could you please run rustup dump-testament and paste the output of that into a comment here; and also run rustup toolchain remove nightly then rustup toolchain install nightly --no-self-update and confirm that either (a) it works or (b) it fails in which case also paste the log of doing that.

[rbtcollins]

@gruber123 the custom build of rustup that @kinnison has linked may fix the problem for you, but you'll need to follow these steps:

1. download the URL he linked, which is a zip file.
2. inside the zip is a rustup-init.exe
3. run that rustup-init.exe; it will re-install rustup on top of your exinsting install
4. try uninstalling and re-installing your toolchain. Please add the option --no-self-update when doing that, as otherwise rustup will download the current released version and re-introduce the problem.

[gruber123]

C:\Users\User>rustup dump-testament
Rustup version renders as: 1.24.1+433 (5360c02 2021-05-08)
Current crate version: 1.24.1
Built from branch: remotes/pull/2756/merge
Commit info: 1.24.1+433 (5360c02 2021-05-08)
Working tree is clean

C:\Users\User>rustup toolchain remove nightly
info: uninstalling toolchain 'nightly-x86_64-pc-windows-msvc'
info: toolchain 'nightly-x86_64-pc-windows-msvc' uninstalled

C:\Users\User>rustup toolchain install nightly --no-self-update
info: syncing channel updates for 'nightly-x86_64-pc-windows-msvc'
info: latest update on 2021-05-09, rust version 1.54.0-nightly (881c1ac40 2021-05-08)
info: downloading component 'cargo'
3.6 MiB / 3.6 MiB (100 %) 2.8 MiB/s in 1s ETA: 0s
info: downloading component 'clippy'
info: downloading component 'rust-docs'
16.6 MiB / 16.6 MiB (100 %) 2.0 MiB/s in 8s ETA: 0s
info: downloading component 'rust-std'
19.3 MiB / 19.3 MiB (100 %) 2.2 MiB/s in 7s ETA: 0s
info: downloading component 'rustc'
59.6 MiB / 59.6 MiB (100 %) 2.8 MiB/s in 27s ETA: 0s
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-docs'
16.6 MiB / 16.6 MiB (100 %) 1.3 MiB/s in 1m 0s ETA: 0s
info: installing component 'rust-std'
19.3 MiB / 19.3 MiB (100 %) 9.2 MiB/s in 1m 46s ETA: 0s
info: installing component 'rustc'
59.6 MiB / 59.6 MiB (100 %) 9.3 MiB/s in 6s ETA: 0s
info: installing component 'rustfmt'

nightly-x86_64-pc-windows-msvc installed - rustc 1.54.0-nightly (881c1ac40 2021-05-08)

C:\Users\User>rustc --version
rustc 1.54.0-nightly (881c1ac40 2021-05-08)

[gruber123]

Ok, it worked without closing extra programs

So what is custom build of rustup-init, how it's different from official version on main page?
Also on failure cases temp directory not cleaned.

[kinnison]

The version we asked you to try is a work-in-progress to fix the issue on the main version. I will look to merge that tomorrow and then we can try and prepare a new beta release. I'll keep you updated when that's done and I'll ask you to install and try the beta release then. I'll provide instructions at that point.