Compilation of a crate using a large static map fails on latest i686-pc-windows-gnu Beta

[urschrei]

I'm trying to build a cdylib (https://github.com/urschrei/lonlat_bng) which requires the https://github.com/urschrei/ostn15_phf crate.

building on AppVeyor, on i686-pc-windows-gnu, using the latest beta, is failing with an OOM error:

Details

The ostn15_phf crate is essentially just a very big static map, built using PHF (the generated, uncompiled map is around 42.9mb)

The build passed when running cargo test, using rustc 1.12.0-beta.3 (341bfe43c 2016-09-16):
https://ci.appveyor.com/project/urschrei/lonlat-bng/build/105/job/3y1llt6luqs3phs3

It's now failing when running cargo test, using rustc 1.12.0-beta.6 (d3eb33ef8 2016-09-23):
https://ci.appveyor.com/project/urschrei/lonlat-bng/build/job/27pgrkx2cnn2gw50

The failure occurs when compiling ostn15_phf with fatal runtime error: out of memory

[sfackler]

cc @brson just want to make sure you're aware of this.

[nikomatsakis]

Related to #36926 perhaps? ("1.12.0: High memory usage when linking in release mode with debug info")

[nikomatsakis]

@urschrei have you tried this on platforms other than windows, out of curiosity?

[nikomatsakis]

I see. It seems to work on x86_64-pc-windows-gnu, but not i686.

[urschrei]

I haven't tried to build i686 on Linux or OSX, but I easily could…

[nikomatsakis]

Well, I just did a run on my linux box. The memory usage is certainly through the roof: https://gist.github.com/nikomatsakis/ea771dd69f12ebc5d3d5848fa59fb43a

This is using nightly (rustc 1.13.0-nightly (a059cb2f3 2016-09-27)). The peak is around 4GB.

[nikomatsakis]

So @alexcrichton has executed a massif run: https://gist.github.com/alexcrichton/d20d685dd7475b1801a2ccac6ba15b08

The peak result Measurement 58 shows something like this:

Percentage	Area
36%	HIR
9%	MIR
14%	type/region arenas
5%	region maps
2%	constants

The peak result (measurement 48) looks pretty similar. More memory used by MIR:

Percentage	Area
26%	HIR
14%	MIR
5%	type/region arenas
4%	region maps
3%	constants

These numbers are just based on a kind of quick scan of the gist.

[nikomatsakis]

It seems clear we need to revisit our handling of statics and constants in a big way. But then this has been clear for a long time. =) I'm wondering if we can find some kind of "quick fix" here.

I also haven't compared with historical records -- but most of that memory we see above, we would have been allocating before too, so I'm guessing this is a case of being pushed just over the threshold on i686, versus a sudden spike.