Collect binary size statistics

[alexcrichton]

Would be perhaps an interesting metric!

[Mark-Simulacrum]

We need a proposal as to what would be good to collect. Ideally, I think we'd have this statistic track the individual ".so" files we see in the unarchived directory. This would be a somewhat unique statistic since it wouldn't depend on the benchmarks directory, instead only looking at the sizes of the artifacts Rust ships.

I'd appreciate advice on whether we can get more fine-grained details for the size (e.g., debuginfo KB, code size KB, etc.) easily on Linux.

[alexcrichton]

Yeah I was thinking we wouldn't necessarily do this for everything if we couldn't clearly map to an artifact, and we should only do it for "final artifacts" for sure (tracking rlibs wouldn't be too useful). I'd imagine something like:

• Size of a "hello world" executable, broken down by:
  • allocator/jemalloc (may be to much of a reach)
  • debuginfo (before/after strip -g)
  • other rustc goop (size after strip -g)
• Size of a "hello world" cdylib, broken down by same as above except for jemalloc
• Maybe like a small Tokio example or regex example binary?

[fitzgen]

I started on a collection of rust+wasm code size micro benchmarks: https://github.com/fitzgen/code-size-benchmarks

Would be awesome to get this integrated here -- is this the kind of thing you were imagining @alexcrichton ?

[alexcrichton]

@fitzgen looks great! I'd probably throw in non-LTO builds as well for comparison, but otherwise that seems like a great starting point to me. It may even be possible to just set up a daily cron job running on Travis pushing the results somewhere?

[Mark-Simulacrum]

Should be feasible to integrate on a surface basis; I'll look into it sometime over the next couple weeks.

[Mark-Simulacrum]

Okay, so integration is primarily blocked on being able to install targets (i.e., rustup target add equivalent) for benchmarks. This is something we need to do anyway for winapi/windows benchmarks so I'll take a look at doing so soon.

[Walther]

Can I help somehow? I'd love to see binary size statistics get tracked (and ultimately, optimized 🙂)

[Mark-Simulacrum]

There are a couple things that need doing here:

• installing targets (not technically critical, since we can track x86_64 sizes, but highly desirable)
• getting a list of output artifacts to track for a given invocation of rustc (maybe we can query Cargo via --message-format=json?)

Order here doesn't matter too much, I think.

For the targets, the first step would be to get the sysroot module capable of installing targets other than the default std. We can't rely on the target existing (due to try builds) so these would have to be optional installations. I don't think this is terribly hard, but does require some legwork. rustup --verbose target add can provide an idea of URLs to use; you'll be requesting from the CI bucket, not static.rust-lang.org, but that shouldn't matter too much in theory.

For Cargo, we'll probably want to try to get a representative artifact for each crate -- likely this'll be a heuristic but I suspect that's the best we can do (so/wasm, rlib, binary). I believe Cargo will dump the output artifacts from rustc if run with --message-format=json so we would want to parse that and load the relevant data, then get file sizes of the artifacts and dump them as if perf stat gave them out, I think.