Get rid of wasm-gc

[pepyakin]

wasm-gc got deprecated but we still use it in our build pipeline.

The author mentions that:

The Rust compiler now natively supports --gc-sections when linking wasm executables, which means wasm executables already have 90% of their garbage removed when coming out of the compiler.

But for us this is not entirely true and the size of the runtime is:

• 882K baseline
• 636K with wasm-gc

And

The wasm-pack (and wasm-bindgen) project will already run this by default for you, so there's no need to run it again.

But we don't use neither of these.

We need to figure out what do we want to use.

[pepyakin]

According to twiggy diff node_runtime.wasm node_runtime.compact.wasm:

 Delta Bytes │ Item
─────────────┼─────────────────────────────────
     -252601 ┊ <total>
      -68707 ┊ custom section '.debug_str'
      -56447 ┊ custom section '.debug_line'
      -52165 ┊ custom section '.debug_info'
      -16040 ┊ custom section '.debug_ranges'
      -10261 ┊ custom section '.debug_pubnames'
       -6072 ┊ custom section '.debug_pubtypes'
       -1432 ┊ custom section '.debug_abbrev'
         -20 ┊ "function names" subsection
         -18 ┊ custom section '.debug_macinfo'
          -3 ┊ __wasm_call_ctors
          -1 ┊ elem[0]
          -1 ┊ func[357]

so major source of the cruft is debug_ sections.

There is a tool called wasm-strip in the wabt suite, which strips all custom sections.

882K baseline
643K after wasm-strip
636K after wasm-gc

however, if we decide to depend on wabt in our build pipeline we could then use wabt's wasm-opt, which could optimize binary (including pruning of dead symbols) and produce a binary with size of 561K.

[bkchr]

Why do we have debug sections in a release build? Shouldn't they be removed?

[pepyakin]

I'd say they should be removed! But for now this is not the case (and it has been this way for a while).
And I'm not sure why is this that way, but I asked why.

[pepyakin]

So, a quick recap:

• substrate uses a wasm file to encode logic of blockchain. We call it wasm runtime
• an example wasm runtime is written in rust, built with cargo. Rust uses LLVM LLD linker to link rust libraries and produce a final wasm blob
• by default, rustc puts a quite some junk into the final binary and that junk is unnecessary.
• to strip this unnecessary stuff, we use wasm-gc.

There is a chance that since the last time we checked the things have changed. Notably, there is a LLD flag -s which can be used to strip all unnecessary sections.

To check if that actually helps we need:

1. Disable wasm-gc. We can comment it on the line below just for the experiment.
   

   substrate/core/utils/wasm-builder/src/wasm_project.rs

   Lines 354 to 360 in 7010ec7

   	let res = Command::new("wasm-gc")
   	.arg(&wasm_file)
   	.arg(&wasm_compact_file)
   	.status()
   	.map(|s| s.success());

2. Try build the wasm runtime passing this flag to LLD and disable wasm-gc. This can be achieved by building substrate with the following command:

   WASM_BUILD_RUSTFLAGS='-C link-arg=-s' cargo build --release
   

You should be able to find the runtime files under target/release/wbuild directory. Keep in mind that if you skip wasm-gc then there won't be the .compact.wasm.

If the results turn out to be satisfying, we can then get rid of wasm-gc enitrely.

[kianenigma]

with wasm-gc

// compact 
→ pwd
/Users/kianenigma/Desktop/Parity/substrate/target/release/wbuild/node-runtime
 → du -sk node_runtime.compact.wasm 
1156	node_runtime.compact.wasm

// Original
→ pwd                      
.../substrate/target/release/wbuild/target/wasm32-unknown-unknown/release
→ du -sk node_runtime.wasm        
1480	node_runtime.wasm

---

without wasm-gc

// run with `let res: Result<bool, std::io::Error> = Ok(true);`
→ pwd
/substrate/target/release/wbuild/target/wasm32-unknown-unknown/release
→ du -sk node_runtime.wasm
1168	node_runtime.wasm

for which, after a cargo clean, I saved the compact file of the previous build, pasted it back here to silent this error

error: couldn't read /Users/kianenigma/Desktop/Parity/substrate/target/release/wbuild/node-runtime/node_runtime.compact.wasm: No such file or directory (os error 2)
 --> /Users/kianenigma/Desktop/Parity/substrate/target/release/build/node-runtime-de07cf4da2bc533b/out/wasm_binary.rs:2:36
  |
2 |                 pub const WASM_BINARY: &[u8] = include_bytes!("/Users/kianenigma/Desktop/Parity/substrate/target/release/wbuild/node-runtime/node_runtime.compact.wasm");
  |                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

and then ran the command with the linker flag. but I am not 100% sure if I have done it correctly. I pasted the cwd as well for someone to check.

[Swader]

Just a note that the test suite depends on wasm-gc and the prerequisites installation script doesn't pull it in. So to reduce new user friction, it should either be added to that one, or removed from the tests imo.

[bkchr]

Which script? The init script installs it.
Nevertheless, it is documented in the readme and having clear install instructions is always better than having any scripts that work on one distribution.

[Swader]

You're right, I've been relying too much on the getsubstrate.io script which does not pull it in, but the README does document it.

[bkchr]

@shawntabrizi ^^^