Improve incrementality of jsoo build

[hhugo]

Instead of compiling *.cma files to JavaScript. We can compile individual *.cmos to javascript and link all *.cmo.js together.
This results in a much better incremental build as jsoo no longer recompile compilation units that were not modified.

Linking *.cmo.js into a cma.js should be much faster than any compilation with jsoo.

This can only be done for local libraries as "installed" ones are not required to provide individual cmo. This is fine.

This is following https://discuss.ocaml.org/t/replace-jsoo-link-with-cat-or-esbuild/11600/13?u=hhugo

[rgrinberg]

Does this affect dead code elimination?

[hhugo]

It should not affect dead code elimination.

[hhugo]

This PR requires a new release of jsoo so that js_of_ocaml link can understand the -a flag. We'll need to add some check to know if the flag is accepted.

[rgrinberg]

I see. I think I discussed doing something like this with @nojb as well to speed up native/bytecode builds and dead code elimination was the sour point as to why this didn't work.

[hhugo]

I see. I think I discussed doing something like this with @nojb as well to speed up native/bytecode builds and dead code elimination was the sour point as to why this didn't work.

I don't understand.. this PR is about making jsoo compilation similar to ocamlc. Maybe you're thinking about another optimization .. ?

[rgrinberg]

Oh I see. I didn't read the PR implementation. I thought you meant producing the final .js from the .cmo.js files while completely side-stepping .cma. I see that you're just speeding up the production of your .cma.js here.

[hhugo]

I see. If you wanted to not produce cma files and pass cmos directly, you could implement the "dead code" logic in dune directly. It's not very difficult (jsoo does it for its link command)

[hhugo]

I see. If you wanted to not produce cma files and pass cmos directly, you could implement the "dead code" logic in dune directly. It's not very difficult (jsoo does it for its link command)

It's actually more complicated for dune as you need to be able to deal with arbitrary compiler version.

[rgrinberg]

It's actually more complicated for dune as you need to be able to deal with arbitrary compiler version.

How come? Wouldn't this knowledge come from ocamlep?

[hhugo]

It's actually more complicated for dune as you need to be able to deal with arbitrary compiler version.

How come? Wouldn't this knowledge come from ocamlep?

I don't think you want to base the logic on ocamldep output. You should let ocamlc decide if a module depend on the implementation of another one. You probably want to just use reloc info inside cmo file format and replicate the small logic for what unit should be linked

[hhugo]

@rgrinberg, it'd be better if we could construct rules dynamically based on path (like done for ppxes executables) instead of relying on a restricted list of variants (e.g. Jsoo_rules.Config.all). I think you told me in the past that this feature was planned for later. Was there any progress made on that front ?

[hhugo]

ping @rgrinberg

[rgrinberg]

The feature still hasn't yet made it in. But you could copy the trick that we use for ppx.

[hhugo]

@vouillon, I've rebased this PR on top of the wasmoo support but I don't know if this workflow is supported. Can you take look ?

[vouillon]

@hhugo Yes, this should work. Could you retry the failing check? I have recreated the wasm_of_ocaml branch used by this workflow.