Enable LTO, EVAL_CTOR

[WeebDataHoarder]

These changes enable LTO for the build process, and decreases the number of files (data / mem files). While this increases total size slightly, it reduces the number of files fetched from network overall. This is further lowered down in a future PR where the fallback font is extracted from the binaries.

The output of the dist folder now looks like this:

-rw-r--r-- 1 root root  46K jun 22 10:22 COPYRIGHT
-rw-r--r-- 1 root root  74K jun 22 10:22 subtitles-octopus.js
-rw-r--r-- 1 root root 584K jun 22 10:16 subtitles-octopus-worker.js
-rw-r--r-- 1 root root 4,7M jun 22 10:17 subtitles-octopus-worker-legacy.js
-rwxr-xr-x 1 root root 2,2M jun 22 10:16 subtitles-octopus-worker.wasm

EDIT: changes regarding number of files are being moved to #146

[TheOneric]

The perf optimisations LTO and EVAL_CTOR seem good to me, but I have question about the other changes (and suggest they a asplit into a seperate commit).

embed and --memory-init-file=1 are both to reduce the number of files iiuc. However they also increase the total size. And in particular for embed, emscripten seems to discourage its use:

Embedding puts the specified files inside the generated JavaScript, while preloading packages the files separately. Embedding files is much less efficient than preloading and should only be used when packaging small numbers of small files. Preloading also enables the option to separately host the data.

What benefit does changing both files to embed bring, especially when #146 removes the embed for the font again.

What is the intent behind -s AUTO_NATIVE_LIBRARIES=0? It doesn't seem to change the total output size in any way.

[WeebDataHoarder]

Regarding --memory-init-file=1, the idea was to split the changes across several commits, even while #146 was going to move the font outside. It does not make sense to have this with the font embedded inside, correct. Maybe moving this to be part of #146 directly makes more sense, as otherwise an intermediate step to extract the font would be necessary.
With the default font outside of the library (after #146 comes) users can replace it directly or not load it at all as well.

After #146, the only file embedded will be fonts.conf which currently weights 803 bytes, where the embedded result (base64 plus decoding code) comes at 1181 bytes (see next line).

var fileData0 = "PD94bWwgdmVyc2lvbj0iMS4wIj8+CjwhRE9DVFlQRSBmb250Y29uZmlnIFNZU1RFTSAiZm9udHMuZHRkIj4KPGZvbnRjb25maWc+Cgk8ZGlyPi9mb250czwvZGlyPgoJPG1hdGNoIHRhcmdldD0icGF0dGVybiI+CgkJPHRlc3QgcXVhbD0iYW55IiBuYW1lPSJmYW1pbHkiPgoJCQk8c3RyaW5nPm1vbm88L3N0cmluZz4KCQk8L3Rlc3Q+CgkJPGVkaXQgbmFtZT0iZmFtaWx5IiBtb2RlPSJhc3NpZ24iIGJpbmRpbmc9InNhbWUiPgoJCQk8c3RyaW5nPm1vbm9zcGFjZTwvc3RyaW5nPgoJCTwvZWRpdD4KCTwvbWF0Y2g+Cgk8bWF0Y2ggdGFyZ2V0PSJwYXR0ZXJuIj4KCQk8dGVzdCBxdWFsPSJhbnkiIG5hbWU9ImZhbWlseSI+CgkJCTxzdHJpbmc+c2FucyBzZXJpZjwvc3RyaW5nPgoJCTwvdGVzdD4KCQk8ZWRpdCBuYW1lPSJmYW1pbHkiIG1vZGU9ImFzc2lnbiIgYmluZGluZz0ic2FtZSI+CgkJCTxzdHJpbmc+c2Fucy1zZXJpZjwvc3RyaW5nPgoJCTwvZWRpdD4KCTwvbWF0Y2g+Cgk8bWF0Y2ggdGFyZ2V0PSJwYXR0ZXJuIj4KCQk8dGVzdCBxdWFsPSJhbnkiIG5hbWU9ImZhbWlseSI+CgkJCTxzdHJpbmc+c2Fuczwvc3RyaW5nPgoJCTwvdGVzdD4KCQk8ZWRpdCBuYW1lPSJmYW1pbHkiIG1vZGU9ImFzc2lnbiIgYmluZGluZz0ic2FtZSI+CgkJCTxzdHJpbmc+c2Fucy1zZXJpZjwvc3RyaW5nPgoJCTwvZWRpdD4KCTwvbWF0Y2g+Cgk8Y2FjaGVkaXI+L2ZvbnRjb25maWc8L2NhY2hlZGlyPgoJPGNvbmZpZz4KCQk8cmVzY2FuPgoJCQk8aW50PjMwPC9pbnQ+CgkJPC9yZXNjYW4+Cgk8L2NvbmZpZz4KPC9mb250Y29uZmlnPgo=";Module["FS_createDataFile"]("/assets", "fonts.conf", decodeBase64(fileData0), true, true, false)

About -s AUTO_NATIVE_LIBRARIES=0, when -flto is enabled, emscripten will generating "system libraries" and cache several .a archives. It does this for all available libraries, including non-used ones, if AUTO_NATIVE_LIBRARIES is enabled. This increases compile time. Currently this includes libGL.a, libal.a, libhtml5.a, which are not used within the project.
Same could be said about AUTO_JS_LIBRARIES but those libraries get used.

cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libstubs.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libstubs.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libc.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libcompiler_rt.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libcompiler_rt.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libc++-noexcept.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc++-noexcept.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libc++abi-noexcept.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc++abi-noexcept.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libdlmalloc.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libdlmalloc.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libc_rt_wasm.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc_rt_wasm.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libsockets.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libsockets.a" for subsequent builds)
cache:INFO:  - ok

[TheOneric]

Another note: the commit message and PR description currently disagree on whether this increases or reduces filesize.

After #146, the only file embedded will be fonts.conf

Yup, fonts.conf seems fine it was mainly the default font itself I’m concerned about. You could either: only cahgne fonts.conf now, or don't change anything in this series but after #146 has dropped the font file from preload (or currently: embed) change the remaining fonts.conf preload to an embed.

About -s AUTO_NATIVE_LIBRARIES=0, when -flto is enabled, emscripten will generating "system libraries" and cache several .a archives. It does this for all available libraries, including non-used ones, if AUTO_NATIVE_LIBRARIES is enabled. This increases compile time. [as those are not actually used in JSO].

Thanks for the explanation! A (concise) version of this would be a great addition to the commit message :)

I suggest this is split into multiple commits, each clearly documenting its purpose and if not blatanly obvious how/why, like this:

• One to improve performance (LTO and CTOR)
• One to speed up the build (AUTONATIVE_LIBRARIES)
• [Possibly moved to Implement lazy font loading, enable embedded font extraction #146, depending on what makes more sense to you:] One to reduce number of files: init-mem and fonts.conf embed

[WeebDataHoarder]

I am moving init-mem and embed to #146, otherwise the commits have been split. I'll change the title of the PR

[TheOneric]

Thanks! One last thing, the first commit’s description still claims LTO would reduce the output size. The PR description suggests and my test seems to verify it actually increases the output size (but iinm LTO is mainly for performance not size, so that’s fine):

$ du --bytes -s dist/js*
6978991	dist/js_noLTO
7597461	dist/js_withLTO

[WeebDataHoarder]

Most of my assumptions are pre-O3 changes on master, so they are all wrong.

Have tested the changes introduced here and -s EVAL_CTORS=1 no longer takes effect (but future Emscripten versions apply it to more methods/sections).

Reworded commit to show this + -flto size increases

6978885: *before changes*
6978885: -s EVAL_CTORS=1
6978885: -s EVAL_CTORS=2
7597539: -flto
7597539: -flto -s EVAL_CTORS=1