base64_encode embedded data files #14526
Conversation
Reduces the embedded data size by about 75% It also plays nicer with the acorn optimizer, making it use less RAM when there is embedded data
|
Thank you for submitting a pull request! If this is your first PR, make sure to add yourself to AUTHORS. |
| @@ -398,7 +398,7 @@ function JSify(functionsOnly) { | |||
| print(preprocess(read('arrayUtils.js'))); | |||
| } | |||
|
|
|||
| if (SUPPORT_BASE64_EMBEDDING && !MINIMAL_RUNTIME) { | |||
| if ((SUPPORT_BASE64_EMBEDDING || FORCE_FILESYSTEM) && !MINIMAL_RUNTIME) { | |||
There was a problem hiding this comment.
How about replacing this specific change with this:
diff --git a/emcc.py b/emcc.py
index eaf519567..269e5cebb 100755
--- a/emcc.py
+++ b/emcc.py
@@ -1877,6 +1877,10 @@ def phase_linker_setup(options, state, newargs, settings_map):
else:
settings.SYSTEM_JS_LIBRARIES.append((0, shared.path_from_root('src', 'library_pthread_stub.js')))
+ if settings.FORCE_FILESYSTEM:
+ # enable base64 decoding for file packager data
+ settings.SUPPORT_BASE64_EMBEDDING = 1
+
if settings.FORCE_FILESYSTEM and not settings.MINIMAL_RUNTIME:
# when the filesystem is forced, we export by default methods that filesystem usage
# may need, including filesystem usage from standalone file packager output (i.e.?
There was a problem hiding this comment.
The only reason I didn't do it that way is because using SUPPORT_BASE64_EMBEDDING has side effects which I'm not sure are needed in this case. Specifically, enabling SUPPORT_BASE64_EMBEDDING adds some calls in the code to tryParseAsDataURI, which wouldn't be called otherwise.
However, if that's OK, then sure,it's easier to just enable SUPPORT_BASE64_EMBEDDING 👍
There was a problem hiding this comment.
Oh, very good point. Yes, thinking on this more carefully, I agree with you.
| @@ -57,6 +57,7 @@ | |||
| subdir\file, in JS it will be subdir/file. For simplicity we treat the web platform as a *NIX. | |||
| """ | |||
|
|
|||
| import base64 | |||
There was a problem hiding this comment.
Nice that python makes this easy...
There was a problem hiding this comment.
Yeah is was rather easy to encode the files. Adding decoding was a little trickier but not by much, it's a small change with a visible impact.
|
Odd, one of the commits mentions adding yourself to AUTHORS, but I don't see it in the final code - ? (That's the last thing to do before merging this.) |
|
Yes I did notice that, for some reason the file didn't change (maybe I forgot to save it?). I was waiting to see if I had any other change to do and I'd add it then. I'll add that change now. |
|
Hopefully #16050 will make this even more efficient and even avoid copies completely. |
When
file_packager --embed(oremcc --embed-file) is used, this change reduces the embedded data size by about 75%.It also plays nicer with the acorn optimizer, making it use less RAM when there is embedded data.
Do note that, as reported in #14482, the acorn optimizer still uses a lot of RAM, which possibly can be improved (that is way beyond my knowledge of this project). However, since the final html is now much smaller, less RAM is used by the optimizer as well.