Better `?worker&inline`

[L2NE-dev]

Description

I want to getting smaller inline code of Web Worker, uses another than Base64 encoding, or using compression API. I want to have API to code transform control.

Suggested solution

• import WorkerCode from "./Worker.ts?worker&inline-compressed"
• Vite API for control JS/TS encoding.

Alternative

No response

Additional context

In general, I currently getting big code of library (for example, 64kB (now 40kB) minified in base case). But just base64 encoding isn't always best solution, I want to reduce library size even more (at least 20kB or lower).

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[sapphi-red]

I guess we can try using URL encoded instead of base64 like mini-svg-data-uri does. It should reduce the length if the code mainly includes ASCII characters.

Regarding using the compression API, I don't think that would actually reduce the code size. In most cases, the code would be served as gzip or other compressed format, so it won't affect the size sent over the network.

[L2NE-dev]

I guess we can try using URL encoded instead of base64 like mini-svg-data-uri does. It should reduce the length if the code mainly includes ASCII characters.

Regarding using the compression API, I don't think that would actually reduce the code size. In most cases, the code would be served as gzip or other compressed format, so it won't affect the size sent over the network.

I would to have single-file library with embedded web worker. Also, input language is TypeScript with some TS modules (which will baked), so I can't just simply encode like SVG assets. So I today try to modify Vite itself. Isn't enough to just encode to base64, needs to pre-compress compiled web worker code.

Update: I tried to do such trick, got 24kB of minified code. Interesting what happen if embed just worker code and compress to GZIP fully (me feels like from 4 to 8 kB of gzipped), but (I think) not all JS module loaders will supports full compressions (i.e. whole module). Before was 40kB.

[sapphi-red]

so I can't just simply encode like SVG assets.

I was suggesting changing Vite's behavior, not doing it on the user land.

Isn't enough to just encode to base64, needs to pre-compress compiled web worker code.

What do you mean by pre-compress?

but (I think) not all JS module loaders will supports full compressions

Effectively all browsers support gzip compressed contents.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding#browser_compatibility

[L2NE-dev]

What do you mean by pre-compress?

That means taking the compiled Worker code (from TypeScript, Terser, etc.) with embedded libraries and modules, compressing it into binary code with some GZIP, and then encoding it into base64 string. Then insert the same code inline into the main library code. To initialize the embedded Worker, you need to use the browser API to decompress this binary code (having decoded base64 beforehand), and start the Worker via the Blob URL. This is the strategy for single-file library coding, i.e. zero-deps.

Some part of code compression mod of Vite (but some moments not shown).

        if (compressRE.test(id)) {
          const chunk = await bundleWorkerEntry(config, id)
          const b64c = Buffer.from(
            await new Promise<Uint8Array>((r) =>
              gzip(strToU8(chunk.code), { level: 9 }, (_, d) => r(d)),
            ),
          ).toString('base64')
          return {
            code: `const b64c = "${b64c}"; export default b64c;`,
            // Empty sourcemap to suppress Rollup warning
            map: { mappings: '' },
          }
        }

Next part is encoded inline info code (de-compress loading of code), runtime unpack.

/*@__MANGLE_PROP__*/ import $raw$ from "./worker.ts?worker&compress"; /*@__MANGLE_PROP__*/ const IW = $raw$;

/*@__PURE__*/ /*@__MANGLE_PROP__*/ 
const loadCompressed = async (b64c: string): Promise<string|null> => {
    const blob = new Blob([Uint8Array.from(atob(b64c), c => c.charCodeAt(0))], {type: "application/gzip"});
    const ds = new DecompressionStream("gzip");
    const decompressedStream = blob.stream().pipeThrough(ds);
    const response = await (new Response(decompressedStream, {headers: new Headers({"Content-Type": "application/javascript" })})).blob();
    return URL.createObjectURL(response);
}

/*@__MANGLE_PROP__*/ export const PRELOAD = !URL.canParse(IW) ? /*@__PURE__*/ /*@__MANGLE_PROP__*/ loadCompressed(IW as unknown as string) : IW;
/*@__MANGLE_PROP__*/ export default PRELOAD;

[sapphi-red]

I see. I still don't see the advantage of doing that (running the compression inside the code). Would it have effectively smaller size than serving the JS file in gzip?

[L2NE-dev]

I see. I still don't see the advantage of doing that (running the compression inside the code). Would it have effectively smaller size than serving the JS file in gzip?

At the moment I can't find a way to insert into raw code (without compression and even without base64) just a string-template ('...JS code'), so that the overall gzip compression would be even more efficient.