Feature Idea: Brotli support in core

[MylesBorins]

Taking a quick peek at "can i use" shows that brotli is now supported on Edge, Firefox, Chrome, Safari, and iOS Safari.

The iltorb package has 912k downloads a month.
The brotli npm package has 250k download a month

If people are open to this I would be willing to collaborate on a patch to add this functionality.

[addaleax]

Sounds like a good idea to me. I’m somewhat familiar with iltorbs code base, but the brotli API has also gotten to a point where it’s similar enough to zlib that we probably wouldn’t even have to write much code to integrate it into Node.

/cc @MayhemYDG

[devsnek]

+1

p.s. if we're adding more compression stuff in core perhaps we should put it under one builtin module name require('compression').zlib/brotli/whateverelsecomesnext

[MylesBorins]

to be honest this may be the point where we decide if we want to use a nodejs namespace... as there is already a compression lib with 9M downloads a month and as mentioned above the brotli lib has 250k downloads a month. That being said I think it is a separate conversation that can be worked through independently

[nstepien]

I think it makes sense. IMO Node, as a web server, should strive to support low-level modern web features, much like http2.

[silverwind]

Yes, please. Brotli usage in Node.js has been held back by the barrier of native modules, this would certainly allow for broader adoption.

[MylesBorins]

Pinging this to see if anyone is interested in taking this on as an action item

[vkurchatkin]

It seems weird that "can I use" and download numbers are enough for adding brotli, but were never enough for adding websockets

[MylesBorins]

@vkurchatkin tbh I think that we should revisit adding websockets.

If we look at the original thread #1010 some of the people who originally were -1 changed their opinions. In fact we ship a partial implementation of ws already, and have an open EPS exploring adding it nodejs/node-eps#6

The original decision by the TC at the time was to rather implement lower level buffer method, but that was abandoned.

#1202

I thinks I'll open another issue to discuss

edit:

issue for reference #19308

[MatonAnthony]

Hello,

Which kind of skillset would it require? Would someone be willing to mentor me if I want to try it?

[jasnell]

Quick question: why would this need to be added to core vs user land? Is there a notable perf improvement or impl improvement to be gained relative to the userland module?

[MylesBorins]

This originally came up in a conversation with @kristoferbaxter where they mentioned having issues with userland implementations, and a lack of support in core being a reason for companies / enterprises to not adopt the format despite performance improvements compared to zlib.

I'm paraphrasing and will let them chime in

[kristoferbaxter]

Hi there.

There are a few userland implementations with differing drawbacks, but the most prominent are iltorb and wasm-brotli afaik.

iltorb – https://github.com/MayhemYDG/iltorb
Uses prebuild to download a pre-compiled binary for your platform, if it exists. Otherwise, it will use node-gyp to build the module.

Pros: Works well for those who can leverage the pre-compiled binaries. Supports compression options.
Cons: Projects including it encounter issues when a pre-compiled binary is not available.

wasm-brotli – https://github.com/dfrankland/wasm-brotli
WebAssembly compiled Brotli library.

Pros: No need for a pre-built binary.
Cons: Performance. Lacks compression options.

---

TLDR;
Since Brotli is quite well supported, can support runtime compression (using quality levels 4-6), and buildtime compression (using quality level 11), it is a prime candidate for use targeting modern browsers or native applications. Its inclusion in core would reduce friction to leverage it and hopefully make a meaningful difference in application performance for many usecases.