Json page data compression

[titusgahissy]

I noticed that page data (json) fetched from the server are not compressed. Is there a particular reason for that or is this something planned ?

[rauchg]

They should be!

[nkzawa]

Including html responses and response headers, I'm wondering how it should be:

• just enable the compression
• make it customizable including modifying headers, possibly by Custom server logic / routing #25
• don't support it at all. you have to prepare a proxy server for compression and modify header contents.

[rauchg]

I would say we can have compression: false in config for the rare case that you'd want to disable it. In most cases even if you have a proxy in front, they look at the headers to avoid double compression. It's a thing that most likely you want out of the box.

[tdurand]

Hello,

Is there some work already made on that ? If not i can try to submit a PR because it's kind of critical, i'm loading 10x more json data when going from page to page as there is no compression, and as there is no prefetch / hook for loading on the router yet , this cause 1-3 s waiting without any visual feedback.

Seems there is no compression as well on the static js assets like common.js, here is an output of webpage test for an app i'm working on.

Use gzip compression for transferring compressable responses: 51/100
802.0 KB total in compressible text, target size = 412.2 KB - potential savings = 389.8 KB

FAILED - (486.6 KB, compressed = 105.3 KB - savings of 381.3 KB) - https://xxx/viewer.json
FAILED - (13.0 KB, compressed = 4.5 KB - savings of 8.4 KB) - https://xxx/_next/commons.js

I had a look at the server implementation internals of next.js but i struggle to find where gzip is enabled for the normal request with html, i don't see any dependencies to zlib or something else: https://nodejs.org/api/zlib.html#zlib_zlib , but it's actually gzipped on the first ssr request.

I'll be happy to help and submit a PR is you can give me some inputs on where gzip if happening for html data, then i could fix it for json and static elements.

Thanks !

[tdurand]

I saw a bunch of PR have been merged, any inputs on this one ? I'd like to help to get that landed for the next release of next ;-) , cc @arunoda

[rauchg]

The one thing I'd like to see to ship something like this is that everything would have to be compressed ahead-of-time. ie: we produce .gz bundles. Otherwise I don't want to waste cycles compressing in Node land, since front proxies like Nginx tend to be much better at this

[rauchg]

Although, I'd also like to see a solution that's pluggable. Compression itself is really opinionated. What if I want to use zopfli and trade off compile time for better compression?

[tdurand]

So the compression is happening in the nginx proxy for html right now ? I'm struggling to find it in the code.

[tdurand]

Ok, got this working using cloudflare in front of the next.js app which gzip the json payload, seems it's not an issue with next.js but with the hosting, in this case now that does not gzip json.

[rauchg]

I think if we can do ahead-of-time compression it's a good way of having gzip support without having to go through so much trouble.

That said, it'd be cool for us to think about how to make that pluggable. Maybe with a config option of a callback to compress. That way people can compress with zopfli if they want :)

[tdurand]

Yeah it will definitely be better than relying on nginx or other proxies. I'll think about it now that I understood how it works. El 16 dic. 2016 7:10 p. m., "Guillermo Rauch" <notifications@github.com> escribió:

…

I think if we can do ahead-of-time compression it's a good way of having gzip support without having to go through so much trouble. That said, it'd be cool for us to think about how to make that pluggable. Maybe with a config option of a callback to compress. That way people can compress with zopfli if they want :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#243 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgkVmtIfGnGRaO-UZJhWadZiVSsUPJZks5rIwxugaJpZM4Kv4g_> .

[rauchg]

I think an elegant solution could be in next.config.js to supply a compress fn that receives the incoming fs stream, and the type ('bundle', 'metadata'), which is not necessary but could be interesting.

// next.config.js
{

  compress (stream, { type }) {
    // pipe to a compression thing here
  }
}

Then the user can choose how to compress. By default, we do a typical gzip compression with the default level.