Enable gzip compression

[kdzwinel]

Summary

Gzip compression wasn't enabled causing yarn to download twice the data npm does. Fixes #945 .

Test plan
yarn add yo

Before - 9.9MB

After - 5.3MB

[addyosmani]

On first looks, this work is solid and appears to work well. Good catch, @kdzwinel! We should land this.

[Daniel15]

Woah, good catch! Thanks for the pull request.

[addyosmani]

🎉

[cpojer]

oops!

[cpojer]

Did anybody measure the performance impact of this change on an empty cache?

[kdzwinel]

Note that this change only impacts "Resolving packages..." phase when metadata (e.g. https://registry.npmjs.org/babel-runtime) is fetched. Actual packages were always compressed (because they are .tgz'ed on the server). I've tested installation of couple of popular packages (no yarn.lock, empty cache) on a good connection (~20Mbps) and got these results:

yo

uncompressed

Done in 14.98s.
Done in 9.15s. < median
Done in 8.24s. < min
Done in 10.21s.
Done in 8.90s.

compressed

Done in 8.25s.
Done in 7.34s. < min
Done in 7.93s.
Done in 8.00s. < median
Done in 8.41s.

9.15s - 8.00s = 1.15s
1.15s / 9.15s = 12.5%

react

uncompressed
Done in 6.26s.
Done in 3.02s. < min
Done in 3.18s.
Done in 3.74s.
Done in 3.34s. < median

compressed
Done in 2.68s.
Done in 2.65s. < min
Done in 2.94s.
Done in 2.69s. < median
Done in 2.70s.

3.34s - 2.69s = 0.65s
0.65s / 3.34s = 19.5%

browserify

uncompressed browserify
Done in 10.64s.
Done in 5.65s.
Done in 5.59s. < median
Done in 4.98s.
Done in 4.80s. < min

compressed browserify
Done in 3.90s. < min
Done in 3.93s.
Done in 4.42s. < median
Done in 4.75s.
Done in 4.77s.

5.59s - 4.42s = 1.18s
1.18s / 5.59s = 21.1%

yo, react, browserify

uncompressed
Done in 16.10s.
Done in 13.05s.
Done in 12.80s.
Done in 14.03s. < median
Done in 15.47s.

compressed

Done in 12.64s.
Done in 12.01s. < median
Done in 13.13s.
Done in 11.51s.
Done in 11.08s.

14.03s - 12.01s = 2.02s
2.02s / 14.03s = 14.3%

[cpojer]

That's awesome. Thanks for following up with a benchmark. 👏

[vjeux]

Not sure how feasible it is but we've seen meaningful speed increase in the node_modules downloader by:

• using xz instead of gzip. It takes more time to compress but since it's a one time operation it was worth it.
• pipe the output of wget directly to tar instead of going through a temporary file in between.