Upgrade webpack to 2.x #14786

samouri · 2017-06-06T03:07:18Z

Heavily building off of the work done by @ehg in #10799, I'm attempting to upgrade our build system to Webpack 2.x. One of the largest draws is the promise of tree shaking which can reduce the size of our bundles.

Problems to fix:

Infinite loop / crash. Seems like this is being caused by WebpackStableBuildPlugin.
enable HMR
fix long term caching Upgrade webpack to 2.x #14786 (comment)

matticbot · 2017-06-06T03:07:19Z

Test live: https://calypso.live/?branch=update/webpack2-2

samouri · 2017-06-06T03:53:09Z

It looks like there is now a webpack-provided way of creating the kind of long-term caching we were striving for with WebpackStableBuildPlugin. I'll follow the instructions here

ehg · 2017-06-06T20:02:42Z

cc @jsnmoon for codemod goodness.

Also, we should just get rid of react-hot for the moment.

ehg · 2017-06-06T20:03:58Z

server/bundler/loader.js

@@ -163,7 +163,7 @@ function singleEnsure( chunkName ) {
 module.exports = function( content ) {
 	var sections;

-	this.cacheable && this.cacheable();
+	// this.cacheable && this.cacheable(); // TODO @ehg why did you comment this out?


No longer needed in webpack2. On by default.

awesomesauce

ehg · 2017-06-06T20:04:24Z

client/reader/stream/index.jsx

@@ -48,6 +48,7 @@ const HEADER_OFFSET_TOP = 46;
 const MIN_DISTANCE_BETWEEN_RECS = 4; // page size is 7, so one in the middle of every page and one on page boundries, sometimes
 const MAX_DISTANCE_BETWEEN_RECS = 30;

+


samouri · 2017-06-07T11:01:57Z

@ehg why do we want codemods in this pr? Ideally we can keep this to just upgrading to webpack2 and trying to maintain the same feature set

ehg · 2017-06-07T17:39:04Z

@ehg why do we want codemods in this pr? Ideally we can keep this to just upgrading to webpack2 and trying to maintain the same feature set

We don't. I'm just looping Jason in :)

ehg · 2017-06-07T17:41:15Z

Also, I should explain myself a bit better here:

Also, we should just get rid of react-hot for the moment.

I've noticed quite a few problems involving tree-shaking and react-hot, so we might as well just remove it all together, then come back to it when/if we want it. It's pretty useless considering out iterative build times atm IMHO :)

samouri · 2017-06-08T09:45:31Z

HMR is now working in this branch! 🎉

What problems were you seeing @ehg

samouri · 2017-06-08T12:58:35Z

Build Perf analysis:

webpack 1.3.x build times:

first build: server: 32s, css: 20s, client: 153s
second build: server: 32s, css: 0s, client: 42s // question: is babel-cache the only thing making this faster?
incremental build: 25s

webpack 2.6.1 build times:

first build: server: 48s, css: 19s, client: 299s
second build: server: 48s, css: 0s, client: 178s
incremental build: 17s

webpack 2.3.2 build times: Airbnb employee says that 2.3.2 is faster.
note: may be an unfair comparison -- needed to remove NamedChunksPlugin.

second build: server: 42s, client: 92s
incremental build: 16s

All tests conducted on a 2015 Macbook Pro

Conclusions:
incremental builds have gotten better, but initial build is much worse

samouri · 2017-06-09T07:42:13Z

I've created a stats.json file with webpack 2.6.1 for use with the webpack build analysis tool located at: http://webpack.github.io/analyse/ .

TODO: drop link to file

I'm currently having some trouble figuring out what it all means, but there are some suggestions re. module name resolution etc that seem like they could speed up the builds. In particular this affects notifications-client

ehg · 2017-06-09T10:39:21Z

What problems were you seeing @ehg

It wasn't so much that HMR/react-hot weren't working on my initial branch. Once I enabled tree-shaking, I got into some trouble. This was because of import hoisting with react-hot v1, but I've seen problems mentioned with v3 too.

samouri · 2017-06-12T22:02:42Z

I'm not too happy about the DevEx performance hit that a straight migration to the latest webpack causes.

I'm spinning a PR off of this one to try to speed things upcreating a dev-only DLL chunk. We used to do this but stopped because we ran into issues re. caching: #11867

By only doing this in dev, it shouldn't affect caching in prod

samouri · 2017-06-12T22:05:06Z

It wasn't so much that HMR/react-hot weren't working on my initial branch. Once I enabled tree-shaking, I got into some trouble

For now i'll keep it in. If we need to strip it out in a future pr then so be it :(

DanReyLop · 2017-06-13T05:58:58Z

I'm not too happy about the DevEx performance hit that a straight migration to the latest webpack causes.

What about we migrate to 2.3.2 for now? The performance regression was in 2.4.0. Hopefully in a few weeks we can jump directly to 3.X when that issue is solved.

samouri · 2017-06-13T15:12:24Z

What about we migrate to 2.3.2 for now? The performance regression was in 2.4.0. Hopefully in a few weeks we can jump directly to 3.X when that issue is solved.

Is there any reason to think 3.X will solve the perf issue?

2.3.2 is missing the two plugins essential for caching

DanReyLop · 2017-06-13T15:21:00Z

Is there any reason to think 3.X will solve the perf issue?

webpack/webpack#4863 is marked as Urgent priority. It's not fixed yet, but there's reason to believe they will try to fix it soon.

2.3.2 is missing the two plugins essential for caching

Oh, damn. I guess you could try forking webpack and reverting webpack/webpack@983904c (which is the cause of the performance regression), but that seems a bit extreme.

samouri · 2017-06-13T15:22:47Z

Oh, damn. I guess you could try forking webpack and reverting webpack/webpack@983904c (which is the cause of the performance regression), but that seems a bit extreme.

We could also bundle calypso with the source for the two plugins in question and then delete the files when we upgrade:

samouri · 2017-06-13T15:53:42Z

By using the cache options for babel-loader, the commit bcb59d4 Speeds up bundler build from 42s --> 13s 🎉

DanReyLop

It works fine now on Windows. If that was the only blocker, you can 🚢 it :)

matticbot · 2017-06-27T06:43:06Z

@samouri This PR needs a rebase

samouri · 2017-06-27T15:17:19Z

Up until now I had only been testing local build times. I'm going to merge this in the next few hours so I just tested the docker build times as well.

when running the command: time npm run build-docker

before: 9.07s user 1.95s system 2% cpu 6:41.59 total
after: 9.14s user 2.03s system 3% cpu 5:00.50 total

Thats a reduction from 6:41 --> 5:00 🎉

…orever

* Perf improvements: disable code-splitting, and use happypack * use happypack also on the server * re-enable code-splitting

blowery · 2017-06-27T16:07:04Z

when running the command: time npm run build-docker

I wonder if we want to tweak https://github.com/amireh/happypack#threads-number at all. How many cores do our docker build instances get...?

samouri · 2017-06-27T16:20:14Z

@blowery: definitely something we can and should tweak! We'll want to do some testing directly on the machines doing our docker builds.

This reverts commit 46d5676.

This reverts commit f1cffef.

* Revert "Revert "Upgrade webpack to 2.x (#14786)" (#15617)" This reverts commit f1cffef. * Add in NameAllModulePlugins + update shrinkwrap * upgrade to 3.0 and name all modules/chunks * vendor chunks minChunks infinity so it only includes what we specify * Update shrinwrap + add peer dependency * cross-env for webpack-dashboard * remove copied NamedModulesPlugin * remove ChunkFileNamePlugin * fix error handling + modify readme.md * remove ajv, bump webpack3, update shrinkwrap * add ajv back in * delete cruft empty concat

samouri added Build [Type] Performance [Type] Enhancement Changes to an existing feature — removing, adding, or changing parts of it labels Jun 6, 2017

samouri self-assigned this Jun 6, 2017

matticbot added the [Size] L Large sized issue label Jun 6, 2017

samouri added the [Status] In Progress label Jun 6, 2017

matticbot added [Size] XL Probably needs to be broken down into multiple smaller issues and removed [Size] L Large sized issue labels Jun 6, 2017

ehg reviewed Jun 6, 2017

View reviewed changes

samouri force-pushed the update/webpack2-2 branch from 8801509 to e1ace8f Compare June 8, 2017 09:44

alexsanford mentioned this pull request Jun 8, 2017

Performance: Use Webpack's AggressiveSplittingPlugin to improve browser caching performance #14922

Closed

DanReyLop approved these changes Jun 26, 2017

View reviewed changes

matticbot added the [Status] Needs Rebase label Jun 27, 2017

samouri added 10 commits June 27, 2017 11:18

Upgrade Build System to Webpack2.0

02da7fd

added back in NamedChunk NamedModules and WebpackChunkHash

e91e302

remove NamedChunksPlugin

3321aaa

Make sure Dashboard only gets included in dev build otherwise hangs f…

340c234

…orever

.m.js --> .min.js

c6c67c7

Performance improvements to webpack2.x configuration (#15413)

e5bec46

* Perf improvements: disable code-splitting, and use happypack * use happypack also on the server * re-enable code-splitting

Do not use happypack for windows

fb84585

getModuleVersion --> window compat dir

ca5da41

flip cond whoops

f5b38b7

reshrinkwrap

0af5962

samouri force-pushed the update/webpack2-2 branch from 5ab3170 to 0af5962 Compare June 27, 2017 15:23

samouri removed the [Status] Needs Rebase label Jun 27, 2017

samouri merged commit 46d5676 into master Jun 27, 2017

samouri deleted the update/webpack2-2 branch June 27, 2017 16:37

matticbot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Jun 27, 2017

samouri added a commit that referenced this pull request Jun 28, 2017

Revert "Upgrade webpack to 2.x (#14786)"

f9a5f24

This reverts commit 46d5676.

samouri added a commit that referenced this pull request Jun 28, 2017

Revert "Upgrade webpack to 2.x (#14786)" (#15617)

f1cffef

This reverts commit 46d5676.

samouri added a commit that referenced this pull request Jun 28, 2017

Revert "Revert "Upgrade webpack to 2.x (#14786)" (#15617)"

9a80cab

This reverts commit f1cffef.

samouri added a commit that referenced this pull request Jul 3, 2017

Revert "Revert "Upgrade webpack to 2.x (#14786)" (#15617)"

a01b36a

This reverts commit f1cffef.

samouri added a commit that referenced this pull request Jul 10, 2017

Revert "Revert "Upgrade webpack to 2.x (#14786)" (#15617)"

429cc72

This reverts commit f1cffef.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade webpack to 2.x #14786

Upgrade webpack to 2.x #14786

samouri commented Jun 6, 2017 •

edited

Loading

matticbot commented Jun 6, 2017

samouri commented Jun 6, 2017 •

edited

Loading

ehg commented Jun 6, 2017

ehg Jun 6, 2017

samouri Jun 7, 2017

ehg Jun 6, 2017

samouri Jun 7, 2017

samouri commented Jun 7, 2017

ehg commented Jun 7, 2017

ehg commented Jun 7, 2017

samouri commented Jun 8, 2017 •

edited

Loading

samouri commented Jun 8, 2017 •

edited

Loading

samouri commented Jun 9, 2017

ehg commented Jun 9, 2017

samouri commented Jun 12, 2017 •

edited

Loading

samouri commented Jun 12, 2017

DanReyLop commented Jun 13, 2017

samouri commented Jun 13, 2017

DanReyLop commented Jun 13, 2017

samouri commented Jun 13, 2017

samouri commented Jun 13, 2017 •

edited

Loading

DanReyLop left a comment

matticbot commented Jun 27, 2017

samouri commented Jun 27, 2017

blowery commented Jun 27, 2017

samouri commented Jun 27, 2017

		@@ -48,6 +48,7 @@ const HEADER_OFFSET_TOP = 46;
		const MIN_DISTANCE_BETWEEN_RECS = 4; // page size is 7, so one in the middle of every page and one on page boundries, sometimes
		const MAX_DISTANCE_BETWEEN_RECS = 30;

Upgrade webpack to 2.x #14786

Upgrade webpack to 2.x #14786

Conversation

samouri commented Jun 6, 2017 • edited Loading

matticbot commented Jun 6, 2017

samouri commented Jun 6, 2017 • edited Loading

ehg commented Jun 6, 2017

ehg Jun 6, 2017

Choose a reason for hiding this comment

samouri Jun 7, 2017

Choose a reason for hiding this comment

ehg Jun 6, 2017

Choose a reason for hiding this comment

samouri Jun 7, 2017

Choose a reason for hiding this comment

samouri commented Jun 7, 2017

ehg commented Jun 7, 2017

ehg commented Jun 7, 2017

samouri commented Jun 8, 2017 • edited Loading

samouri commented Jun 8, 2017 • edited Loading

samouri commented Jun 9, 2017

ehg commented Jun 9, 2017

samouri commented Jun 12, 2017 • edited Loading

samouri commented Jun 12, 2017

DanReyLop commented Jun 13, 2017

samouri commented Jun 13, 2017

DanReyLop commented Jun 13, 2017

samouri commented Jun 13, 2017

samouri commented Jun 13, 2017 • edited Loading

DanReyLop left a comment

Choose a reason for hiding this comment

matticbot commented Jun 27, 2017

samouri commented Jun 27, 2017

blowery commented Jun 27, 2017

samouri commented Jun 27, 2017

samouri commented Jun 6, 2017 •

edited

Loading

samouri commented Jun 6, 2017 •

edited

Loading

samouri commented Jun 8, 2017 •

edited

Loading

samouri commented Jun 8, 2017 •

edited

Loading

samouri commented Jun 12, 2017 •

edited

Loading

samouri commented Jun 13, 2017 •

edited

Loading