Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize webpack builds #968

Merged
merged 3 commits into from
Jan 22, 2019
Merged

Parallelize webpack builds #968

merged 3 commits into from
Jan 22, 2019

Conversation

wincent
Copy link
Contributor

@wincent wincent commented Jan 22, 2019

This is just an intermediate step towards making builds faster.

Before this PR, our build process looked like this:

  1. Run "build:assets" using the gulp executable.
  2. Do webpack.prod build using the webpack executable.
  3. Do webpack.dev build using the webpack executable.

These serial processes took about 2s, 12s, and 38s respectively on my machine: a total of about 52s.

After this commit, we do this:

  1. Use gulp to do everything the "build:assets" task did.
  2. In parallel, run the two webpack builds from the same gulp invocation.

In the end, we wind shave only a few seconds from the original runtime (ie. going down to just under 50s) because we're always going to be limited by the length of the slowest job, and that job will actually run a little slower than usual when in parallel.

I have some follow-ups I'll be doing after this, such as seeing if switching to a different minifier is faster (we're spending most of the time in Uglify). Another will be asking why we commit the "dist" files at all, which seems unconventional.

Test plan:

  • Confirm in-browser that dev environment still works after doing npm run dev and npm run start.
  • Compare dist output with current HEAD of "develop" branch is identical.
  • Compare dist output is same for:
    • Gulp (dev): node_modules/.bin/gulp --gulpfile ...`
    • Webpack(dev): node_modules/.bin/webpack --config webpack.dev.js`
    • Gulp (prod): node_modules/.bin/gulp --gulpfile ... --release`
    • Webpack(prod): node_modules/.bin/webpack --config webpack.prod.js`
      32d3ea1

@wincent wincent requested a review from julien January 22, 2019 13:44
@wincent
Copy link
Contributor Author

wincent commented Jan 22, 2019

Rebased on top of #970, which means we will test the build process and the build result in Travis.

Before this commit, our build process looked like this:

1. Run "build:assets" using the `gulp` executable.
2. Do webpack.prod build using the `webpack` executable.
3. Do webpack.dev build using the `webpack` executable.

These serial processes took about 2s, 12s, and 38s respectively on my
machine: a total of about 52s.

After this commit, we do this:

1. Use `gulp` to do everything the "build:assets" task did.
2. In parallel, run the two webpack builds from the same `gulp`
   invocation.

In the end, we wind shave only a few seconds from the original runtime
(ie. going down to just under 50s) because we're always going to be
limited by the length of the slowest job, and that job will actually run
a little slower than usual when in parallel.

I have some follow-ups I'll be doing after this, such as seeing if
switching to a different minifier is faster (we're spending most of the
time in Uglify). Another will be asking why we commit the "dist" files
at all, which seems unconventional.

Test plan:

- Confirm in-browser that dev environment still works after doing `npm
  run dev` and `npm run start`.
- Compare `dist` output with current HEAD of "develop" branch is
  identical.
- Compare `dist` output is same for:
  - Gulp (dev): node_modules/.bin/gulp --gulpfile ...`
  - Webpack(dev): node_modules/.bin/webpack --config webpack.dev.js`
  - Gulp (prod): node_modules/.bin/gulp --gulpfile ... --release`
  - Webpack(prod): node_modules/.bin/webpack --config webpack.prod.js`
@wincent
Copy link
Contributor Author

wincent commented Jan 22, 2019

And rebased again on top op #971, which will increase our chances of avoiding spurious failures on Travis.

@julien julien merged commit b5094b0 into liferay:2.x-develop Jan 22, 2019
@wincent wincent deleted the parallel-build branch January 24, 2019 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants