Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add browser polyfils for Node.js modules (webpack 5 backwards compat) #16022

Merged
merged 2 commits into from
Aug 10, 2020
Merged

Add browser polyfils for Node.js modules (webpack 5 backwards compat) #16022

merged 2 commits into from
Aug 10, 2020

Conversation

timneutkens
Copy link
Member

This adds the following Node.js core polyfills only when the import is used:

  • path
  • stream
  • vm
  • crypto
  • buffer

Fixes #15948

We'll have a separate issue about adding warnings for the usage of these modules in the browser, some polyfills like crypto are quite heavy and generally not needed for most applications (included accidentally through node_modules).

@ijjk ijjk added the type: next label Aug 9, 2020
@ijjk
Copy link
Member

ijjk commented Aug 9, 2020

Stats from current PR

Default Server Mode (Decrease detected ✓)
General Overall increase ⚠️
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
buildDuration 13.1s 13.8s ⚠️ +689ms
nodeModulesSize 65.6 MB 65.7 MB ⚠️ +194 kB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.37 2.422 ⚠️ +0.05
/ avg req/sec 1054.92 1032.31 ⚠️ -22.61
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.428 1.442 ⚠️ +0.01
/error-in-render avg req/sec 1750.13 1733.28 ⚠️ -16.85
Client Bundles (main, webpack, commons)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..ec1a.js gzip 9.99 kB 9.99 kB ✓
framework.HASH.js gzip 39.1 kB 39.1 kB ✓
main-008324e..83e7.js gzip 6.74 kB 6.74 kB ✓
webpack-ccf5..276a.js gzip 751 B 751 B ✓
Overall change 56.6 kB 56.6 kB ✓
Client Bundles (main, webpack, commons) Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..dule.js gzip 5.89 kB 5.89 kB ✓
framework.HA..dule.js gzip 39.1 kB 39.1 kB ✓
main-c621fe2..dule.js gzip 5.81 kB 5.81 kB ✓
webpack-10c7..dule.js gzip 751 B 751 B ✓
Overall change 51.6 kB 51.6 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
polyfills-75..1629.js gzip 31 kB 31 kB ✓
Overall change 31 kB 31 kB ✓
Client Pages
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-874bd8a..0103.js gzip 1.28 kB 1.28 kB ✓
_error-fa39c..ec40.js gzip 3.45 kB 3.45 kB ✓
hooks-585f07..95a3.js gzip 887 B 887 B ✓
index-c7b63f..fc02.js gzip 227 B 227 B ✓
link-f4d2979..e57b.js gzip 1.29 kB 1.29 kB ✓
routerDirect..ebc7.js gzip 284 B 284 B ✓
withRouter-2..db68.js gzip 284 B 284 B ✓
Overall change 7.71 kB 7.71 kB ✓
Client Pages Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-97e743e..dule.js gzip 626 B 626 B ✓
_error-b4004..dule.js gzip 2.3 kB 2.3 kB ✓
hooks-696209..dule.js gzip 387 B 387 B ✓
index-a4dd74..dule.js gzip 226 B 226 B ✓
link-653c74f..dule.js gzip 1.26 kB 1.26 kB ✓
routerDirect..dule.js gzip 284 B 284 B ✓
withRouter-1..dule.js gzip 282 B 282 B ✓
Overall change 5.37 kB 5.37 kB ✓
Client Build Manifests
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_buildManifest.js gzip 273 B 273 B ✓
_buildManife..dule.js gzip 280 B 280 B ✓
Overall change 553 B 553 B ✓
Rendered Page Sizes
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
index.html gzip 948 B 948 B ✓
link.html gzip 954 B 954 B ✓
withRouter.html gzip 941 B 941 B ✓
Overall change 2.84 kB 2.84 kB ✓
Serverless Mode
General Overall increase ⚠️
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
buildDuration 14.5s 14.4s -15ms
nodeModulesSize 65.6 MB 65.7 MB ⚠️ +194 kB
Client Bundles (main, webpack, commons)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..ec1a.js gzip 9.99 kB 9.99 kB ✓
framework.HASH.js gzip 39.1 kB 39.1 kB ✓
main-008324e..83e7.js gzip 6.74 kB 6.74 kB ✓
webpack-ccf5..276a.js gzip 751 B 751 B ✓
Overall change 56.6 kB 56.6 kB ✓
Client Bundles (main, webpack, commons) Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..dule.js gzip 5.89 kB 5.89 kB ✓
framework.HA..dule.js gzip 39.1 kB 39.1 kB ✓
main-c621fe2..dule.js gzip 5.81 kB 5.81 kB ✓
webpack-10c7..dule.js gzip 751 B 751 B ✓
Overall change 51.6 kB 51.6 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
polyfills-75..1629.js gzip 31 kB 31 kB ✓
Overall change 31 kB 31 kB ✓
Client Pages
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-874bd8a..0103.js gzip 1.28 kB 1.28 kB ✓
_error-fa39c..ec40.js gzip 3.45 kB 3.45 kB ✓
hooks-585f07..95a3.js gzip 887 B 887 B ✓
index-c7b63f..fc02.js gzip 227 B 227 B ✓
link-f4d2979..e57b.js gzip 1.29 kB 1.29 kB ✓
routerDirect..ebc7.js gzip 284 B 284 B ✓
withRouter-2..db68.js gzip 284 B 284 B ✓
Overall change 7.71 kB 7.71 kB ✓
Client Pages Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-97e743e..dule.js gzip 626 B 626 B ✓
_error-b4004..dule.js gzip 2.3 kB 2.3 kB ✓
hooks-696209..dule.js gzip 387 B 387 B ✓
index-a4dd74..dule.js gzip 226 B 226 B ✓
link-653c74f..dule.js gzip 1.26 kB 1.26 kB ✓
routerDirect..dule.js gzip 284 B 284 B ✓
withRouter-1..dule.js gzip 282 B 282 B ✓
Overall change 5.37 kB 5.37 kB ✓
Client Build Manifests
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_buildManifest.js gzip 273 B 273 B ✓
_buildManife..dule.js gzip 280 B 280 B ✓
Overall change 553 B 553 B ✓
Serverless bundles
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_error.js 1.03 MB 1.03 MB ✓
404.html 4.18 kB 4.18 kB ✓
hooks.html 3.82 kB 3.82 kB ✓
index.js 1.03 MB 1.03 MB ✓
link.js 1.07 MB 1.07 MB ✓
routerDirect.js 1.06 MB 1.06 MB ✓
withRouter.js 1.06 MB 1.06 MB ✓
Overall change 5.26 MB 5.26 MB ✓
Commit: 173e1c0

ijjk
ijjk approved these changes Aug 10, 2020
View reviewed changes
Copy link
Member

@ijjk ijjk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ijjk
Copy link
Member

ijjk commented Aug 10, 2020

Stats from current PR

Default Server Mode (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
buildDuration 12.4s 11.7s -712ms
nodeModulesSize 65.6 MB 65.7 MB ⚠️ +194 kB
Page Load Tests Overall increase ✓
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.183 2.265 ⚠️ +0.08
/ avg req/sec 1145.41 1103.73 ⚠️ -41.68
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.368 1.311 -0.06
/error-in-render avg req/sec 1827.98 1906.55 +78.57
Client Bundles (main, webpack, commons)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..ec1a.js gzip 9.99 kB 9.99 kB ✓
framework.HASH.js gzip 39.1 kB 39.1 kB ✓
main-008324e..83e7.js gzip 6.74 kB 6.74 kB ✓
webpack-ccf5..276a.js gzip 751 B 751 B ✓
Overall change 56.6 kB 56.6 kB ✓
Client Bundles (main, webpack, commons) Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..dule.js gzip 5.89 kB 5.89 kB ✓
framework.HA..dule.js gzip 39.1 kB 39.1 kB ✓
main-c621fe2..dule.js gzip 5.81 kB 5.81 kB ✓
webpack-10c7..dule.js gzip 751 B 751 B ✓
Overall change 51.6 kB 51.6 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
polyfills-75..1629.js gzip 31 kB 31 kB ✓
Overall change 31 kB 31 kB ✓
Client Pages
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-874bd8a..0103.js gzip 1.28 kB 1.28 kB ✓
_error-fa39c..ec40.js gzip 3.45 kB 3.45 kB ✓
hooks-585f07..95a3.js gzip 887 B 887 B ✓
index-c7b63f..fc02.js gzip 227 B 227 B ✓
link-f4d2979..e57b.js gzip 1.29 kB 1.29 kB ✓
routerDirect..ebc7.js gzip 284 B 284 B ✓
withRouter-2..db68.js gzip 284 B 284 B ✓
Overall change 7.71 kB 7.71 kB ✓
Client Pages Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-97e743e..dule.js gzip 626 B 626 B ✓
_error-b4004..dule.js gzip 2.3 kB 2.3 kB ✓
hooks-696209..dule.js gzip 387 B 387 B ✓
index-a4dd74..dule.js gzip 226 B 226 B ✓
link-653c74f..dule.js gzip 1.26 kB 1.26 kB ✓
routerDirect..dule.js gzip 284 B 284 B ✓
withRouter-1..dule.js gzip 282 B 282 B ✓
Overall change 5.37 kB 5.37 kB ✓
Client Build Manifests
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_buildManifest.js gzip 273 B 273 B ✓
_buildManife..dule.js gzip 280 B 280 B ✓
Overall change 553 B 553 B ✓
Rendered Page Sizes
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
index.html gzip 948 B 948 B ✓
link.html gzip 954 B 954 B ✓
withRouter.html gzip 941 B 941 B ✓
Overall change 2.84 kB 2.84 kB ✓
Serverless Mode
General Overall increase ⚠️
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
buildDuration 12.7s 13.3s ⚠️ +598ms
nodeModulesSize 65.6 MB 65.7 MB ⚠️ +194 kB
Client Bundles (main, webpack, commons)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..ec1a.js gzip 9.99 kB 9.99 kB ✓
framework.HASH.js gzip 39.1 kB 39.1 kB ✓
main-008324e..83e7.js gzip 6.74 kB 6.74 kB ✓
webpack-ccf5..276a.js gzip 751 B 751 B ✓
Overall change 56.6 kB 56.6 kB ✓
Client Bundles (main, webpack, commons) Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
677f882d2ed8..dule.js gzip 5.89 kB 5.89 kB ✓
framework.HA..dule.js gzip 39.1 kB 39.1 kB ✓
main-c621fe2..dule.js gzip 5.81 kB 5.81 kB ✓
webpack-10c7..dule.js gzip 751 B 751 B ✓
Overall change 51.6 kB 51.6 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
polyfills-75..1629.js gzip 31 kB 31 kB ✓
Overall change 31 kB 31 kB ✓
Client Pages
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-874bd8a..0103.js gzip 1.28 kB 1.28 kB ✓
_error-fa39c..ec40.js gzip 3.45 kB 3.45 kB ✓
hooks-585f07..95a3.js gzip 887 B 887 B ✓
index-c7b63f..fc02.js gzip 227 B 227 B ✓
link-f4d2979..e57b.js gzip 1.29 kB 1.29 kB ✓
routerDirect..ebc7.js gzip 284 B 284 B ✓
withRouter-2..db68.js gzip 284 B 284 B ✓
Overall change 7.71 kB 7.71 kB ✓
Client Pages Modern
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_app-97e743e..dule.js gzip 626 B 626 B ✓
_error-b4004..dule.js gzip 2.3 kB 2.3 kB ✓
hooks-696209..dule.js gzip 387 B 387 B ✓
index-a4dd74..dule.js gzip 226 B 226 B ✓
link-653c74f..dule.js gzip 1.26 kB 1.26 kB ✓
routerDirect..dule.js gzip 284 B 284 B ✓
withRouter-1..dule.js gzip 282 B 282 B ✓
Overall change 5.37 kB 5.37 kB ✓
Client Build Manifests
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_buildManifest.js gzip 273 B 273 B ✓
_buildManife..dule.js gzip 280 B 280 B ✓
Overall change 553 B 553 B ✓
Serverless bundles
vercel/next.js canary timneutkens/next.js add/webpack-polyfills Change
_error.js 1.03 MB 1.03 MB ✓
404.html 4.18 kB 4.18 kB ✓
hooks.html 3.82 kB 3.82 kB ✓
index.js 1.03 MB 1.03 MB ✓
link.js 1.07 MB 1.07 MB ✓
routerDirect.js 1.06 MB 1.06 MB ✓
withRouter.js 1.06 MB 1.06 MB ✓
Overall change 5.26 MB 5.26 MB ✓
Commit: 9db5593

@kodiakhq kodiakhq bot merged commit 843d584 into vercel:canary Aug 10, 2020
@timneutkens timneutkens deleted the add/webpack-polyfills branch August 10, 2020 07:37
merceyz
merceyz reviewed Aug 10, 2020
View reviewed changes
packages/next/build/webpack-config.ts
Comment on lines +362 to +366
stream: 'stream-browserify',
path: 'path-browserify',
crypto: 'crypto-browserify',
buffer: 'buffer',
vm: 'vm-browserify',
Copy link
Contributor

@merceyz merceyz Aug 10, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timneutkens These should go through require.resolve

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point, forgot those don't resolve automatically. Opened #16033 with you as co-author.

@timneutkens timneutkens mentioned this pull request Aug 10, 2020
Merged
@tbergquist-godaddy tbergquist-godaddy mentioned this pull request Aug 17, 2020
Closed
@Timer Timer mentioned this pull request Aug 21, 2020
Merged
kodiakhq bot pushed a commit that referenced this pull request Aug 21, 2020
@Timer
Do not alias Node modules for webpack 4 (#16452)
7c7ecaa 
Fixes #16259
Caused by #16022
Follow up issue: #16450

This PR skips polyfilling Node modules in webpack 4 which are ignored by the `browsers` key.
@ntucker ntucker mentioned this pull request Oct 16, 2020
Merged
10 tasks
wooorm added a commit to vfile/vfile that referenced this pull request Dec 3, 2020
@wooorm
Add manual browser polyfills for path, process
41fff71 
Many new developers do not know how to configure webpack.
When they use a package that somewhere deep down uses `path` or `process` or
whatnot, through webpack 5, they mistakingly think this package does not work
in browsers.

This infuriating mistake by webpack leads to superfluous work for maintainers
and bigger bundles for users because polyfills add bloat.

See:
* <https://blog.sindresorhus.com/webpack-5-headache-b6ac24973bf1>
* <vercel/next.js#16022>
@wooorm wooorm mentioned this pull request Dec 3, 2020
Merged
wooorm added a commit to vfile/vfile that referenced this pull request Dec 6, 2020
@wooorm
Add manual browser polyfills for path, process (#59)
91f1303 
Many new developers do not know how to configure webpack.
When they use a package that somewhere deep down uses `path` or `process` or
whatnot, through webpack 5, they mistakingly think this package does not work in
browsers.

This infuriating mistake by webpack leads to superfluous work for maintainers
and bigger bundles for users because polyfills add bloat.

See:
* <https://blog.sindresorhus.com/webpack-5-headache-b6ac24973bf1>
* <vercel/next.js#16022>

Note that this change shaves off 40% from the bundle size compared to including
`path` and `process`, however, it will double the bundlephobia size (because it
normally does not include path/process) and will hurt folks who depend on
another project that includes `path`.

Related to #16.
Related to #28.
Related to #38.
Related to #35.
Related to #56.
Related to #57.
Related to #58.
Related to remarkjs/react-markdown#492.
Related to remarkjs/react-markdown#514.

Reviewed-by: Christian Murphy <christian.murphy.42@gmail.com>
@JayaKrishnaNamburu JayaKrishnaNamburu mentioned this pull request Jun 11, 2021
Closed
This was referenced Jun 26, 2021
Open
Open
Open
Open
Open
This was referenced Sep 17, 2021
Open
Open
Open
Open
This was referenced Sep 18, 2021
Open
Open
@vercel vercel locked as resolved and limited conversation to collaborators Jan 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@merceyz merceyz merceyz left review comments

@ijjk ijjk ijjk approved these changes

@lfades lfades Awaiting requested review from lfades

@Timer Timer Awaiting requested review from Timer

Assignees
No one assigned
Labels
type: next
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Polyfill common Node.js packages for webpack 5
3 participants
@timneutkens @ijjk @merceyz