Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor webpack loader execution #4880

Merged
merged 6 commits into from
May 23, 2023
Merged

refactor webpack loader execution #4880

merged 6 commits into from
May 23, 2023

Conversation

sokra
Copy link
Member

@sokra sokra commented May 9, 2023

@vercel
Copy link

vercel bot commented May 9, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
examples-nonmonorepo 🔄 Building (Inspect) Visit Preview 💬 Add feedback May 23, 2023 7:55am
turbo-site ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 23, 2023 7:55am
9 Ignored Deployments
Name Status Preview Comments Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Visit Preview May 23, 2023 7:55am
examples-cra-web ⬜️ Ignored (Inspect) May 23, 2023 7:55am
examples-designsystem-docs ⬜️ Ignored (Inspect) May 23, 2023 7:55am
examples-gatsby-web ⬜️ Ignored (Inspect) Visit Preview May 23, 2023 7:55am
examples-kitchensink-blog ⬜️ Ignored (Inspect) Visit Preview May 23, 2023 7:55am
examples-native-web ⬜️ Ignored (Inspect) Visit Preview May 23, 2023 7:55am
examples-svelte-web ⬜️ Ignored (Inspect) Visit Preview May 23, 2023 7:55am
examples-tailwind-web ⬜️ Ignored (Inspect) Visit Preview May 23, 2023 7:55am
examples-vite-web ⬜️ Ignored (Inspect) Visit Preview May 23, 2023 7:55am

@github-actions
Copy link
Contributor

github-actions bot commented May 9, 2023

✅ This change can build next-swc

@github-actions
Copy link
Contributor

github-actions bot commented May 9, 2023

⚠️ CI failed ⚠️

The following steps have failed in CI:

  • Turbopack Rust tests (mac/win, non-blocking)

See workflow summary for details

@sokra sokra marked this pull request as ready for review May 10, 2023 10:22
@sokra sokra requested a review from a team as a code owner May 10, 2023 10:22
@sokra sokra requested a review from alexkirsz May 10, 2023 12:40
crates/turbo-tasks-fs/src/glob.rs Show resolved Hide resolved
pub async fn rename_as(self, pattern: &str) -> Result<Self> {
let mut this = self.await?.clone_value();
this.rename_as(pattern).await?;
Ok(Self::new(Value::new(this)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: We could take advantage of the new TaskInput derive to remove the Value<T> wrapper.

crates/turbopack-core/src/ident.rs Show resolved Hide resolved
crates/turbopack-core/src/ident.rs Show resolved Hide resolved
crates/turbopack-core/src/ident.rs Show resolved Hide resolved
crates/turbopack-node/js/src/ipc/evaluate.ts Show resolved Hide resolved
crates/turbopack/src/lib.rs Show resolved Hide resolved
crates/turbopack/src/module_options/mod.rs Show resolved Hide resolved
@github-actions
Copy link
Contributor

Linux Benchmark for e2d143d

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8882.22µs ± 59.06µs 8676.45µs ± 56.92µs -2.32%
bench_hmr_to_eval/Turbopack CSR/1000 modules 7941.21µs ± 57.58µs 7941.93µs ± 118.16µs +0.01%
bench_startup/Turbopack CSR/1000 modules 938.00ms ± 1.86ms 939.26ms ± 4.41ms +0.13%

@github-actions
Copy link
Contributor

MacOS Benchmark for e2d143d

Test Base PR % Significant %
bench_startup/Turbopack CSR/1000 modules 2592.75ms ± 23.87ms 2770.06ms ± 51.12ms +6.84% +1.03%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 26.20ms ± 0.11ms 26.59ms ± 0.14ms +1.52%
bench_hmr_to_eval/Turbopack CSR/1000 modules 25.60ms ± 0.17ms 26.08ms ± 0.08ms +1.85%
bench_startup/Turbopack CSR/1000 modules 2592.75ms ± 23.87ms 2770.06ms ± 51.12ms +6.84% +1.03%

crates/turbo-tasks-fs/src/glob.rs Outdated Show resolved Hide resolved
crates/turbopack/src/module_options/mod.rs Outdated Show resolved Hide resolved
@github-actions
Copy link
Contributor

Linux Benchmark for 6e887f9

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8797.54µs ± 47.74µs 8744.96µs ± 52.84µs -0.60%
bench_hmr_to_eval/Turbopack CSR/1000 modules 7866.55µs ± 62.98µs 7967.47µs ± 223.07µs +1.28%
bench_startup/Turbopack CSR/1000 modules 883.74ms ± 1.29ms 888.81ms ± 6.06ms +0.57%

@sokra sokra requested a review from jridgewell May 17, 2023 14:49
@github-actions
Copy link
Contributor

MacOS Benchmark for 6e887f9

Test Base PR % Significant %
bench_hmr_to_eval/Turbopack CSR/1000 modules 43.00ms ± 6.10ms 25.30ms ± 0.21ms -41.15% -16.48%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 22.52ms ± 1.14ms 27.46ms ± 1.45ms +21.94%
bench_hmr_to_eval/Turbopack CSR/1000 modules 43.00ms ± 6.10ms 25.30ms ± 0.21ms -41.15% -16.48%
bench_startup/Turbopack CSR/1000 modules 2526.19ms ± 28.55ms 2555.86ms ± 28.58ms +1.17%

@github-actions
Copy link
Contributor

Linux Benchmark for 04069f0

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 7718.13µs ± 66.45µs 7661.07µs ± 72.08µs -0.74%
bench_hmr_to_eval/Turbopack CSR/1000 modules 6834.43µs ± 51.03µs 7332.83µs ± 545.74µs +7.29%
bench_startup/Turbopack CSR/1000 modules 874.36ms ± 1.85ms 875.77ms ± 6.28ms +0.16%

@github-actions
Copy link
Contributor

MacOS Benchmark for 04069f0

Test Base PR % Significant %
bench_hmr_to_eval/Turbopack CSR/1000 modules 23.14ms ± 0.82ms 49.98ms ± 5.38ms +116.01% +58.30%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 25.91ms ± 0.17ms 25.80ms ± 0.19ms -0.43%
bench_hmr_to_eval/Turbopack CSR/1000 modules 23.14ms ± 0.82ms 49.98ms ± 5.38ms +116.01% +58.30%
bench_startup/Turbopack CSR/1000 modules 2932.74ms ± 179.56ms 2693.01ms ± 52.87ms -8.17%

@sokra sokra merged commit 2a0256b into main May 23, 2023
@sokra sokra deleted the sokra/advanced-rules branch May 23, 2023 11:04
sokra added a commit to vercel/next.js that referenced this pull request May 23, 2023
### What?

* allow to apply existing pipeline
* change webpack loader key to glob for more flexibility
* add test cases
* add special error message when using the old config syntax

Old config:

```js
experimental: {
 turbo: {
  loaders: {
   ".mdx": ["mdx-loader"]
  }
 }
}
```

New config

``` js
experimental: {
 turbo: {
  rules: {
   // key is a glob now
   // normal syntax will treat the result as ecmascript code
   "*.mdx": ["mdx-loader"],
   // glob allows more advanced matching
   "./images/**/*.png": {
    loaders: ["image-optimize-loader"],
    // result of loader will be handled in other steps
    // under the same name/path (here .png)
    // This will use the existing .png pipeline (static asset)
    // It might also use other rules matching .png.
    as: "*"
   },
   "*.generate-image.js": {
    loaders: ["image-generation-loader"],
    // It also possible to pass this under a different name
    // into the pipeline. Here it will treat the result as png image
    as: "*.png"
   }
  }
 }
}
```

### Why?

More flexibility and allowing to use the builtin module handling over
non-js types.

fixes WEB-1009

### Turbopack changes

* vercel/turborepo#4955 <!-- OJ Kwon -
refactor(turbopack-ecmascript): deprecate enable_emotion, enable_styled*
-->
* vercel/turborepo#4880 <!-- Tobias Koppers -
refactor webpack loader execution -->

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
hydRAnger pushed a commit to hydRAnger/next.js that referenced this pull request Jun 12, 2023
### What?

* allow to apply existing pipeline
* change webpack loader key to glob for more flexibility
* add test cases
* add special error message when using the old config syntax

Old config:

```js
experimental: {
 turbo: {
  loaders: {
   ".mdx": ["mdx-loader"]
  }
 }
}
```

New config

``` js
experimental: {
 turbo: {
  rules: {
   // key is a glob now
   // normal syntax will treat the result as ecmascript code
   "*.mdx": ["mdx-loader"],
   // glob allows more advanced matching
   "./images/**/*.png": {
    loaders: ["image-optimize-loader"],
    // result of loader will be handled in other steps
    // under the same name/path (here .png)
    // This will use the existing .png pipeline (static asset)
    // It might also use other rules matching .png.
    as: "*"
   },
   "*.generate-image.js": {
    loaders: ["image-generation-loader"],
    // It also possible to pass this under a different name
    // into the pipeline. Here it will treat the result as png image
    as: "*.png"
   }
  }
 }
}
```

### Why?

More flexibility and allowing to use the builtin module handling over
non-js types.

fixes WEB-1009

### Turbopack changes

* vercel/turborepo#4955 <!-- OJ Kwon -
refactor(turbopack-ecmascript): deprecate enable_emotion, enable_styled*
-->
* vercel/turborepo#4880 <!-- Tobias Koppers -
refactor webpack loader execution -->

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Jul 25, 2024
### Description

* [fix error propagation in
ContextCondition](vercel/turborepo@e0a8a54)
* [improve JSON deserialization error
message](vercel/turborepo@b032330)
* [fix glob
matching](vercel/turborepo@d868be4)
* allow to rename assets and apply pipeline on renamed asset

#49535
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Jul 29, 2024
### Description

* [fix error propagation in
ContextCondition](vercel/turborepo@e0a8a54)
* [improve JSON deserialization error
message](vercel/turborepo@b032330)
* [fix glob
matching](vercel/turborepo@d868be4)
* allow to rename assets and apply pipeline on renamed asset

#49535
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Aug 1, 2024
### Description

* [fix error propagation in
ContextCondition](vercel/turborepo@e0a8a54)
* [improve JSON deserialization error
message](vercel/turborepo@b032330)
* [fix glob
matching](vercel/turborepo@d868be4)
* allow to rename assets and apply pipeline on renamed asset

#49535
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants