Skip to content

Commit

Permalink
docs(benchmark): update benchmark
Browse files Browse the repository at this point in the history
  • Loading branch information
@unadlib
unadlib committed Sep 22, 2024
1 parent ac061f2 commit 4e08b31
Show file tree
Hide file tree
Showing 10 changed files with 60 additions and 60 deletions.
22 changes: 11 additions & 11 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -115,29 +115,29 @@ Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view sou
![Benchmark](benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,439 ops/sec ±0.65% (98 runs sampled)
Mutative - No Freeze x 6,300 ops/sec ±1.19% (94 runs sampled)
Immer - No Freeze x 5.26 ops/sec ±0.59% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,486 ops/sec ±0.85% (98 runs sampled)
Mutative - No Freeze x 6,338 ops/sec ±1.14% (93 runs sampled)
Immer - No Freeze x 5.32 ops/sec ±0.43% (18 runs sampled)
Mutative - Freeze x 937 ops/sec ±1.25% (95 runs sampled)
Immer - Freeze x 378 ops/sec ±0.66% (93 runs sampled)
Mutative - Freeze x 969 ops/sec ±1.02% (98 runs sampled)
Immer - Freeze x 383 ops/sec ±0.44% (94 runs sampled)
Mutative - Patches and No Freeze x 975 ops/sec ±0.17% (99 runs sampled)
Immer - Patches and No Freeze x 5.29 ops/sec ±0.30% (18 runs sampled)
Mutative - Patches and No Freeze x 978 ops/sec ±0.22% (97 runs sampled)
Immer - Patches and No Freeze x 5.32 ops/sec ±0.54% (18 runs sampled)
Mutative - Patches and Freeze x 512 ops/sec ±0.85% (98 runs sampled)
Immer - Patches and Freeze x 278 ops/sec ±0.57% (90 runs sampled)
Mutative - Patches and Freeze x 507 ops/sec ±0.28% (97 runs sampled)
Immer - Patches and Freeze x 279 ops/sec ±0.76% (91 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.2.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,300 ops/sec`) and Immer (`378 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,338 ops/sec`) and Immer (`383 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance). Run `yarn performance` to get all the performance results locally.

Expand Down
Binary file modified benchmark.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 8 additions & 8 deletions test/performance/benchmark.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ suite
() => {
const state = create(baseState, (draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
},
{
Expand All @@ -106,7 +106,7 @@ suite
() => {
const state = produce(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
},
{
Expand All @@ -125,7 +125,7 @@ suite
baseState,
(draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
},
{
enableAutoFreeze: true,
Expand All @@ -145,7 +145,7 @@ suite
() => {
const state = produce(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
},
{
Expand All @@ -164,7 +164,7 @@ suite
baseState,
(draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
},
{
enableAutoFreeze: false,
Expand All @@ -184,7 +184,7 @@ suite
() => {
const state = produceWithPatches(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
},
{
Expand All @@ -204,7 +204,7 @@ suite
baseState,
(draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
},
{
enableAutoFreeze: true,
Expand All @@ -224,7 +224,7 @@ suite
() => {
const state = produceWithPatches(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
},
{
Expand Down
16 changes: 8 additions & 8 deletions test/performance/big-object.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ measure(
for (let i = 0; i < MAX; i++) {
const state = create(baseState, (draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
}
}
Expand All @@ -73,7 +73,7 @@ measure(
for (let i = 0; i < MAX; i++) {
const state = produce(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
}
}
Expand All @@ -90,7 +90,7 @@ measure(
baseState,
(draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
},
{
enableAutoFreeze: true,
Expand All @@ -112,7 +112,7 @@ measure(
for (let i = 0; i < MAX; i++) {
const state = produce(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
}
}
Expand All @@ -129,7 +129,7 @@ measure(
baseState,
(draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
},
{
enableAutoFreeze: false,
Expand All @@ -152,7 +152,7 @@ measure(
for (let i = 0; i < MAX; i++) {
const state = produceWithPatches(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
}
}
Expand All @@ -169,7 +169,7 @@ measure(
baseState,
(draft) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
},
{
enableAutoFreeze: true,
Expand All @@ -192,7 +192,7 @@ measure(
for (let i = 0; i < MAX; i++) {
const state = produceWithPatches(baseState, (draft: any) => {
draft.arr.push(i);
draft.map[i] = i;
draft.map[i] = { i };
});
}
}
Expand Down
Binary file modified website/blog/releases/1.0/img/benchmark.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
22 changes: 11 additions & 11 deletions website/blog/releases/1.0/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,29 +99,29 @@ Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view sou
![Benchmark](img/benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,439 ops/sec ±0.65% (98 runs sampled)
Mutative - No Freeze x 6,300 ops/sec ±1.19% (94 runs sampled)
Immer - No Freeze x 5.26 ops/sec ±0.59% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,486 ops/sec ±0.85% (98 runs sampled)
Mutative - No Freeze x 6,338 ops/sec ±1.14% (93 runs sampled)
Immer - No Freeze x 5.32 ops/sec ±0.43% (18 runs sampled)
Mutative - Freeze x 937 ops/sec ±1.25% (95 runs sampled)
Immer - Freeze x 378 ops/sec ±0.66% (93 runs sampled)
Mutative - Freeze x 969 ops/sec ±1.02% (98 runs sampled)
Immer - Freeze x 383 ops/sec ±0.44% (94 runs sampled)
Mutative - Patches and No Freeze x 975 ops/sec ±0.17% (99 runs sampled)
Immer - Patches and No Freeze x 5.29 ops/sec ±0.30% (18 runs sampled)
Mutative - Patches and No Freeze x 978 ops/sec ±0.22% (97 runs sampled)
Immer - Patches and No Freeze x 5.32 ops/sec ±0.54% (18 runs sampled)
Mutative - Patches and Freeze x 512 ops/sec ±0.85% (98 runs sampled)
Immer - Patches and Freeze x 278 ops/sec ±0.57% (90 runs sampled)
Mutative - Patches and Freeze x 507 ops/sec ±0.28% (97 runs sampled)
Immer - Patches and Freeze x 279 ops/sec ±0.76% (91 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.2.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,300 ops/sec`) and Immer (`378 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,338 ops/sec`) and Immer (`383 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance).

Expand Down
22 changes: 11 additions & 11 deletions website/docs/extra-topics/comparison-with-immer.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,29 +30,29 @@ Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view sou
![Benchmark](img/benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,439 ops/sec ±0.65% (98 runs sampled)
Mutative - No Freeze x 6,300 ops/sec ±1.19% (94 runs sampled)
Immer - No Freeze x 5.26 ops/sec ±0.59% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,486 ops/sec ±0.85% (98 runs sampled)
Mutative - No Freeze x 6,338 ops/sec ±1.14% (93 runs sampled)
Immer - No Freeze x 5.32 ops/sec ±0.43% (18 runs sampled)
Mutative - Freeze x 937 ops/sec ±1.25% (95 runs sampled)
Immer - Freeze x 378 ops/sec ±0.66% (93 runs sampled)
Mutative - Freeze x 969 ops/sec ±1.02% (98 runs sampled)
Immer - Freeze x 383 ops/sec ±0.44% (94 runs sampled)
Mutative - Patches and No Freeze x 975 ops/sec ±0.17% (99 runs sampled)
Immer - Patches and No Freeze x 5.29 ops/sec ±0.30% (18 runs sampled)
Mutative - Patches and No Freeze x 978 ops/sec ±0.22% (97 runs sampled)
Immer - Patches and No Freeze x 5.32 ops/sec ±0.54% (18 runs sampled)
Mutative - Patches and Freeze x 512 ops/sec ±0.85% (98 runs sampled)
Immer - Patches and Freeze x 278 ops/sec ±0.57% (90 runs sampled)
Mutative - Patches and Freeze x 507 ops/sec ±0.28% (97 runs sampled)
Immer - Patches and Freeze x 279 ops/sec ±0.76% (91 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.2.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,300 ops/sec`) and Immer (`378 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,338 ops/sec`) and Immer (`383 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance). Run `yarn performance` to get all the performance results locally.

Binary file modified website/docs/extra-topics/img/benchmark.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/getting-started/img/benchmark.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
22 changes: 11 additions & 11 deletions website/docs/getting-started/performance.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,29 +87,29 @@ Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view sou
![Benchmark](img/benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,439 ops/sec ±0.65% (98 runs sampled)
Mutative - No Freeze x 6,300 ops/sec ±1.19% (94 runs sampled)
Immer - No Freeze x 5.26 ops/sec ±0.59% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,486 ops/sec ±0.85% (98 runs sampled)
Mutative - No Freeze x 6,338 ops/sec ±1.14% (93 runs sampled)
Immer - No Freeze x 5.32 ops/sec ±0.43% (18 runs sampled)
Mutative - Freeze x 937 ops/sec ±1.25% (95 runs sampled)
Immer - Freeze x 378 ops/sec ±0.66% (93 runs sampled)
Mutative - Freeze x 969 ops/sec ±1.02% (98 runs sampled)
Immer - Freeze x 383 ops/sec ±0.44% (94 runs sampled)
Mutative - Patches and No Freeze x 975 ops/sec ±0.17% (99 runs sampled)
Immer - Patches and No Freeze x 5.29 ops/sec ±0.30% (18 runs sampled)
Mutative - Patches and No Freeze x 978 ops/sec ±0.22% (97 runs sampled)
Immer - Patches and No Freeze x 5.32 ops/sec ±0.54% (18 runs sampled)
Mutative - Patches and Freeze x 512 ops/sec ±0.85% (98 runs sampled)
Immer - Patches and Freeze x 278 ops/sec ±0.57% (90 runs sampled)
Mutative - Patches and Freeze x 507 ops/sec ±0.28% (97 runs sampled)
Immer - Patches and Freeze x 279 ops/sec ±0.76% (91 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.2.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,300 ops/sec`) and Immer (`378 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,338 ops/sec`) and Immer (`383 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance). Run `yarn performance` to get all the performance results locally.

Expand Down

0 comments on commit 4e08b31

Please sign in to comment.