Skip to content

Commit

Permalink
docs(benchmark): update benchmark
Browse files Browse the repository at this point in the history
  • Loading branch information
unadlib committed Nov 22, 2024
1 parent 93159a8 commit 9410f7d
Show file tree
Hide file tree
Showing 8 changed files with 48 additions and 48 deletions.
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,34 +109,34 @@ Mutative is up to 6x faster than naive handcrafted reducer for updating immutabl

> Mutative passed all of Immer's test cases.
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.0.11 vs Immer v10.1.1]
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.1.0 vs Immer v10.1.1]

![Benchmark](benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,469 ops/sec ±0.82% (99 runs sampled)
Mutative - No Freeze x 6,015 ops/sec ±1.17% (94 runs sampled)
Immer - No Freeze x 5.36 ops/sec ±0.28% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,670 ops/sec ±0.64% (96 runs sampled)
Mutative - No Freeze x 6,747 ops/sec ±0.61% (95 runs sampled)
Immer - No Freeze x 5.65 ops/sec ±1.53% (19 runs sampled)
Mutative - Freeze x 959 ops/sec ±0.90% (99 runs sampled)
Immer - Freeze x 382 ops/sec ±0.53% (94 runs sampled)
Mutative - Freeze x 1,062 ops/sec ±0.74% (95 runs sampled)
Immer - Freeze x 394 ops/sec ±0.85% (93 runs sampled)
Mutative - Patches and No Freeze x 970 ops/sec ±0.99% (96 runs sampled)
Immer - Patches and No Freeze x 5.22 ops/sec ±0.80% (18 runs sampled)
Mutative - Patches and No Freeze x 1,011 ops/sec ±0.24% (98 runs sampled)
Immer - Patches and No Freeze x 5.64 ops/sec ±0.22% (19 runs sampled)
Mutative - Patches and Freeze x 505 ops/sec ±0.85% (93 runs sampled)
Immer - Patches and Freeze x 275 ops/sec ±0.49% (89 runs sampled)
Mutative - Patches and Freeze x 545 ops/sec ±1.19% (94 runs sampled)
Immer - Patches and Freeze x 215 ops/sec ±0.70% (86 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.7, CPU: Apple M1 Max, Node.js: v22.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,015 ops/sec`) and Immer (`382 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 17x performance gap between Mutative (`6,747 ops/sec`) and Immer (`394 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance). Run `yarn performance` to get all the performance results locally.

Expand Down
Binary file modified benchmark.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/blog/releases/1.0/img/benchmark.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 12 additions & 12 deletions website/blog/releases/1.0/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,34 +94,34 @@ const state = create(baseState, (draft) => {

> Mutative passed all of Immer's test cases.
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.0.11 vs Immer v10.1.1]
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.1.0 vs Immer v10.1.1]

![Benchmark](img/benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,469 ops/sec ±0.82% (99 runs sampled)
Mutative - No Freeze x 6,015 ops/sec ±1.17% (94 runs sampled)
Immer - No Freeze x 5.36 ops/sec ±0.28% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,670 ops/sec ±0.64% (96 runs sampled)
Mutative - No Freeze x 6,747 ops/sec ±0.61% (95 runs sampled)
Immer - No Freeze x 5.65 ops/sec ±1.53% (19 runs sampled)
Mutative - Freeze x 959 ops/sec ±0.90% (99 runs sampled)
Immer - Freeze x 382 ops/sec ±0.53% (94 runs sampled)
Mutative - Freeze x 1,062 ops/sec ±0.74% (95 runs sampled)
Immer - Freeze x 394 ops/sec ±0.85% (93 runs sampled)
Mutative - Patches and No Freeze x 970 ops/sec ±0.99% (96 runs sampled)
Immer - Patches and No Freeze x 5.22 ops/sec ±0.80% (18 runs sampled)
Mutative - Patches and No Freeze x 1,011 ops/sec ±0.24% (98 runs sampled)
Immer - Patches and No Freeze x 5.64 ops/sec ±0.22% (19 runs sampled)
Mutative - Patches and Freeze x 505 ops/sec ±0.85% (93 runs sampled)
Immer - Patches and Freeze x 275 ops/sec ±0.49% (89 runs sampled)
Mutative - Patches and Freeze x 545 ops/sec ±1.19% (94 runs sampled)
Immer - Patches and Freeze x 215 ops/sec ±0.70% (86 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.7, CPU: Apple M1 Max, Node.js: v22.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,015 ops/sec`) and Immer (`382 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 17x performance gap between Mutative (`6,747 ops/sec`) and Immer (`394 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance).

Expand Down
24 changes: 12 additions & 12 deletions website/docs/extra-topics/comparison-with-immer.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,33 +26,33 @@ Mutative has fewer bugs such as accidental draft escapes than Immer, [view detai

> Mutative passed all of Immer's test cases.
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.0.11 vs Immer v10.1.1]
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.1.0 vs Immer v10.1.1]

![Benchmark](img/benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,469 ops/sec ±0.82% (99 runs sampled)
Mutative - No Freeze x 6,015 ops/sec ±1.17% (94 runs sampled)
Immer - No Freeze x 5.36 ops/sec ±0.28% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,670 ops/sec ±0.64% (96 runs sampled)
Mutative - No Freeze x 6,747 ops/sec ±0.61% (95 runs sampled)
Immer - No Freeze x 5.65 ops/sec ±1.53% (19 runs sampled)
Mutative - Freeze x 959 ops/sec ±0.90% (99 runs sampled)
Immer - Freeze x 382 ops/sec ±0.53% (94 runs sampled)
Mutative - Freeze x 1,062 ops/sec ±0.74% (95 runs sampled)
Immer - Freeze x 394 ops/sec ±0.85% (93 runs sampled)
Mutative - Patches and No Freeze x 970 ops/sec ±0.99% (96 runs sampled)
Immer - Patches and No Freeze x 5.22 ops/sec ±0.80% (18 runs sampled)
Mutative - Patches and No Freeze x 1,011 ops/sec ±0.24% (98 runs sampled)
Immer - Patches and No Freeze x 5.64 ops/sec ±0.22% (19 runs sampled)
Mutative - Patches and Freeze x 505 ops/sec ±0.85% (93 runs sampled)
Immer - Patches and Freeze x 275 ops/sec ±0.49% (89 runs sampled)
Mutative - Patches and Freeze x 545 ops/sec ±1.19% (94 runs sampled)
Immer - Patches and Freeze x 215 ops/sec ±0.70% (86 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.7, CPU: Apple M1 Max, Node.js: v22.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,015 ops/sec`) and Immer (`382 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 17x performance gap between Mutative (`6,747 ops/sec`) and Immer (`394 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance). Run `yarn performance` to get all the performance results locally.
Binary file modified website/docs/extra-topics/img/benchmark.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/getting-started/img/benchmark.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 12 additions & 12 deletions website/docs/getting-started/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,34 +82,34 @@ const state = create(baseState, (draft) => {

> Mutative passed all of Immer's test cases.
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.0.11 vs Immer v10.1.1]
Measure(ops/sec) to update 50K arrays and 1K objects, bigger is better([view source](https://github.com/unadlib/mutative/blob/main/test/performance/benchmark.ts)). [Mutative v1.1.0 vs Immer v10.1.1]

![Benchmark](img/benchmark.jpg)

```
Naive handcrafted reducer - No Freeze x 4,469 ops/sec ±0.82% (99 runs sampled)
Mutative - No Freeze x 6,015 ops/sec ±1.17% (94 runs sampled)
Immer - No Freeze x 5.36 ops/sec ±0.28% (18 runs sampled)
Naive handcrafted reducer - No Freeze x 4,670 ops/sec ±0.64% (96 runs sampled)
Mutative - No Freeze x 6,747 ops/sec ±0.61% (95 runs sampled)
Immer - No Freeze x 5.65 ops/sec ±1.53% (19 runs sampled)
Mutative - Freeze x 959 ops/sec ±0.90% (99 runs sampled)
Immer - Freeze x 382 ops/sec ±0.53% (94 runs sampled)
Mutative - Freeze x 1,062 ops/sec ±0.74% (95 runs sampled)
Immer - Freeze x 394 ops/sec ±0.85% (93 runs sampled)
Mutative - Patches and No Freeze x 970 ops/sec ±0.99% (96 runs sampled)
Immer - Patches and No Freeze x 5.22 ops/sec ±0.80% (18 runs sampled)
Mutative - Patches and No Freeze x 1,011 ops/sec ±0.24% (98 runs sampled)
Immer - Patches and No Freeze x 5.64 ops/sec ±0.22% (19 runs sampled)
Mutative - Patches and Freeze x 505 ops/sec ±0.85% (93 runs sampled)
Immer - Patches and Freeze x 275 ops/sec ±0.49% (89 runs sampled)
Mutative - Patches and Freeze x 545 ops/sec ±1.19% (94 runs sampled)
Immer - Patches and Freeze x 215 ops/sec ±0.70% (86 runs sampled)
The fastest method is Mutative - No Freeze
```

Run `yarn benchmark` to measure performance.

> OS: macOS 14.6.1, CPU: Apple M1 Max, Node.js: v20.11.0
> OS: macOS 14.7, CPU: Apple M1 Max, Node.js: v22.11.0
Immer relies on auto-freeze to be enabled, if auto-freeze is disabled, Immer will have a huge performance drop and Mutative will have a huge performance lead, especially with large data structures it will have a performance lead of more than 50x.

So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 16x performance gap between Mutative (`6,015 ops/sec`) and Immer (`382 ops/sec`).
So if you are using Immer, you will have to enable auto-freeze for performance. Mutative is disabled auto-freeze by default. With the default configuration of both, we can see the 17x performance gap between Mutative (`6,747 ops/sec`) and Immer (`394 ops/sec`).

Overall, Mutative has a huge performance lead over Immer in [more performance testing scenarios](https://github.com/unadlib/mutative/tree/main/test/performance). Run `yarn performance` to get all the performance results locally.

Expand Down

0 comments on commit 9410f7d

Please sign in to comment.