Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the attributes reducer to use a map instead of a regular object #46146

Merged
merged 4 commits into from
Nov 30, 2022

Conversation

youknowriad
Copy link
Contributor

What and why?

While profiling the typing performance, I noticed that the block editor reducer is taking some milliseconds there, so I added a "performance.js" unit test to try to check how the reducer performance is impacted by a high number of blocks. With no surprise, the more blocks we have, the slower the reducer gets, so my conclusion was that since this is being called on each type synchronously, we need to make sure the reducer is as fast as possible. This PR is a first step.

How?

I noticed that the reducer time is almost entirely spent on object destructuring, so I tried several approaches here and one that seemed the most impactful was using maps instead of objects. As it stands the PR only updates the "attributes" sub reducer to use maps, but I suspect that all block related reducer would benefit from using maps.

Here are the results of my small benchmark (which I included in the commit)

  • Running the performance.js test in trunk is taking 130ms (pretty consistent)
  • Running the performance.js test in this branch is taking 65ms (consistent)

While I'm not sure how much impact this will have on the typing performance (we'll see), I believe we should probably update all these blocksByClientId reducer states to use maps instead of objects.

@youknowriad youknowriad added the [Type] Performance Related to performance efforts label Nov 29, 2022
@youknowriad youknowriad requested review from jsnajdr, tyxla and a team November 29, 2022 09:49
@youknowriad youknowriad self-assigned this Nov 29, 2022
@github-actions
Copy link

github-actions bot commented Nov 29, 2022

Size Change: +1.51 kB (0%)

Total Size: 1.32 MB

Filename Size Change
build/block-editor/index.min.js 180 kB +1.23 kB (+1%)
build/block-editor/style-rtl.css 16.5 kB -6 B (0%)
build/block-editor/style.css 16.5 kB -6 B (0%)
build/block-library/index.min.js 195 kB +184 B (0%)
build/components/index.min.js 203 kB +2 B (0%)
build/edit-navigation/index.min.js 16.2 kB +19 B (0%)
build/edit-site/index.min.js 61.2 kB +73 B (0%)
build/edit-site/style-rtl.css 8.52 kB +7 B (0%)
build/edit-site/style.css 8.51 kB +8 B (0%)
ℹ️ View Unchanged
Filename Size
build/a11y/index.min.js 993 B
build/annotations/index.min.js 2.78 kB
build/api-fetch/index.min.js 2.27 kB
build/autop/index.min.js 2.15 kB
build/blob/index.min.js 487 B
build/block-directory/index.min.js 7.16 kB
build/block-directory/style-rtl.css 1.03 kB
build/block-directory/style.css 1.04 kB
build/block-editor/default-editor-styles-rtl.css 401 B
build/block-editor/default-editor-styles.css 401 B
build/block-library/blocks/archives/editor-rtl.css 107 B
build/block-library/blocks/archives/editor.css 106 B
build/block-library/blocks/archives/style-rtl.css 129 B
build/block-library/blocks/archives/style.css 129 B
build/block-library/blocks/audio/editor-rtl.css 185 B
build/block-library/blocks/audio/editor.css 185 B
build/block-library/blocks/audio/style-rtl.css 158 B
build/block-library/blocks/audio/style.css 158 B
build/block-library/blocks/audio/theme-rtl.css 172 B
build/block-library/blocks/audio/theme.css 172 B
build/block-library/blocks/avatar/editor-rtl.css 154 B
build/block-library/blocks/avatar/editor.css 154 B
build/block-library/blocks/avatar/style-rtl.css 126 B
build/block-library/blocks/avatar/style.css 126 B
build/block-library/blocks/block/editor-rtl.css 338 B
build/block-library/blocks/block/editor.css 338 B
build/block-library/blocks/button/editor-rtl.css 517 B
build/block-library/blocks/button/editor.css 517 B
build/block-library/blocks/button/style-rtl.css 566 B
build/block-library/blocks/button/style.css 566 B
build/block-library/blocks/buttons/editor-rtl.css 373 B
build/block-library/blocks/buttons/editor.css 373 B
build/block-library/blocks/buttons/style-rtl.css 368 B
build/block-library/blocks/buttons/style.css 368 B
build/block-library/blocks/calendar/style-rtl.css 270 B
build/block-library/blocks/calendar/style.css 270 B
build/block-library/blocks/categories/editor-rtl.css 125 B
build/block-library/blocks/categories/editor.css 124 B
build/block-library/blocks/categories/style-rtl.css 138 B
build/block-library/blocks/categories/style.css 138 B
build/block-library/blocks/code/editor-rtl.css 102 B
build/block-library/blocks/code/editor.css 102 B
build/block-library/blocks/code/style-rtl.css 159 B
build/block-library/blocks/code/style.css 159 B
build/block-library/blocks/code/theme-rtl.css 160 B
build/block-library/blocks/code/theme.css 160 B
build/block-library/blocks/columns/editor-rtl.css 147 B
build/block-library/blocks/columns/editor.css 147 B
build/block-library/blocks/columns/style-rtl.css 442 B
build/block-library/blocks/columns/style.css 442 B
build/block-library/blocks/comment-author-avatar/editor-rtl.css 163 B
build/block-library/blocks/comment-author-avatar/editor.css 163 B
build/block-library/blocks/comment-content/style-rtl.css 134 B
build/block-library/blocks/comment-content/style.css 134 B
build/block-library/blocks/comment-template/style-rtl.css 237 B
build/block-library/blocks/comment-template/style.css 236 B
build/block-library/blocks/comments-pagination-numbers/editor-rtl.css 159 B
build/block-library/blocks/comments-pagination-numbers/editor.css 157 B
build/block-library/blocks/comments-pagination/editor-rtl.css 258 B
build/block-library/blocks/comments-pagination/editor.css 249 B
build/block-library/blocks/comments-pagination/style-rtl.css 272 B
build/block-library/blocks/comments-pagination/style.css 268 B
build/block-library/blocks/comments-title/editor-rtl.css 118 B
build/block-library/blocks/comments-title/editor.css 118 B
build/block-library/blocks/comments/editor-rtl.css 875 B
build/block-library/blocks/comments/editor.css 874 B
build/block-library/blocks/comments/style-rtl.css 672 B
build/block-library/blocks/comments/style.css 671 B
build/block-library/blocks/cover/editor-rtl.css 646 B
build/block-library/blocks/cover/editor.css 647 B
build/block-library/blocks/cover/style-rtl.css 1.61 kB
build/block-library/blocks/cover/style.css 1.6 kB
build/block-library/blocks/embed/editor-rtl.css 327 B
build/block-library/blocks/embed/editor.css 327 B
build/block-library/blocks/embed/style-rtl.css 446 B
build/block-library/blocks/embed/style.css 446 B
build/block-library/blocks/embed/theme-rtl.css 172 B
build/block-library/blocks/embed/theme.css 172 B
build/block-library/blocks/file/editor-rtl.css 335 B
build/block-library/blocks/file/editor.css 335 B
build/block-library/blocks/file/style-rtl.css 288 B
build/block-library/blocks/file/style.css 288 B
build/block-library/blocks/file/view.min.js 353 B
build/block-library/blocks/freeform/editor-rtl.css 2.47 kB
build/block-library/blocks/freeform/editor.css 2.47 kB
build/block-library/blocks/gallery/editor-rtl.css 987 B
build/block-library/blocks/gallery/editor.css 993 B
build/block-library/blocks/gallery/style-rtl.css 1.58 kB
build/block-library/blocks/gallery/style.css 1.58 kB
build/block-library/blocks/gallery/theme-rtl.css 157 B
build/block-library/blocks/gallery/theme.css 157 B
build/block-library/blocks/group/editor-rtl.css 687 B
build/block-library/blocks/group/editor.css 687 B
build/block-library/blocks/group/style-rtl.css 105 B
build/block-library/blocks/group/style.css 105 B
build/block-library/blocks/group/theme-rtl.css 125 B
build/block-library/blocks/group/theme.css 125 B
build/block-library/blocks/heading/style-rtl.css 128 B
build/block-library/blocks/heading/style.css 128 B
build/block-library/blocks/html/editor-rtl.css 365 B
build/block-library/blocks/html/editor.css 366 B
build/block-library/blocks/image/editor-rtl.css 912 B
build/block-library/blocks/image/editor.css 912 B
build/block-library/blocks/image/style-rtl.css 662 B
build/block-library/blocks/image/style.css 666 B
build/block-library/blocks/image/theme-rtl.css 172 B
build/block-library/blocks/image/theme.css 172 B
build/block-library/blocks/latest-comments/style-rtl.css 333 B
build/block-library/blocks/latest-comments/style.css 333 B
build/block-library/blocks/latest-posts/editor-rtl.css 250 B
build/block-library/blocks/latest-posts/editor.css 249 B
build/block-library/blocks/latest-posts/style-rtl.css 514 B
build/block-library/blocks/latest-posts/style.css 514 B
build/block-library/blocks/list/style-rtl.css 135 B
build/block-library/blocks/list/style.css 135 B
build/block-library/blocks/media-text/editor-rtl.css 300 B
build/block-library/blocks/media-text/editor.css 298 B
build/block-library/blocks/media-text/style-rtl.css 540 B
build/block-library/blocks/media-text/style.css 539 B
build/block-library/blocks/more/editor-rtl.css 465 B
build/block-library/blocks/more/editor.css 465 B
build/block-library/blocks/navigation-link/editor-rtl.css 746 B
build/block-library/blocks/navigation-link/editor.css 744 B
build/block-library/blocks/navigation-link/style-rtl.css 153 B
build/block-library/blocks/navigation-link/style.css 153 B
build/block-library/blocks/navigation-submenu/editor-rtl.css 333 B
build/block-library/blocks/navigation-submenu/editor.css 333 B
build/block-library/blocks/navigation/editor-rtl.css 2.19 kB
build/block-library/blocks/navigation/editor.css 2.19 kB
build/block-library/blocks/navigation/style-rtl.css 2.26 kB
build/block-library/blocks/navigation/style.css 2.25 kB
build/block-library/blocks/navigation/view-modal.min.js 2.81 kB
build/block-library/blocks/navigation/view.min.js 447 B
build/block-library/blocks/nextpage/editor-rtl.css 428 B
build/block-library/blocks/nextpage/editor.css 428 B
build/block-library/blocks/page-list/editor-rtl.css 397 B
build/block-library/blocks/page-list/editor.css 398 B
build/block-library/blocks/page-list/style-rtl.css 212 B
build/block-library/blocks/page-list/style.css 212 B
build/block-library/blocks/paragraph/editor-rtl.css 214 B
build/block-library/blocks/paragraph/editor.css 214 B
build/block-library/blocks/paragraph/style-rtl.css 321 B
build/block-library/blocks/paragraph/style.css 321 B
build/block-library/blocks/post-author/style-rtl.css 212 B
build/block-library/blocks/post-author/style.css 212 B
build/block-library/blocks/post-comments-form/editor-rtl.css 137 B
build/block-library/blocks/post-comments-form/editor.css 137 B
build/block-library/blocks/post-comments-form/style-rtl.css 536 B
build/block-library/blocks/post-comments-form/style.css 537 B
build/block-library/blocks/post-date/style-rtl.css 107 B
build/block-library/blocks/post-date/style.css 107 B
build/block-library/blocks/post-excerpt/editor-rtl.css 119 B
build/block-library/blocks/post-excerpt/editor.css 119 B
build/block-library/blocks/post-excerpt/style-rtl.css 116 B
build/block-library/blocks/post-excerpt/style.css 116 B
build/block-library/blocks/post-featured-image/editor-rtl.css 620 B
build/block-library/blocks/post-featured-image/editor.css 618 B
build/block-library/blocks/post-featured-image/style-rtl.css 349 B
build/block-library/blocks/post-featured-image/style.css 349 B
build/block-library/blocks/post-navigation-link/style-rtl.css 190 B
build/block-library/blocks/post-navigation-link/style.css 189 B
build/block-library/blocks/post-template/editor-rtl.css 140 B
build/block-library/blocks/post-template/editor.css 139 B
build/block-library/blocks/post-template/style-rtl.css 317 B
build/block-library/blocks/post-template/style.css 317 B
build/block-library/blocks/post-terms/style-rtl.css 136 B
build/block-library/blocks/post-terms/style.css 136 B
build/block-library/blocks/post-title/style-rtl.css 138 B
build/block-library/blocks/post-title/style.css 138 B
build/block-library/blocks/preformatted/style-rtl.css 139 B
build/block-library/blocks/preformatted/style.css 139 B
build/block-library/blocks/pullquote/editor-rtl.css 170 B
build/block-library/blocks/pullquote/editor.css 170 B
build/block-library/blocks/pullquote/style-rtl.css 357 B
build/block-library/blocks/pullquote/style.css 357 B
build/block-library/blocks/pullquote/theme-rtl.css 201 B
build/block-library/blocks/pullquote/theme.css 201 B
build/block-library/blocks/query-pagination-numbers/editor-rtl.css 158 B
build/block-library/blocks/query-pagination-numbers/editor.css 156 B
build/block-library/blocks/query-pagination/editor-rtl.css 258 B
build/block-library/blocks/query-pagination/editor.css 247 B
build/block-library/blocks/query-pagination/style-rtl.css 326 B
build/block-library/blocks/query-pagination/style.css 322 B
build/block-library/blocks/query-title/style-rtl.css 108 B
build/block-library/blocks/query-title/style.css 108 B
build/block-library/blocks/query/editor-rtl.css 475 B
build/block-library/blocks/query/editor.css 477 B
build/block-library/blocks/quote/style-rtl.css 253 B
build/block-library/blocks/quote/style.css 253 B
build/block-library/blocks/quote/theme-rtl.css 255 B
build/block-library/blocks/quote/theme.css 259 B
build/block-library/blocks/read-more/style-rtl.css 168 B
build/block-library/blocks/read-more/style.css 168 B
build/block-library/blocks/rss/editor-rtl.css 239 B
build/block-library/blocks/rss/editor.css 240 B
build/block-library/blocks/rss/style-rtl.css 323 B
build/block-library/blocks/rss/style.css 323 B
build/block-library/blocks/search/editor-rtl.css 205 B
build/block-library/blocks/search/editor.css 205 B
build/block-library/blocks/search/style-rtl.css 441 B
build/block-library/blocks/search/style.css 439 B
build/block-library/blocks/search/theme-rtl.css 149 B
build/block-library/blocks/search/theme.css 149 B
build/block-library/blocks/separator/editor-rtl.css 184 B
build/block-library/blocks/separator/editor.css 184 B
build/block-library/blocks/separator/style-rtl.css 269 B
build/block-library/blocks/separator/style.css 269 B
build/block-library/blocks/separator/theme-rtl.css 229 B
build/block-library/blocks/separator/theme.css 229 B
build/block-library/blocks/shortcode/editor-rtl.css 508 B
build/block-library/blocks/shortcode/editor.css 508 B
build/block-library/blocks/site-logo/editor-rtl.css 522 B
build/block-library/blocks/site-logo/editor.css 522 B
build/block-library/blocks/site-logo/style-rtl.css 238 B
build/block-library/blocks/site-logo/style.css 238 B
build/block-library/blocks/site-tagline/editor-rtl.css 129 B
build/block-library/blocks/site-tagline/editor.css 129 B
build/block-library/blocks/site-title/editor-rtl.css 155 B
build/block-library/blocks/site-title/editor.css 155 B
build/block-library/blocks/site-title/style-rtl.css 101 B
build/block-library/blocks/site-title/style.css 101 B
build/block-library/blocks/social-link/editor-rtl.css 219 B
build/block-library/blocks/social-link/editor.css 219 B
build/block-library/blocks/social-links/editor-rtl.css 709 B
build/block-library/blocks/social-links/editor.css 708 B
build/block-library/blocks/social-links/style-rtl.css 1.43 kB
build/block-library/blocks/social-links/style.css 1.43 kB
build/block-library/blocks/spacer/editor-rtl.css 372 B
build/block-library/blocks/spacer/editor.css 372 B
build/block-library/blocks/spacer/style-rtl.css 96 B
build/block-library/blocks/spacer/style.css 96 B
build/block-library/blocks/table/editor-rtl.css 547 B
build/block-library/blocks/table/editor.css 547 B
build/block-library/blocks/table/style-rtl.css 670 B
build/block-library/blocks/table/style.css 669 B
build/block-library/blocks/table/theme-rtl.css 220 B
build/block-library/blocks/table/theme.css 220 B
build/block-library/blocks/tag-cloud/style-rtl.css 287 B
build/block-library/blocks/tag-cloud/style.css 288 B
build/block-library/blocks/template-part/editor-rtl.css 436 B
build/block-library/blocks/template-part/editor.css 436 B
build/block-library/blocks/template-part/theme-rtl.css 139 B
build/block-library/blocks/template-part/theme.css 139 B
build/block-library/blocks/text-columns/editor-rtl.css 135 B
build/block-library/blocks/text-columns/editor.css 135 B
build/block-library/blocks/text-columns/style-rtl.css 198 B
build/block-library/blocks/text-columns/style.css 198 B
build/block-library/blocks/verse/style-rtl.css 130 B
build/block-library/blocks/verse/style.css 130 B
build/block-library/blocks/video/editor-rtl.css 720 B
build/block-library/blocks/video/editor.css 723 B
build/block-library/blocks/video/style-rtl.css 218 B
build/block-library/blocks/video/style.css 218 B
build/block-library/blocks/video/theme-rtl.css 171 B
build/block-library/blocks/video/theme.css 171 B
build/block-library/classic-rtl.css 193 B
build/block-library/classic.css 193 B
build/block-library/common-rtl.css 1.05 kB
build/block-library/common.css 1.05 kB
build/block-library/editor-elements-rtl.css 126 B
build/block-library/editor-elements.css 126 B
build/block-library/editor-rtl.css 11.7 kB
build/block-library/editor.css 11.7 kB
build/block-library/elements-rtl.css 105 B
build/block-library/elements.css 105 B
build/block-library/reset-rtl.css 514 B
build/block-library/reset.css 514 B
build/block-library/style-rtl.css 12.4 kB
build/block-library/style.css 12.4 kB
build/block-library/theme-rtl.css 749 B
build/block-library/theme.css 753 B
build/block-serialization-default-parser/index.min.js 1.13 kB
build/block-serialization-spec-parser/index.min.js 2.83 kB
build/blocks/index.min.js 50 kB
build/components/style-rtl.css 11.6 kB
build/components/style.css 11.6 kB
build/compose/index.min.js 12.3 kB
build/core-data/index.min.js 15.6 kB
build/customize-widgets/index.min.js 11.3 kB
build/customize-widgets/style-rtl.css 1.41 kB
build/customize-widgets/style.css 1.41 kB
build/data-controls/index.min.js 663 B
build/data/index.min.js 8.12 kB
build/date/index.min.js 32.1 kB
build/deprecated/index.min.js 518 B
build/dom-ready/index.min.js 336 B
build/dom/index.min.js 4.74 kB
build/edit-navigation/style-rtl.css 4.1 kB
build/edit-navigation/style.css 4.1 kB
build/edit-post/classic-rtl.css 569 B
build/edit-post/classic.css 570 B
build/edit-post/index.min.js 34.5 kB
build/edit-post/style-rtl.css 7.45 kB
build/edit-post/style.css 7.44 kB
build/edit-widgets/index.min.js 16.8 kB
build/edit-widgets/style-rtl.css 4.46 kB
build/edit-widgets/style.css 4.46 kB
build/editor/index.min.js 44 kB
build/editor/style-rtl.css 3.65 kB
build/editor/style.css 3.64 kB
build/element/index.min.js 4.72 kB
build/escape-html/index.min.js 548 B
build/experiments/index.min.js 882 B
build/format-library/index.min.js 6.96 kB
build/format-library/style-rtl.css 596 B
build/format-library/style.css 596 B
build/hooks/index.min.js 1.66 kB
build/html-entities/index.min.js 454 B
build/i18n/index.min.js 3.79 kB
build/is-shallow-equal/index.min.js 535 B
build/keyboard-shortcuts/index.min.js 1.79 kB
build/keycodes/index.min.js 1.86 kB
build/list-reusable-blocks/index.min.js 2.13 kB
build/list-reusable-blocks/style-rtl.css 858 B
build/list-reusable-blocks/style.css 857 B
build/media-utils/index.min.js 2.94 kB
build/notices/index.min.js 977 B
build/nux/index.min.js 2.07 kB
build/nux/style-rtl.css 772 B
build/nux/style.css 768 B
build/plugins/index.min.js 1.95 kB
build/preferences-persistence/index.min.js 2.23 kB
build/preferences/index.min.js 1.35 kB
build/primitives/index.min.js 960 B
build/priority-queue/index.min.js 1.59 kB
build/react-i18n/index.min.js 702 B
build/react-refresh-entry/index.min.js 8.44 kB
build/react-refresh-runtime/index.min.js 7.31 kB
build/redux-routine/index.min.js 2.75 kB
build/reusable-blocks/index.min.js 2.22 kB
build/reusable-blocks/style-rtl.css 281 B
build/reusable-blocks/style.css 281 B
build/rich-text/index.min.js 10.7 kB
build/server-side-render/index.min.js 1.77 kB
build/shortcode/index.min.js 1.52 kB
build/style-engine/index.min.js 1.51 kB
build/token-list/index.min.js 650 B
build/url/index.min.js 3.7 kB
build/vendors/inert-polyfill.min.js 2.48 kB
build/vendors/react-dom.min.js 38.5 kB
build/vendors/react.min.js 4.34 kB
build/viewport/index.min.js 1.09 kB
build/warning/index.min.js 280 B
build/widgets/index.min.js 7.23 kB
build/widgets/style-rtl.css 1.21 kB
build/widgets/style.css 1.21 kB
build/wordcount/index.min.js 1.06 kB

compressed-size-action

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

The performance test constructs a map (with two implementations, old and new) with 100k entries and then performs UPDATE_BLOCK_ATTRIBUTES on it, updating exactly one block. Is that the important operation we are speeding up here?

The most expensive operation seems to be copying the map. Old way is:

const newState = { ...oldState };
newState[ clientId ] = attributes;

The new way is:

const newState = new Map( oldState );
newState.set( clientId, attributes );

The performance results say that copying the Map is two times faster.

Wondering about two things:

  • is the tested { ...oldState } spread operator really running natively, or is it transpiled by Babel?
  • could we use some immutable implementation of Map that doesn't need to copy the entire thing, and can do immutable updates with some internal branching, sharing the unchanged parts? I think Immutable.js used to implement that.

@youknowriad
Copy link
Contributor Author

The performance test constructs a map (with two implementations, old and new) with 100k entries and then performs UPDATE_BLOCK_ATTRIBUTES on it, updating exactly one block. Is that the important operation we are speeding up here?

Yes, because it's the actual action that is triggered when you type and it's the most important metric for us.

is the tested { ...oldState } spread operator really running natively, or is it transpiled by Babel?

I was wondering the same, I'd assume right now we only support browsers that support this natively but I may be wrong. I'll see if I can spot it in the built files.

could we use some immutable implementation of Map that doesn't need to copy the entire thing, and can do immutable updates with some internal branching, sharing the unchanged parts? I think Immutable.js used to implement that.

Potentially, this sounds like something that needs a dedicated library. Immutable.js looks too heavy to me. Do you have any suggestions here? I tried "Immer" as it's also a popular library in Redux land, but it turned out be way less performant.

@youknowriad
Copy link
Contributor Author

I was wondering the same, I'd assume right now we only support browsers that support this natively but I may be wrong. I'll see if I can spot it in the built files.

Confirmed this, it's not transpiled, it uses native spreading in built files. (I'm assuming that jest is using the same config though)

@youknowriad
Copy link
Contributor Author

I did the same test again with 2000 blocks (an example that is more likely to be found in real life). Here are the results:

  • using objects: 24ms
  • using Map: 2ms

@youknowriad
Copy link
Contributor Author

Here's a simpler benchmark on JSBench

Screenshot 2022-11-29 at 12 20 32 PM

@youknowriad
Copy link
Contributor Author

I'm having trouble understanding why this PR is breaking the mobile unit test. cc @geriux @fluiddot maybe

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

I tried to do a local benchmark in plain Node.js with no Jest and no potential transpilation, and the results are interesting: native object spread is (by far) the fastest, map is in the middle, and Object.assign is slowest.

This is the test script:

const k = (i) => i + "";
const v = (i) => ({ content: "post " + i });

function test(body) {
  let value;
  const s = new Date();
  for (let j = 0; j < 4000; j++) {
    value = body(j, value);
  }
  const e = new Date();
  return e - s;
}

function objAssign(j, o = {}) {
  o = Object.assign({}, o);
  o[k(j)] = v(j);
  return o;
}

function objSpread(j, o = {}) {
  o = { ...o };
  o[k(j)] = v(j);
  return o;
}

function mapCopies(j, m = new Map()) {
  m = new Map(m);
  m.set(k(j), v(j));
  return m;
}

for (let k = 0; k < 3; k++) {
  console.log("obj assign", test(objAssign));
  console.log("obj spread", test(objSpread));
  console.log("map copies", test(mapCopies));
  console.log();
}

and the output:

obj assign 1339
obj spread 9
map copies 391

obj assign 1276
obj spread 7
map copies 386

obj assign 1279
obj spread 6
map copies 382

@tyxla
Copy link
Member

tyxla commented Nov 29, 2022

Summoning the wisdom of @sgomes because if I recall correctly, he had an opinion about that kind of benchmarking 😉

@youknowriad
Copy link
Contributor Author

youknowriad commented Nov 29, 2022

@jsnajdr I think the test you wrote is slightly incorrect in the sense that it tests both "set" and "assign" at the same going from an empty object to a full object. What we care about instead is to run the body of your test (same bodies) but after the object contains already a high number of elements. Basically, this corresponds to typing in a large post.

In other words, something like this

const k = (i) => i + "";
const v = (i) => ({ content: "post " + i });

function test(body) {
  let value;
  const object = {};
  const map = new Map();
  for (let i = 0; i < 20000; i++) {
    object[ k(i) ] = v(i);
    map.set( k(i), v(i) );
  }
  const s = new Date();
  value = body(20001, object, map);
  const e = new Date();
  return e - s;
}

function objAssign(j, object, map) {
  o = Object.assign({}, object);
  o[k(j)] = v(j);
  return o;
}

function objSpread(j, object, map) {
  o = { ...object };
  o[k(j)] = v(j);
  return o;
}

function mapCopies(j, object, map) {
  m = new Map(map);
  m.set(k(j), v(j));
  return m;
}

for (let k = 0; k < 3; k++) {
  console.log("obj assign", test(objAssign));
  console.log("obj spread", test(objSpread));
  console.log("map copies", test(mapCopies));
  console.log();
}

and here are the results I got

obj assign 4
obj spread 3
map copies 2

obj assign 5
obj spread 3
map copies 1

obj assign 3
obj spread 3
map copies 9

map is always faster aside the last attempt (I suspect something related to garbage collection but I'm not sure).

Edit: if you increase the number of attempts (from 3 to say 10), you'll see that map is always faster except maybe for one of the iterations.

@sgomes
Copy link
Contributor

sgomes commented Nov 29, 2022

Thanks, @tyxla, I do! 😄

In general, micro-benchmarks like that are not as useful as we'd like them to be for a number of reasons, but the two most important are:

  • They only reflect things as they are now. New engine versions constantly tweak things to improve performance in slower features, remove de-opts where possible, and at times completely revamp entire classes of optimisation.
  • They don't reflect real-world usage. Testing the same code and data thousands of times in a loop will trigger different optimisations than real-world usage scenarios where things aren't quite as simple.

Then there's also the important matter that we need to make sure that the thing we're testing is in fact the hot path! It seems that some due diligence has been made here, but we should probably go further and identify the slow line in a profiling run, and make sure that it's the one we expect.

Testing should always be done as close as possible to a real-world scenario, so I think the initial strategy of using a performance unit test is the best, even though it's a somewhat artificial test.

When testing A/B changes like this, however, we need to ensure that the results are statistically significant. Ideally, that means multiple runs, setting up the hypothesis that the new version is faster, and using a Welch's t-test to ensure that the p-value of that hypothesis is significant. I wrote a tool that does that for a bunch of different metrics on a page load. It's probably overkill to do the work of applying it here, but I'd be happy to help validate this fix with some perf measurements if you're interested.

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

I think the test you wrote is slightly incorrect in the sense that it tests both "set" and "assign" at the same going from an empty object to a full object.

I updated the test to create an initial object/map with 2000 entries, and then benchmark adding the 2001st entry. Your test does the addition only once, or maybe 3 or 5 times. I ran it 1000 times, always adding one entry to the original 2000-sized object. The results still favor native spread:

obj assign 348
obj spread 5
map copies 97

obj assign 329
obj spread 3
map copies 88

obj assign 328
obj spread 2
map copies 88

Check out the updated script:

const k = (i) => i + "";
const v = (i) => ({ content: "post " + i });

const initialObj = {};
const initialMap = new Map();

console.log('initializing...')
for (let i = 0; i < 2000; i++) {
  initialObj[k(i)] = v(i);
  initialMap.set(k(i), v(i));
}

function test(add, value) {
  const s = new Date();
  for (let j = 0; j < 1000; j++) {
    add(value, 2001);
  }
  const e = new Date();
  return e - s;
}

function objAssign(o, j) {
  return Object.assign({}, o, { [k(j)]: v(j) });
}

function objSpread(o, j) {
  return { ...o, [k(j)]: v(j) };
}

function mapCopies(m, j) {
  m = new Map(m);
  m.set(k(j), v(j));
  return m;
}

console.log('testing...');
for (let k = 0; k < 3; k++) {
  console.log();
  console.log("obj assign", test(objAssign, initialObj));
  console.log("obj spread", test(objSpread, initialObj));
  console.log("map copies", test(mapCopies, initialMap));
}

@youknowriad
Copy link
Contributor Author

@jsnajdr I see you're still looping 1000 times inside the test function (between the dates) so I think that's not really close to what happens when you type.

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

I see you're still looping 1000 times inside the test function

Yes, I want to take 1000 measurements and calculate their average. In every loop iteration, I'm adding one new entry to an existing object with 2000 entries. It's not like adding 1000 new entries to the same object, I'm always starting from the original object.

That's like doing 1000 keypresses when typing, isn't it? When I type, every keystroke means adding/updating just one entry (the current block) in the large map, is that right?

I want to avoid measuring a single event that takes only 1-3 ms to finish. That's too imprecise, too much noise. That's all.

@youknowriad
Copy link
Contributor Author

That's like doing 1000 keypresses when typing, isn't it? When I type, every keystroke means adding/updating just one entry (the current block) in the large map, is that right?

I understand but the way these JS engines work is weird and doing 1000 similar operations in a row make the engines trigger optimizations that favor an approach over another making it unrealistic in real life. That's why running only 1 addition is IMO more accurate.

@youknowriad
Copy link
Contributor Author

If you want bigger numbers, maybe you can use a bigger initial state (bigger array, map)

@sgomes
Copy link
Contributor

sgomes commented Nov 29, 2022

doing 1000 similar operations in a row make the engines trigger optimizations that favor an approach over another making it unrealistic in real life

This is correct. Chromium/V8, for example, has various stages of optimisation that only kick in at certain thresholds.

If the goal is to increase confidence by repeating a test, it should be done with multiple runs, rather than a loop within a single run.

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

If you want bigger numbers, maybe you can use a bigger initial state (bigger array, map)

OK, I did that, and have some interesting results 🙂 I increased the size of the initial object/map to 50000 entries, and am measuring the addition of just one 50001st entry. I also switched from new Date() to hi-res performance.now() timers. And I do each addition measurement 15 times in a row, on a new object. The loop is like:

function test(add, create) {
  const measurements = [];
  for (let i = 0; i < 15; i++) {
    const value = create();
    const s = performance.now();
    add(value, 50001);
    const e = performance.now();
    measurements.push(e - s);
  }
  return measurements;
}

The results are:

obj assign 11.7 09.4 10.3 11.1 10.2 09.7 09.6 10.1 10.1 09.9 10.7 10.4 09.7 09.7 09.6
obj spread 09.5 09.8 11.1 10.0 09.3 09.6 09.7 09.4 09.6 00.2 00.2 00.2 00.2 00.2 00.2
map copies 09.8 04.4 04.5 04.5 04.4 04.5 10.4 15.7 04.4 04.4 04.2 04.3 04.6 09.8 04.4

All three methods have approximately the same speed at the beginning, about 10ms to perform the addition. But in further iterations it gets interesting:

  • Object.assign speed doesn't change, it's 10ms the entire time
  • native spread does 9-10 iterations at the original speed, but then it gets superfast. It can add a new entry in 0.02ms!
  • Map gets better right in the second iteration, it's 2x faster, at 4ms. And it keeps there, except sometimes it randomly slows down back to ~10ms. This random variation is consistent, happens on all runs

Apparently it's the objSpread function, passed as the add parameter to test:

function objSpread(o, j) {
  return { ...o, [k(j)]: v(j) };
}

that gets optimized. What else could it be? We are creating a brand new initial object on every run.

the engines trigger optimizations that favor an approach over another making it unrealistic in real life

If it's really the spread operation that gets optimized after 10 invocations, then that's relevant for the real-life Gutenberg. The reducers are called all the time, and if they get optimized after a few runs, we are getting that improvement in real usage.

@sgomes
Copy link
Contributor

sgomes commented Nov 29, 2022

The reducers are called all the time, and if they get optimized after a few runs, we are getting that improvement in real usage.

And to test this hypothesis and get some real numbers, we could run a more real-world test using the real reducers, and actually the whole state framework, rather than a synthetic JS benchmark 🙂

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

And to test this hypothesis and get some real numbers, we could run a more real-world test using the real reducers

That's a cool idea! We could take the whole block-editor reducer, and measure dispatching UPDATE_BLOCK_ATTRIBUTES to it. That's pretty close to real life, and still fairly simple test.

A typical reducer is just a few spread operators nested in each other, so the optimization we're seeing should be preserved.

@youknowriad
Copy link
Contributor Author

That's a cool idea! We could take the whole block-editor reducer, and measure dispatching UPDATE_BLOCK_ATTRIBUTES to it. That's pretty close to real life, and still fairly simple test.

Isn't this what my test is doing?

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

Isn't this what my test is doing?

Oh yes, this test got evicted from my LRU memory cache over the course of the day 🙂

@youknowriad
Copy link
Contributor Author

youknowriad commented Nov 29, 2022

@jsnajdr the brain's micro-optimizations can be tricky :)

@jsnajdr
Copy link
Member

jsnajdr commented Nov 29, 2022

Isn't this what my test is doing?

I ran this reducer performance test 20 times in a row now, to see if repeated calls get optimized. For both versions, making sure the spread operator is native. And unfortunately the optimization is not there. Results for the original object map (20 timings in ms):

196.6 187.6 156.0 155.7 159.6 156.3 156.8 157.8 158.8 154.8 155.2 155.9 158.0 164.2 156.9 157.9 158.8 162.6 187.7 189.9

And the results for Map, which are consistently better:

108.3 102.8 101.2 105.2 101.0 107.7 104.9 102.9 100.0 104.3 100.6 99.2 101.7 100.7 98.9 102.5 101.5 99.7 107.6 99.7

@youknowriad
Copy link
Contributor Author

Thanks for checking @jsnajdr I guess we can move forward with this PR then? As a follow-up, I want to also see if we can make further improvements by updating other similar reducers to maps.

Copy link
Member

@jsnajdr jsnajdr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it's really a realistic speed improvement 🙂 :shipit:

@youknowriad youknowriad merged commit 6d11d32 into trunk Nov 30, 2022
@youknowriad youknowriad deleted the perf/block-editor-reducer branch November 30, 2022 11:17
@github-actions github-actions bot added this to the Gutenberg 14.7 milestone Nov 30, 2022
@fluiddot
Copy link
Contributor

I'm having trouble understanding why this PR is breaking the mobile unit test. cc @geriux @fluiddot maybe

Sorry for the delayed response @youknowriad. Looks like the mobile unit tests have finally succeeded 🎊 , is there anything else that we'd need to take a look at from the mobile side? Thanks!

@youknowriad
Copy link
Contributor Author

@fluiddot Thanks for chiming in, it's all good I think.

...getFlattenedBlockAttributes( action.blocks ),
};
case 'INSERT_BLOCKS': {
const newState = new Map( state );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@youknowriad I'm inclined to add a comment documenting the choice of operations for a tangible performance hot-path optimization here. It'd be easy to lose track of it in a future refactor

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are reasons why we normally don't put Maps or other non-serializable values in the Redux store. It's no longer showing the state in the Redux devtools for me after this PR.

I tried "Immer" as it's also a popular library in Redux land, but it turned out be way less performant.

I wonder if there's any benchmark on this? Seems like Immer should also be performant in most cases. Were we testing it in the production build?

(Side note: adopting Immer should also help us when we want to integrate yjs more closely into the system for collaborative editing.)

I think we should at least bring back the ability to debug values in the Redux devtools. Maybe the serialize option would help?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any specific reason why Maps should be avoided, other than Redux DevTools not working with them (which to me sounds like a Redux DevTools problem)?

The link only goes on to mention:

"It also ensures that the UI will update as expected."

I'm not sure which UI this refers to, nor why it would stop updating as expected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there's any benchmark on this?

It's a benchmark I did on my own using the same test included in this PR. A refactor using immer is actually very simple but it was three times worse in terms of performance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call on the "serialize" thing, I'll be fixing that in a follow-up PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up here #46282

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Package] Block editor /packages/block-editor [Type] Performance Related to performance efforts
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants