Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance tests: Fix #28026

Merged
merged 2 commits into from
Jan 7, 2021
Merged

Performance tests: Fix #28026

merged 2 commits into from
Jan 7, 2021

Conversation

ockham
Copy link
Contributor

@ockham ockham commented Jan 6, 2021

Description

Purports to fix an issue that @youknowriad shared via Slack:

./bin/plugin/cli.js perf wp/5.6 release/9.6 release/9.7

when it switches to release/5.6 for some reason, there’s a “module not found error” on npm run build (even if just before there’s rm -rf node_modules && npm install

Specifically, the module that can't be found is levenary (which is required by babel).

I was able to reproduce by following those steps manually. It seems like the problem is with nested node_modules/ directories, e.g. packages/babel-preset-default/node_modules/. Consequently, this PR adds a new distclean command to package.json to also remove those nested node_modules/, and uses that command in the perf test script.

How has this been tested?

It hasn't really -- I'm having trouble running the perf tests on my Linux box 😬

The command to test them is

./bin/plugin/cli.js perf wp/5.6 release/9.6 release/9.7

Types of changes

Bug fix

@ockham ockham added [Type] Bug An existing feature does not function as intended [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. labels Jan 6, 2021
@ockham ockham requested review from youknowriad and a team January 6, 2021 19:46
@ockham ockham self-assigned this Jan 6, 2021
@github-actions
Copy link

github-actions bot commented Jan 6, 2021

Size Change: 0 B

Total Size: 1.3 MB

ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.14 kB 0 B
build/annotations/index.js 3.8 kB 0 B
build/api-fetch/index.js 3.42 kB 0 B
build/autop/index.js 2.83 kB 0 B
build/blob/index.js 664 B 0 B
build/block-directory/index.js 9.04 kB 0 B
build/block-directory/style-rtl.css 1.01 kB 0 B
build/block-directory/style.css 1.01 kB 0 B
build/block-editor/index.js 130 kB 0 B
build/block-editor/style-rtl.css 11.4 kB 0 B
build/block-editor/style.css 11.3 kB 0 B
build/block-library/blocks/archives/editor-rtl.css 196 B 0 B
build/block-library/blocks/archives/editor.css 196 B 0 B
build/block-library/blocks/audio/editor-rtl.css 194 B 0 B
build/block-library/blocks/audio/editor.css 194 B 0 B
build/block-library/blocks/audio/style-rtl.css 225 B 0 B
build/block-library/blocks/audio/style.css 225 B 0 B
build/block-library/blocks/block/editor-rtl.css 283 B 0 B
build/block-library/blocks/block/editor.css 283 B 0 B
build/block-library/blocks/button/editor-rtl.css 576 B 0 B
build/block-library/blocks/button/editor.css 577 B 0 B
build/block-library/blocks/button/style-rtl.css 552 B 0 B
build/block-library/blocks/button/style.css 552 B 0 B
build/block-library/blocks/buttons/editor-rtl.css 345 B 0 B
build/block-library/blocks/buttons/editor.css 346 B 0 B
build/block-library/blocks/buttons/style-rtl.css 419 B 0 B
build/block-library/blocks/buttons/style.css 419 B 0 B
build/block-library/blocks/calendar/style-rtl.css 319 B 0 B
build/block-library/blocks/calendar/style.css 319 B 0 B
build/block-library/blocks/categories/editor-rtl.css 210 B 0 B
build/block-library/blocks/categories/editor.css 209 B 0 B
build/block-library/blocks/categories/style-rtl.css 208 B 0 B
build/block-library/blocks/categories/style.css 208 B 0 B
build/block-library/blocks/code/style-rtl.css 216 B 0 B
build/block-library/blocks/code/style.css 216 B 0 B
build/block-library/blocks/columns/editor-rtl.css 300 B 0 B
build/block-library/blocks/columns/editor.css 299 B 0 B
build/block-library/blocks/columns/style-rtl.css 529 B 0 B
build/block-library/blocks/columns/style.css 528 B 0 B
build/block-library/blocks/cover/editor-rtl.css 508 B 0 B
build/block-library/blocks/cover/editor.css 506 B 0 B
build/block-library/blocks/cover/style-rtl.css 1.33 kB 0 B
build/block-library/blocks/cover/style.css 1.32 kB 0 B
build/block-library/blocks/embed/editor-rtl.css 594 B 0 B
build/block-library/blocks/embed/editor.css 595 B 0 B
build/block-library/blocks/embed/style-rtl.css 489 B 0 B
build/block-library/blocks/embed/style.css 489 B 0 B
build/block-library/blocks/file/editor-rtl.css 314 B 0 B
build/block-library/blocks/file/editor.css 313 B 0 B
build/block-library/blocks/file/style-rtl.css 352 B 0 B
build/block-library/blocks/file/style.css 352 B 0 B
build/block-library/blocks/freeform/editor-rtl.css 2.55 kB 0 B
build/block-library/blocks/freeform/editor.css 2.55 kB 0 B
build/block-library/blocks/gallery/editor-rtl.css 749 B 0 B
build/block-library/blocks/gallery/editor.css 750 B 0 B
build/block-library/blocks/gallery/style-rtl.css 1.17 kB 0 B
build/block-library/blocks/gallery/style.css 1.17 kB 0 B
build/block-library/blocks/group/editor-rtl.css 433 B 0 B
build/block-library/blocks/group/editor.css 432 B 0 B
build/block-library/blocks/group/style-rtl.css 190 B 0 B
build/block-library/blocks/group/style.css 190 B 0 B
build/block-library/blocks/heading/editor-rtl.css 248 B 0 B
build/block-library/blocks/heading/editor.css 248 B 0 B
build/block-library/blocks/heading/style-rtl.css 212 B 0 B
build/block-library/blocks/heading/style.css 212 B 0 B
build/block-library/blocks/html/editor-rtl.css 384 B 0 B
build/block-library/blocks/html/editor.css 385 B 0 B
build/block-library/blocks/image/editor-rtl.css 801 B 0 B
build/block-library/blocks/image/editor.css 800 B 0 B
build/block-library/blocks/image/style-rtl.css 569 B 0 B
build/block-library/blocks/image/style.css 570 B 0 B
build/block-library/blocks/latest-comments/editor-rtl.css 277 B 0 B
build/block-library/blocks/latest-comments/editor.css 275 B 0 B
build/block-library/blocks/latest-comments/style-rtl.css 382 B 0 B
build/block-library/blocks/latest-comments/style.css 382 B 0 B
build/block-library/blocks/latest-posts/editor-rtl.css 254 B 0 B
build/block-library/blocks/latest-posts/editor.css 254 B 0 B
build/block-library/blocks/latest-posts/style-rtl.css 634 B 0 B
build/block-library/blocks/latest-posts/style.css 634 B 0 B
build/block-library/blocks/list/editor-rtl.css 203 B 0 B
build/block-library/blocks/list/editor.css 203 B 0 B
build/block-library/blocks/list/style-rtl.css 201 B 0 B
build/block-library/blocks/list/style.css 201 B 0 B
build/block-library/blocks/media-text/editor-rtl.css 311 B 0 B
build/block-library/blocks/media-text/editor.css 311 B 0 B
build/block-library/blocks/media-text/style-rtl.css 642 B 0 B
build/block-library/blocks/media-text/style.css 640 B 0 B
build/block-library/blocks/more/editor-rtl.css 545 B 0 B
build/block-library/blocks/more/editor.css 545 B 0 B
build/block-library/blocks/navigation-link/editor-rtl.css 503 B 0 B
build/block-library/blocks/navigation-link/editor.css 504 B 0 B
build/block-library/blocks/navigation-link/style-rtl.css 805 B 0 B
build/block-library/blocks/navigation-link/style.css 803 B 0 B
build/block-library/blocks/navigation/editor-rtl.css 1.38 kB 0 B
build/block-library/blocks/navigation/editor.css 1.38 kB 0 B
build/block-library/blocks/navigation/style-rtl.css 274 B 0 B
build/block-library/blocks/navigation/style.css 274 B 0 B
build/block-library/blocks/nextpage/editor-rtl.css 507 B 0 B
build/block-library/blocks/nextpage/editor.css 507 B 0 B
build/block-library/blocks/paragraph/editor-rtl.css 236 B 0 B
build/block-library/blocks/paragraph/editor.css 236 B 0 B
build/block-library/blocks/paragraph/style-rtl.css 351 B 0 B
build/block-library/blocks/paragraph/style.css 352 B 0 B
build/block-library/blocks/post-author/editor-rtl.css 329 B 0 B
build/block-library/blocks/post-author/editor.css 329 B 0 B
build/block-library/blocks/post-author/style-rtl.css 303 B 0 B
build/block-library/blocks/post-author/style.css 303 B 0 B
build/block-library/blocks/post-comments-form/style-rtl.css 358 B 0 B
build/block-library/blocks/post-comments-form/style.css 358 B 0 B
build/block-library/blocks/post-content/editor-rtl.css 262 B 0 B
build/block-library/blocks/post-content/editor.css 262 B 0 B
build/block-library/blocks/post-excerpt/editor-rtl.css 206 B 0 B
build/block-library/blocks/post-excerpt/editor.css 206 B 0 B
build/block-library/blocks/post-featured-image/editor-rtl.css 453 B 0 B
build/block-library/blocks/post-featured-image/editor.css 453 B 0 B
build/block-library/blocks/post-featured-image/style-rtl.css 223 B 0 B
build/block-library/blocks/post-featured-image/style.css 223 B 0 B
build/block-library/blocks/preformatted/style-rtl.css 193 B 0 B
build/block-library/blocks/preformatted/style.css 193 B 0 B
build/block-library/blocks/pullquote/editor-rtl.css 304 B 0 B
build/block-library/blocks/pullquote/editor.css 304 B 0 B
build/block-library/blocks/pullquote/style-rtl.css 428 B 0 B
build/block-library/blocks/pullquote/style.css 428 B 0 B
build/block-library/blocks/query-loop/editor-rtl.css 217 B 0 B
build/block-library/blocks/query-loop/editor.css 216 B 0 B
build/block-library/blocks/query-loop/style-rtl.css 427 B 0 B
build/block-library/blocks/query-loop/style.css 429 B 0 B
build/block-library/blocks/query/editor-rtl.css 279 B 0 B
build/block-library/blocks/query/editor.css 279 B 0 B
build/block-library/blocks/quote/editor-rtl.css 195 B 0 B
build/block-library/blocks/quote/editor.css 195 B 0 B
build/block-library/blocks/quote/style-rtl.css 284 B 0 B
build/block-library/blocks/quote/style.css 285 B 0 B
build/block-library/blocks/rss/editor-rtl.css 307 B 0 B
build/block-library/blocks/rss/editor.css 309 B 0 B
build/block-library/blocks/rss/style-rtl.css 394 B 0 B
build/block-library/blocks/rss/style.css 393 B 0 B
build/block-library/blocks/search/editor-rtl.css 285 B 0 B
build/block-library/blocks/search/editor.css 285 B 0 B
build/block-library/blocks/search/style-rtl.css 454 B 0 B
build/block-library/blocks/search/style.css 456 B 0 B
build/block-library/blocks/separator/editor-rtl.css 229 B 0 B
build/block-library/blocks/separator/editor.css 229 B 0 B
build/block-library/blocks/separator/style-rtl.css 352 B 0 B
build/block-library/blocks/separator/style.css 352 B 0 B
build/block-library/blocks/shortcode/editor-rtl.css 603 B 0 B
build/block-library/blocks/shortcode/editor.css 603 B 0 B
build/block-library/blocks/site-logo/editor-rtl.css 321 B 0 B
build/block-library/blocks/site-logo/editor.css 321 B 0 B
build/block-library/blocks/site-logo/style-rtl.css 238 B 0 B
build/block-library/blocks/site-logo/style.css 238 B 0 B
build/block-library/blocks/social-link/editor-rtl.css 283 B 0 B
build/block-library/blocks/social-link/editor.css 283 B 0 B
build/block-library/blocks/social-links/editor-rtl.css 811 B 0 B
build/block-library/blocks/social-links/editor.css 810 B 0 B
build/block-library/blocks/social-links/style-rtl.css 1.44 kB 0 B
build/block-library/blocks/social-links/style.css 1.44 kB 0 B
build/block-library/blocks/spacer/editor-rtl.css 390 B 0 B
build/block-library/blocks/spacer/editor.css 390 B 0 B
build/block-library/blocks/spacer/style-rtl.css 184 B 0 B
build/block-library/blocks/spacer/style.css 184 B 0 B
build/block-library/blocks/subhead/editor-rtl.css 223 B 0 B
build/block-library/blocks/subhead/editor.css 223 B 0 B
build/block-library/blocks/subhead/style-rtl.css 210 B 0 B
build/block-library/blocks/subhead/style.css 210 B 0 B
build/block-library/blocks/table/editor-rtl.css 593 B 0 B
build/block-library/blocks/table/editor.css 593 B 0 B
build/block-library/blocks/table/style-rtl.css 501 B 0 B
build/block-library/blocks/table/style.css 501 B 0 B
build/block-library/blocks/tag-cloud/editor-rtl.css 237 B 0 B
build/block-library/blocks/tag-cloud/editor.css 235 B 0 B
build/block-library/blocks/tag-cloud/style-rtl.css 221 B 0 B
build/block-library/blocks/tag-cloud/style.css 221 B 0 B
build/block-library/blocks/template-part/editor-rtl.css 714 B 0 B
build/block-library/blocks/template-part/editor.css 714 B 0 B
build/block-library/blocks/text-columns/editor-rtl.css 220 B 0 B
build/block-library/blocks/text-columns/editor.css 220 B 0 B
build/block-library/blocks/text-columns/style-rtl.css 283 B 0 B
build/block-library/blocks/text-columns/style.css 283 B 0 B
build/block-library/blocks/verse/editor-rtl.css 194 B 0 B
build/block-library/blocks/verse/editor.css 194 B 0 B
build/block-library/blocks/verse/style-rtl.css 214 B 0 B
build/block-library/blocks/verse/style.css 214 B 0 B
build/block-library/blocks/video/editor-rtl.css 617 B 0 B
build/block-library/blocks/video/editor.css 617 B 0 B
build/block-library/blocks/video/style-rtl.css 303 B 0 B
build/block-library/blocks/video/style.css 304 B 0 B
build/block-library/common-rtl.css 1.01 kB 0 B
build/block-library/common.css 1.01 kB 0 B
build/block-library/editor-rtl.css 8.93 kB 0 B
build/block-library/editor.css 8.93 kB 0 B
build/block-library/index.js 151 kB 0 B
build/block-library/style-rtl.css 8.53 kB 0 B
build/block-library/style.css 8.53 kB 0 B
build/block-library/theme-rtl.css 860 B 0 B
build/block-library/theme.css 860 B 0 B
build/block-serialization-default-parser/index.js 1.88 kB 0 B
build/block-serialization-spec-parser/index.js 3.06 kB 0 B
build/blocks/index.js 48 kB 0 B
build/components/index.js 172 kB 0 B
build/components/style-rtl.css 15.5 kB 0 B
build/components/style.css 15.5 kB 0 B
build/compose/index.js 11.2 kB 0 B
build/core-data/index.js 15.2 kB 0 B
build/data-controls/index.js 830 B 0 B
build/data/index.js 8.98 kB 0 B
build/date/index.js 31.8 kB 0 B
build/deprecated/index.js 768 B 0 B
build/dom-ready/index.js 571 B 0 B
build/dom/index.js 4.95 kB 0 B
build/edit-navigation/index.js 11.1 kB 0 B
build/edit-navigation/style-rtl.css 938 B 0 B
build/edit-navigation/style.css 944 B 0 B
build/edit-post/index.js 306 kB 0 B
build/edit-post/style-rtl.css 6.64 kB 0 B
build/edit-post/style.css 6.63 kB 0 B
build/edit-site/index.js 24.4 kB 0 B
build/edit-site/style-rtl.css 4.04 kB 0 B
build/edit-site/style.css 4.04 kB 0 B
build/edit-widgets/index.js 26.1 kB 0 B
build/edit-widgets/style-rtl.css 3.22 kB 0 B
build/edit-widgets/style.css 3.22 kB 0 B
build/editor/editor-styles-rtl.css 476 B 0 B
build/editor/editor-styles.css 478 B 0 B
build/editor/index.js 42.8 kB 0 B
build/editor/style-rtl.css 3.89 kB 0 B
build/editor/style.css 3.89 kB 0 B
build/element/index.js 4.62 kB 0 B
build/escape-html/index.js 735 B 0 B
build/format-library/index.js 6.75 kB 0 B
build/format-library/style-rtl.css 620 B 0 B
build/format-library/style.css 621 B 0 B
build/hooks/index.js 2.27 kB 0 B
build/html-entities/index.js 623 B 0 B
build/i18n/index.js 3.57 kB 0 B
build/is-shallow-equal/index.js 698 B 0 B
build/keyboard-shortcuts/index.js 2.54 kB 0 B
build/keycodes/index.js 1.94 kB 0 B
build/list-reusable-blocks/index.js 3.15 kB 0 B
build/list-reusable-blocks/style-rtl.css 629 B 0 B
build/list-reusable-blocks/style.css 628 B 0 B
build/media-utils/index.js 5.31 kB 0 B
build/notices/index.js 1.86 kB 0 B
build/nux/index.js 3.43 kB 0 B
build/nux/style-rtl.css 731 B 0 B
build/nux/style.css 727 B 0 B
build/plugins/index.js 2.54 kB 0 B
build/primitives/index.js 1.43 kB 0 B
build/priority-queue/index.js 790 B 0 B
build/redux-routine/index.js 2.84 kB 0 B
build/reusable-blocks/index.js 2.92 kB 0 B
build/rich-text/index.js 13.4 kB 0 B
build/server-side-render/index.js 2.77 kB 0 B
build/shortcode/index.js 1.7 kB 0 B
build/token-list/index.js 1.27 kB 0 B
build/url/index.js 3.02 kB 0 B
build/viewport/index.js 1.86 kB 0 B
build/warning/index.js 1.14 kB 0 B
build/wordcount/index.js 1.22 kB 0 B

compressed-size-action

@gziolo
Copy link
Member

gziolo commented Jan 7, 2021

I was able to reproduce by following those steps manually. It seems like the problem is with nested node_modules/ directories, e.g. packages/babel-preset-default/node_modules/. Consequently, this PR adds a new distclean command to package.json to also remove those nested node_modules/, and uses that command in the perf test script.

It looks like the issue is with the lock file in one of the older branches. In general, all those development dependencies should be hoisted to the root node_modules folder. In some cases removing orphan node_modules in packages might help, but overall I would expect that npm will install them again if they are listed as local dependencies.

Example:

Screen Shot 2021-01-07 at 08 17 17

one of the React Native packages overrides packages on purpose and therefore they are installed as local dependencies.

Screen Shot 2021-01-07 at 08 22 07

Copy link
Contributor

@youknowriad youknowriad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the original issue is that these node_modules shouldn't exist, that said, it seems that it doesn't hurt to make the script less prone to these errors.

@gziolo
Copy link
Member

gziolo commented Jan 7, 2021

I agree that the original issue is that these node_modules shouldn't exist, that said, it seems that it doesn't hurt to make the script less prone to these errors.

If it helps then sure, but I'm afraid it's a fix that will work only one time 😄

@youknowriad
Copy link
Contributor

it seems as written it doesn't work anyway because distclean command is not available in old branches, we should just move the command to the command instead.

@gziolo what do you mean it works only one time.

@gziolo
Copy link
Member

gziolo commented Jan 7, 2021

In general, we tend to avoid having different versions of the same development packages (in fact all of them, see lodash, reakit, etc) used across the repository to avoid errors like that. I think you are right that it might prevent issues if there is more occurrences like this time when you switch between branches that have different setup of the package lock file. I didn't think about it this way when writing my previous comment.

@ockham
Copy link
Contributor Author

ockham commented Jan 7, 2021

it seems as written it doesn't work anyway because distclean command is not available in old branches, we should just move the command to the command instead.

Hmm right, makes sense. It's a bit confusing to keep track of which files are used from the current branch, and which ones from the ones that are being evaluated 😅

Anyway, I'll change accordingly.

@ockham ockham force-pushed the fix/performance-tests branch from 0cb43e5 to ca61b4a Compare January 7, 2021 12:56
@ockham
Copy link
Contributor Author

ockham commented Jan 7, 2021

Anyway, I'll change accordingly.

Done (and rebased).

@ockham
Copy link
Contributor Author

ockham commented Jan 7, 2021

Going to merge (per Riad's approval here, and having applied this change) 🙂

@ockham ockham merged commit cf31360 into master Jan 7, 2021
@ockham ockham deleted the fix/performance-tests branch January 7, 2021 19:43
@github-actions github-actions bot added this to the Gutenberg 9.8 milestone Jan 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Bug An existing feature does not function as intended
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants