Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Site Editor: Add e2e tests for templates export #28324

Merged
merged 6 commits into from
Jan 29, 2021

Conversation

david-szabo97
Copy link
Member

@david-szabo97 david-szabo97 commented Jan 19, 2021

Description

Add end-to-end tests for site editor templates export functionality.

How has this been tested?

npm run test:e2e

Types of changes

Code Quality

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR.

@david-szabo97 david-szabo97 force-pushed the add/site-editor-export-e2e-tests branch from 07f7f0f to 9019132 Compare January 19, 2021 14:01
@github-actions
Copy link

github-actions bot commented Jan 19, 2021

Size Change: -101 B (0%)

Total Size: 1.37 MB

Filename Size Change
build/block-editor/index.js 123 kB +144 B (0%)
build/edit-navigation/index.js 11.1 kB -15 B (0%)
build/editor/editor-styles-rtl.css 543 B -115 B (-17%) 👏
build/editor/editor-styles.css 545 B -115 B (-17%) 👏
ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.14 kB 0 B
build/annotations/index.js 3.77 kB 0 B
build/api-fetch/index.js 3.4 kB 0 B
build/autop/index.js 2.83 kB 0 B
build/blob/index.js 665 B 0 B
build/block-directory/index.js 9.08 kB 0 B
build/block-directory/style-rtl.css 1.01 kB 0 B
build/block-directory/style.css 1.01 kB 0 B
build/block-editor/style-rtl.css 11.9 kB 0 B
build/block-editor/style.css 11.9 kB 0 B
build/block-library/blocks/archives/editor-rtl.css 61 B 0 B
build/block-library/blocks/archives/editor.css 60 B 0 B
build/block-library/blocks/audio/editor-rtl.css 58 B 0 B
build/block-library/blocks/audio/editor.css 58 B 0 B
build/block-library/blocks/audio/style-rtl.css 103 B 0 B
build/block-library/blocks/audio/style.css 103 B 0 B
build/block-library/blocks/block/editor-rtl.css 161 B 0 B
build/block-library/blocks/block/editor.css 161 B 0 B
build/block-library/blocks/button/editor-rtl.css 475 B 0 B
build/block-library/blocks/button/editor.css 474 B 0 B
build/block-library/blocks/button/style-rtl.css 453 B 0 B
build/block-library/blocks/button/style.css 451 B 0 B
build/block-library/blocks/buttons/editor-rtl.css 227 B 0 B
build/block-library/blocks/buttons/editor.css 227 B 0 B
build/block-library/blocks/buttons/style-rtl.css 297 B 0 B
build/block-library/blocks/buttons/style.css 297 B 0 B
build/block-library/blocks/calendar/style-rtl.css 208 B 0 B
build/block-library/blocks/calendar/style.css 208 B 0 B
build/block-library/blocks/categories/editor-rtl.css 84 B 0 B
build/block-library/blocks/categories/editor.css 83 B 0 B
build/block-library/blocks/categories/style-rtl.css 79 B 0 B
build/block-library/blocks/categories/style.css 79 B 0 B
build/block-library/blocks/code/style-rtl.css 90 B 0 B
build/block-library/blocks/code/style.css 90 B 0 B
build/block-library/blocks/columns/editor-rtl.css 190 B 0 B
build/block-library/blocks/columns/editor.css 190 B 0 B
build/block-library/blocks/columns/style-rtl.css 421 B 0 B
build/block-library/blocks/columns/style.css 421 B 0 B
build/block-library/blocks/cover/editor-rtl.css 392 B 0 B
build/block-library/blocks/cover/editor.css 393 B 0 B
build/block-library/blocks/cover/style-rtl.css 1.25 kB 0 B
build/block-library/blocks/cover/style.css 1.25 kB 0 B
build/block-library/blocks/embed/editor-rtl.css 486 B 0 B
build/block-library/blocks/embed/editor.css 486 B 0 B
build/block-library/blocks/embed/style-rtl.css 375 B 0 B
build/block-library/blocks/embed/style.css 375 B 0 B
build/block-library/blocks/file/editor-rtl.css 199 B 0 B
build/block-library/blocks/file/editor.css 198 B 0 B
build/block-library/blocks/file/style-rtl.css 248 B 0 B
build/block-library/blocks/file/style.css 248 B 0 B
build/block-library/blocks/freeform/editor-rtl.css 2.45 kB 0 B
build/block-library/blocks/freeform/editor.css 2.45 kB 0 B
build/block-library/blocks/gallery/editor-rtl.css 679 B 0 B
build/block-library/blocks/gallery/editor.css 679 B 0 B
build/block-library/blocks/gallery/style-rtl.css 1.07 kB 0 B
build/block-library/blocks/gallery/style.css 1.06 kB 0 B
build/block-library/blocks/group/editor-rtl.css 318 B 0 B
build/block-library/blocks/group/editor.css 317 B 0 B
build/block-library/blocks/group/style-rtl.css 57 B 0 B
build/block-library/blocks/group/style.css 57 B 0 B
build/block-library/blocks/heading/editor-rtl.css 129 B 0 B
build/block-library/blocks/heading/editor.css 129 B 0 B
build/block-library/blocks/heading/style-rtl.css 76 B 0 B
build/block-library/blocks/heading/style.css 76 B 0 B
build/block-library/blocks/html/editor-rtl.css 281 B 0 B
build/block-library/blocks/html/editor.css 281 B 0 B
build/block-library/blocks/image/editor-rtl.css 717 B 0 B
build/block-library/blocks/image/editor.css 716 B 0 B
build/block-library/blocks/image/style-rtl.css 477 B 0 B
build/block-library/blocks/image/style.css 478 B 0 B
build/block-library/blocks/latest-comments/editor-rtl.css 159 B 0 B
build/block-library/blocks/latest-comments/editor.css 158 B 0 B
build/block-library/blocks/latest-comments/style-rtl.css 269 B 0 B
build/block-library/blocks/latest-comments/style.css 269 B 0 B
build/block-library/blocks/latest-posts/editor-rtl.css 137 B 0 B
build/block-library/blocks/latest-posts/editor.css 137 B 0 B
build/block-library/blocks/latest-posts/style-rtl.css 523 B 0 B
build/block-library/blocks/latest-posts/style.css 522 B 0 B
build/block-library/blocks/list/editor-rtl.css 65 B 0 B
build/block-library/blocks/list/editor.css 65 B 0 B
build/block-library/blocks/list/style-rtl.css 63 B 0 B
build/block-library/blocks/list/style.css 63 B 0 B
build/block-library/blocks/media-text/editor-rtl.css 191 B 0 B
build/block-library/blocks/media-text/editor.css 191 B 0 B
build/block-library/blocks/media-text/style-rtl.css 535 B 0 B
build/block-library/blocks/media-text/style.css 532 B 0 B
build/block-library/blocks/more/editor-rtl.css 434 B 0 B
build/block-library/blocks/more/editor.css 434 B 0 B
build/block-library/blocks/navigation-link/editor-rtl.css 392 B 0 B
build/block-library/blocks/navigation-link/editor.css 394 B 0 B
build/block-library/blocks/navigation-link/style-rtl.css 704 B 0 B
build/block-library/blocks/navigation-link/style.css 702 B 0 B
build/block-library/blocks/navigation/editor-rtl.css 1.38 kB 0 B
build/block-library/blocks/navigation/editor.css 1.37 kB 0 B
build/block-library/blocks/navigation/style-rtl.css 171 B 0 B
build/block-library/blocks/navigation/style.css 171 B 0 B
build/block-library/blocks/nextpage/editor-rtl.css 395 B 0 B
build/block-library/blocks/nextpage/editor.css 395 B 0 B
build/block-library/blocks/paragraph/editor-rtl.css 109 B 0 B
build/block-library/blocks/paragraph/editor.css 109 B 0 B
build/block-library/blocks/paragraph/style-rtl.css 273 B 0 B
build/block-library/blocks/paragraph/style.css 273 B 0 B
build/block-library/blocks/post-author/editor-rtl.css 209 B 0 B
build/block-library/blocks/post-author/editor.css 209 B 0 B
build/block-library/blocks/post-author/style-rtl.css 183 B 0 B
build/block-library/blocks/post-author/style.css 184 B 0 B
build/block-library/blocks/post-comments-form/style-rtl.css 249 B 0 B
build/block-library/blocks/post-comments-form/style.css 249 B 0 B
build/block-library/blocks/post-content/editor-rtl.css 139 B 0 B
build/block-library/blocks/post-content/editor.css 139 B 0 B
build/block-library/blocks/post-excerpt/editor-rtl.css 73 B 0 B
build/block-library/blocks/post-excerpt/editor.css 73 B 0 B
build/block-library/blocks/post-featured-image/editor-rtl.css 338 B 0 B
build/block-library/blocks/post-featured-image/editor.css 338 B 0 B
build/block-library/blocks/post-featured-image/style-rtl.css 100 B 0 B
build/block-library/blocks/post-featured-image/style.css 100 B 0 B
build/block-library/blocks/preformatted/style-rtl.css 63 B 0 B
build/block-library/blocks/preformatted/style.css 63 B 0 B
build/block-library/blocks/pullquote/editor-rtl.css 183 B 0 B
build/block-library/blocks/pullquote/editor.css 183 B 0 B
build/block-library/blocks/pullquote/style-rtl.css 316 B 0 B
build/block-library/blocks/pullquote/style.css 316 B 0 B
build/block-library/blocks/query-loop/editor-rtl.css 90 B 0 B
build/block-library/blocks/query-loop/editor.css 89 B 0 B
build/block-library/blocks/query-loop/style-rtl.css 315 B 0 B
build/block-library/blocks/query-loop/style.css 317 B 0 B
build/block-library/blocks/query-pagination-numbers/editor-rtl.css 122 B 0 B
build/block-library/blocks/query-pagination-numbers/editor.css 121 B 0 B
build/block-library/blocks/query-pagination/editor-rtl.css 270 B 0 B
build/block-library/blocks/query-pagination/editor.css 262 B 0 B
build/block-library/blocks/query-pagination/style-rtl.css 168 B 0 B
build/block-library/blocks/query-pagination/style.css 168 B 0 B
build/block-library/blocks/query/editor-rtl.css 159 B 0 B
build/block-library/blocks/query/editor.css 160 B 0 B
build/block-library/blocks/quote/editor-rtl.css 61 B 0 B
build/block-library/blocks/quote/editor.css 61 B 0 B
build/block-library/blocks/quote/style-rtl.css 169 B 0 B
build/block-library/blocks/quote/style.css 169 B 0 B
build/block-library/blocks/rss/editor-rtl.css 201 B 0 B
build/block-library/blocks/rss/editor.css 202 B 0 B
build/block-library/blocks/rss/style-rtl.css 290 B 0 B
build/block-library/blocks/rss/style.css 290 B 0 B
build/block-library/blocks/search/editor-rtl.css 165 B 0 B
build/block-library/blocks/search/editor.css 165 B 0 B
build/block-library/blocks/search/style-rtl.css 342 B 0 B
build/block-library/blocks/search/style.css 344 B 0 B
build/block-library/blocks/separator/editor-rtl.css 99 B 0 B
build/block-library/blocks/separator/editor.css 99 B 0 B
build/block-library/blocks/separator/style-rtl.css 236 B 0 B
build/block-library/blocks/separator/style.css 236 B 0 B
build/block-library/blocks/shortcode/editor-rtl.css 504 B 0 B
build/block-library/blocks/shortcode/editor.css 504 B 0 B
build/block-library/blocks/site-logo/editor-rtl.css 201 B 0 B
build/block-library/blocks/site-logo/editor.css 201 B 0 B
build/block-library/blocks/site-logo/style-rtl.css 117 B 0 B
build/block-library/blocks/site-logo/style.css 117 B 0 B
build/block-library/blocks/social-link/editor-rtl.css 164 B 0 B
build/block-library/blocks/social-link/editor.css 165 B 0 B
build/block-library/blocks/social-links/editor-rtl.css 711 B 0 B
build/block-library/blocks/social-links/editor.css 712 B 0 B
build/block-library/blocks/social-links/style-rtl.css 1.37 kB 0 B
build/block-library/blocks/social-links/style.css 1.37 kB 0 B
build/block-library/blocks/spacer/editor-rtl.css 302 B 0 B
build/block-library/blocks/spacer/editor.css 302 B 0 B
build/block-library/blocks/spacer/style-rtl.css 48 B 0 B
build/block-library/blocks/spacer/style.css 48 B 0 B
build/block-library/blocks/subhead/editor-rtl.css 99 B 0 B
build/block-library/blocks/subhead/editor.css 99 B 0 B
build/block-library/blocks/subhead/style-rtl.css 80 B 0 B
build/block-library/blocks/subhead/style.css 80 B 0 B
build/block-library/blocks/table/editor-rtl.css 489 B 0 B
build/block-library/blocks/table/editor.css 489 B 0 B
build/block-library/blocks/table/style-rtl.css 386 B 0 B
build/block-library/blocks/table/style.css 386 B 0 B
build/block-library/blocks/tag-cloud/editor-rtl.css 118 B 0 B
build/block-library/blocks/tag-cloud/editor.css 118 B 0 B
build/block-library/blocks/tag-cloud/style-rtl.css 94 B 0 B
build/block-library/blocks/tag-cloud/style.css 94 B 0 B
build/block-library/blocks/template-part/editor-rtl.css 680 B 0 B
build/block-library/blocks/template-part/editor.css 679 B 0 B
build/block-library/blocks/text-columns/editor-rtl.css 95 B 0 B
build/block-library/blocks/text-columns/editor.css 95 B 0 B
build/block-library/blocks/text-columns/style-rtl.css 166 B 0 B
build/block-library/blocks/text-columns/style.css 166 B 0 B
build/block-library/blocks/verse/editor-rtl.css 62 B 0 B
build/block-library/blocks/verse/editor.css 62 B 0 B
build/block-library/blocks/verse/style-rtl.css 87 B 0 B
build/block-library/blocks/verse/style.css 87 B 0 B
build/block-library/blocks/video/editor-rtl.css 504 B 0 B
build/block-library/blocks/video/editor.css 503 B 0 B
build/block-library/blocks/video/style-rtl.css 193 B 0 B
build/block-library/blocks/video/style.css 193 B 0 B
build/block-library/common-rtl.css 1.01 kB 0 B
build/block-library/common.css 1.01 kB 0 B
build/block-library/editor-rtl.css 9.06 kB 0 B
build/block-library/editor.css 9.05 kB 0 B
build/block-library/index.js 143 kB 0 B
build/block-library/style-rtl.css 8.61 kB 0 B
build/block-library/style.css 8.61 kB 0 B
build/block-library/theme-rtl.css 748 B 0 B
build/block-library/theme.css 748 B 0 B
build/block-serialization-default-parser/index.js 1.88 kB 0 B
build/block-serialization-spec-parser/index.js 3.06 kB 0 B
build/blocks/index.js 48.3 kB 0 B
build/components/index.js 275 kB 0 B
build/components/style-rtl.css 15.5 kB 0 B
build/components/style.css 15.5 kB 0 B
build/compose/index.js 11.2 kB 0 B
build/core-data/index.js 15.2 kB 0 B
build/data-controls/index.js 829 B 0 B
build/data/index.js 8.97 kB 0 B
build/date/index.js 31.8 kB 0 B
build/deprecated/index.js 769 B 0 B
build/dom-ready/index.js 571 B 0 B
build/dom/index.js 4.93 kB 0 B
build/edit-navigation/style-rtl.css 938 B 0 B
build/edit-navigation/style.css 944 B 0 B
build/edit-post/index.js 306 kB 0 B
build/edit-post/style-rtl.css 6.51 kB 0 B
build/edit-post/style.css 6.5 kB 0 B
build/edit-site/index.js 24 kB 0 B
build/edit-site/style-rtl.css 4.01 kB 0 B
build/edit-site/style.css 4.01 kB 0 B
build/edit-widgets/index.js 23.6 kB 0 B
build/edit-widgets/style-rtl.css 3.17 kB 0 B
build/edit-widgets/style.css 3.18 kB 0 B
build/editor/index.js 41.8 kB 0 B
build/editor/style-rtl.css 3.89 kB 0 B
build/editor/style.css 3.89 kB 0 B
build/element/index.js 4.61 kB 0 B
build/escape-html/index.js 735 B 0 B
build/format-library/index.js 6.77 kB 0 B
build/format-library/style-rtl.css 637 B 0 B
build/format-library/style.css 639 B 0 B
build/hooks/index.js 2.27 kB 0 B
build/html-entities/index.js 622 B 0 B
build/i18n/index.js 3.56 kB 0 B
build/is-shallow-equal/index.js 698 B 0 B
build/keyboard-shortcuts/index.js 2.53 kB 0 B
build/keycodes/index.js 1.93 kB 0 B
build/list-reusable-blocks/index.js 3.14 kB 0 B
build/list-reusable-blocks/style-rtl.css 629 B 0 B
build/list-reusable-blocks/style.css 628 B 0 B
build/media-utils/index.js 5.32 kB 0 B
build/notices/index.js 1.85 kB 0 B
build/nux/index.js 3.41 kB 0 B
build/nux/style-rtl.css 731 B 0 B
build/nux/style.css 727 B 0 B
build/plugins/index.js 2.54 kB 0 B
build/primitives/index.js 1.42 kB 0 B
build/priority-queue/index.js 791 B 0 B
build/redux-routine/index.js 2.84 kB 0 B
build/reusable-blocks/index.js 2.92 kB 0 B
build/rich-text/index.js 13.4 kB 0 B
build/server-side-render/index.js 2.76 kB 0 B
build/shortcode/index.js 1.7 kB 0 B
build/token-list/index.js 1.27 kB 0 B
build/url/index.js 3.02 kB 0 B
build/viewport/index.js 1.86 kB 0 B
build/warning/index.js 1.14 kB 0 B
build/wordcount/index.js 1.22 kB 0 B

compressed-size-action

@david-szabo97 david-szabo97 self-assigned this Jan 19, 2021
@david-szabo97 david-szabo97 added [Feature] Full Site Editing [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. labels Jan 19, 2021
@david-szabo97 david-szabo97 marked this pull request as ready for review January 19, 2021 14:28
Copy link
Contributor

@Addison-Stavlo Addison-Stavlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good and the test is passing for me locally! A couple small Qs - also maybe @youknowriad may have some input here since he has worked with the export stuff a bit more.

@@ -100,3 +100,33 @@ export const navigationPanel = {
await item.click();
},
};

export const siteEditor = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This was a bit overdue and is good to see. 😁

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When things become more stable (and core merge approaches), we could start extracting some of these to e2e-test-utills.

@david-szabo97 david-szabo97 removed the request for review from noahtallen January 20, 2021 09:05
}
}

function delay( time ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a duplication of page.waitFor it seems.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

page.waitFor is deprecated, so I'd rather avoid introducing it in new tests. I'm thinking about replacing delay with page.waitForTimeout.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may have to update our waitFor lint rule to also target the usage of waitForTimeout

Copy link
Contributor

@youknowriad youknowriad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to see more tests being added for FSE :)

Copy link
Contributor

@Addison-Stavlo Addison-Stavlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! ✅

New and effected tests are passing both locally and on CI.

Copy link
Member

@vindl vindl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried running these test locally but they are failing for me for some reason on: TimeoutError: waiting for selector .edit-site-visual-editor iframe failed: timeout 30000ms exceeded.

I rerun the tests on the PR in CircleCI to double-check and it's still passing here, so I'm assuming something is messed up with my local setup. But now some seemingly unrelated e2e tests have failed here. :[

@david-szabo97 david-szabo97 force-pushed the add/site-editor-export-e2e-tests branch from 5e0b59d to 6cfee3b Compare January 26, 2021 21:56
@david-szabo97
Copy link
Member Author

Did a rebase, hopefully fixing CI tests. It works fine locally. I usually run npm run build before running e2e tests. Maybe that helps?

@ockham
Copy link
Contributor

ockham commented Jan 28, 2021

I tried running these test locally but they are failing for me for some reason on: TimeoutError: waiting for selector .edit-site-visual-editor iframe failed: timeout 30000ms exceeded.

I rerun the tests on the PR in CircleCI to double-check and it's still passing here, so I'm assuming something is messed up with my local setup. But now some seemingly unrelated e2e tests have failed here. :[

Curiously enough, I'm getting the same error lately whenever I try to run the e2e tests locally (while they do work in CI) -- most recently for #28554.

npm run build unfortunately didn't help 😕, nor did rm -rf node_modules && npm i.

Maybe it's my Linux box -- tho AFAIK you're on macOS @vindl, right?

Anyway, let's keep in touch in case either of us finds a solution to this.

@ockham
Copy link
Contributor

ockham commented Jan 28, 2021

I think I tracked it down by running e2e tests in interactive mode:

npm run test-e2e -- --puppeteer-interactive

Turns out e2e tests are stuck at a (pre-login) "Your database needs and upgrade" screen. I clicked the 'Upgrade' button, closed the testing window, and restarted tests. This time, they went through 🎉

We should document and/or fix this somewhere, tho 🤔

@mcsf
Copy link
Contributor

mcsf commented Jan 29, 2021

We should document and/or fix this somewhere, tho 🤔

Possibly https://github.com/WordPress/gutenberg/blob/master/docs/contributors/testing-overview.md

@vindl
Copy link
Member

vindl commented Jan 29, 2021

Maybe it's my Linux box -- tho AFAIK you're on macOS @vindl, right?

Yes, that's correct.

Turns out e2e tests are stuck at a (pre-login) "Your database needs and upgrade" screen. I clicked the 'Upgrade' button, closed the testing window, and restarted tests. This time, they went through 🎉

Thanks, I'll try this out! (update: yep, that fixed it 🎉)

@vindl vindl merged commit 37e1579 into master Jan 29, 2021
@vindl vindl deleted the add/site-editor-export-e2e-tests branch January 29, 2021 11:48
@github-actions github-actions bot added this to the Gutenberg 9.9 milestone Jan 29, 2021
@david-szabo97
Copy link
Member Author

@vindl @ockham Created a PR regarding the E2E tests upgrade screen: #28592

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants