Try running audio/video tests on more browsers and platforms #30437
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stephenwade
changed the title
|
@microsoft-github-policy-service agree |
This comment has been minimized.
This comment has been minimized.
stephenwade
changed the title
|
cc @mxschmitt I think this needs a tag to run the affected tests. |
This comment has been minimized.
This comment has been minimized.
|
Done! |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@mxschmitt Looks like "tests 2" didn't run when I pushed another commit, and the Installation Test needs to be re-run. Could you help? |
This comment has been minimized.
This comment has been minimized.
|
@mxschmitt Looks like the "should play audio" and "should play video" tests are flaky on |
|
Hey @mxschmitt, just following up on this. I pushed a commit to skip the "should play audio" and "should play video" tests on Webkit on macos-12, since that's where I have observed flakiness. Please re-add the label to re-run the tests 😀 |
Test results for "tests 1"1 flaky27249 passed, 670 skipped Merge workflow run. |
mxschmitt
reviewed
May 8, 2024
| @@ -84,8 +83,7 @@ it('should play video @smoke', async ({ page, asset, browserName, platform, mode | |||
| }); | |||
|
|
|||
| it('should play webm video @smoke', async ({ page, asset, browserName, platform, mode }) => { | |||
| it.fixme(browserName === 'webkit' && platform === 'darwin' && parseInt(os.release(), 10) === 20, 'Does not work on BigSur'); | |||
| it.fixme(browserName === 'webkit' && platform === 'win32'); | |||
| it.fixme(browserName === 'webkit' && platform !== 'darwin'); | |||
There was a problem hiding this comment.
so this means it doesn't work on linux?
mxschmitt
reviewed
May 8, 2024
| it.fixme(browserName === 'firefox' && platform === 'win32', 'https://github.com/microsoft/playwright/issues/10887'); | ||
| it.fixme(browserName === 'firefox' && platform === 'linux', 'https://github.com/microsoft/playwright/issues/10887'); | ||
| it.fixme(browserName === 'webkit' && platform === 'win32', 'https://github.com/microsoft/playwright/issues/10892'); | ||
| it.fixme(browserName === 'webkit' && platform !== 'darwin'); |
There was a problem hiding this comment.
it wasn't fixme on linux before?
lets try to stick with === rather than !== I think thats easier to read.
|
Re-triggered the bot. Also landed this #30634 which should make things simpler (I think it doesn't work for macOS 12 / maybe 13 but does a good job for 14). |
Test results for "tests 2"6 fatal errors, not part of any test 48 flaky167883 passed, 7560 skipped, 3960 did not run Merge workflow run. |
|
Archiving this |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm trying to debug a Playwright issue with
AudioContexton Windows runners (will file a separate issue if I can't figure it out), and I'm interested to see if Playwright's audio/video tests now work on all platforms and browsers.This PR removes some
it.fixmeannotations from the audio/video tests incapabilities.spec.ts. All theit.fixmelines I removed were added more than a year ago, and all the linked issues look like they are resolved.Related: #30410