Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch WebKit browser product to WebDriver executors #13339

Conversation

zdobersek
Copy link
Contributor

Switch away from Selenium executors to WebDriverTestharnessExecutor and
WebDriverRefTestExecutor for the WebKit browser product.

For the moment this requires manually mapping Selenium-style capability names
to the W3C equivalents that can be handled by WebKit's WebDriver implementation.
The browserVersion value is also hard-coded to the 2.20 release series for the
GTK+ port of WebKit as that's when the WebDriver functionality was introduced.

Switch away from Selenium executors to WebDriverTestharnessExecutor and
WebDriverRefTestExecutor for the WebKit browser product.

For the moment this requires manually mapping Selenium-style capability names
to the W3C equivalents that can be handled by WebKit's WebDriver implementation.
The browserVersion value is also hard-coded to the 2.20 release series for the
GTK+ port of WebKit as that's when the WebDriver functionality was introduced.
@wpt-pr-bot wpt-pr-bot added infra wptrunner The automated test runner, commonly called through ./wpt run labels Oct 3, 2018
@gsnedders
Copy link
Member

What testing have you done of this? Have you run at least /infrastructure and checked for regressions? (Or, better still, run everything?)

Copy link
Contributor

@jgraham jgraham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea here, but it would be good to remove the dependency on Selenium entirely.

@@ -43,6 +43,16 @@ def capabilities_for_port(server_config, **kwargs):
"certificateFile": kwargs["host_cert_path"]}
]
}

# Manually map Selenium's key names to the corresponding W3C versions.
Copy link
Contributor

@jgraham jgraham Oct 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't you just statically define a set of capabilities rather than still depending on the selenium defaults? Looks like the DesiredCapabilites.WEBKITGTK capabilities are rather simple anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'll do that.

@zdobersek
Copy link
Contributor Author

A small set of tests is run currently. Nothing changes there.

Things in /infrastructure also remain without big changes. Before:

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 114 checks (46 tests, 68 subtests)
Expected results: 84
Unexpected results: 30
  test: 8 (4 fail, 3 timeout, 1 error)
  subtest: 22 (20 fail, 1 notrun, 1 error)

Now:

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 114 checks (46 tests, 68 subtests)
Expected results: 85
Unexpected results: 29
  test: 8 (4 fail, 1 crash, 3 timeout)
  subtest: 21 (20 fail, 1 notrun)

@foolip foolip mentioned this pull request Oct 3, 2018
7 tasks
@Ms2ger Ms2ger merged commit dc97ad5 into web-platform-tests:master Oct 3, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infra wptrunner The automated test runner, commonly called through ./wpt run
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants