-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Specify WebDriver command for testing #89
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Excited to see this PR! Looks good, but need a way to handle errors on the penultimate steps which.
8. Let |settings| be the <a>environment settings object</a> of the | ||
<a>current browsing context</a>'s <a>active document</a>. | ||
|
||
9. Execute [[#queue-report]] with |body|, "test", |group|, and |settings|. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if this fails/throws an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we're okay here. The #queue-report
algorithm always succeeds — it just adds the report to a queue and immediately returns.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Though that does mean we might need another extension command that lets you trigger the #send-reports
algorithm, and that one does have interesting failure modes]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if there is some unforeseen memory error? Or it's implemented in some more complicated way in a different browser?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd argue that there's not much that you can do in that case. Report delivery is already best-effort, not guaranteed (see §1.1). A report can be dropped at any point down the line (by being evicted when a later report is queued, for instance, or during a GC run), and there's no way to signal that back to whatever originally generated the report.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahh I see what you are saying now. In the scope of just the reporting API that makes sense to me, but I think I would still prefer to have some sort of catch for an error. Maybe WebDriver is doing this by invoking a call to another system and that system changes the name of the method WebDriver usually calls. In that case we never reach the queue but we still error out. I think Unknown Error in the WebDriver spec could be used for this case.
WYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you give an example of a specific error condition that you'd want to write a test for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If ChromeDriver implements this it will probably use the DevTools Protocol to dispatch the queue-report. Would want to have an error catch error in case there is an error thrown in the protocol code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After a conversation offline with Paul I've changed my opinion and I think this is good to go as is. For further reference this can be compared to step 6 in the maximize window command in the WebDriver (spec)[https://w3c.github.io/webdriver/#maximize-window] which is handled in a similar way to this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
8. Let |settings| be the <a>environment settings object</a> of the | ||
<a>current browsing context</a>'s <a>active document</a>. | ||
|
||
9. Execute [[#queue-report]] with |body|, "test", |group|, and |settings|. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we're okay here. The #queue-report
algorithm always succeeds — it just adds the report to a queue and immediately returns.
index.src.html
Outdated
<tr> | ||
<td>POST</td> | ||
<td>/session/{session id}/reporting</td> | ||
<td>generate_test_report</td> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Super nit: can you wrap the prefix and name in backticks / tt
to make them monospace?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done!
These changes add an "Automation" section, which defines a new WebDriver extension command, Generate Test Report. This command will allow generic reports to be generated for testing purposes, e.g. in web platform tests.
Preview | Diff