Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify WebDriver command for testing #89

Merged
merged 2 commits into from
Jun 5, 2018
Merged

Specify WebDriver command for testing #89

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ade1ecb
Specify WebDriver command for testing
paulmeyer90 Jun 4, 2018
80bb62b
Cleaned up formatting
paulmeyer90 Jun 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 80 additions & 1 deletion index.src.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,20 @@ <h1>Reporting API</h1>
text: Date object; url: sec-date-objects
type: interface
text: Date; url: sec-date-objects

spec: webdriver; urlPrefix: https://w3c.github.io/webdriver/webdriver-spec.html#
type: dfn
text: current browsing context; url: dfn-current-browsing-context
text: WebDriver error; url: dfn-error
text: WebDriver error code; url: dfn-error-code
text: extension command; url: dfn-extension-commands
text: extension command name; url: dfn-extension-command-name
text: extension command prefix; url: dfn-extension-command-prefix
text: invalid argument; url: dfn-invalid-argument
text: local end; url: dfn-local-end
text: remote end steps; url: dfn-remote-end-steps
text: session; url: dfn-session
text: success; url: dfn-success
text: trying; url: dfn-try
</pre>
<pre class="biblio">
{
Expand Down Expand Up @@ -1229,6 +1242,72 @@ <h2 id="sample-reports">Sample Reports</h2>
</div>
</section>

<section>
<h2 id="automation">Automation</h2>

For the purposes of user-agent automation and application testing, this
document defines a number of <a>extension commands</a> for the [[WebDriver]]
specification.

<h3 id="generate-test-report-command">Generate Test Report</h3>

The <dfn>Generate Test Report</dfn> <a>extension command</a> simulates the
generation of a <a>report</a> for the purposes of testing. This report will be
observed by any <a>registered</a> <a>reporting observers</a>.

The <a>extension command</a> is defined as follows:

<pre class='idl'>
dictionary GenerateTestReportParameters {
required DOMString message;
DOMString group;
};
</pre>

<table style="border-spacing: 20px 0px;">
<tbody>
<tr>
<th>HTTP Method</th>
<th><a lt="extension command prefix">Prefix</a></th>
<th><a lt="extension command name">Name</a></th>
</tr>
<tr>
<td>`POST`</td>
<td>`/session/{session id}/reporting`</td>
<td>`generate_test_report`</td>
</tr>
</tbody>
</table>

The <a>remote end steps</a> are:

1. If |parameters| is not a JSON <a>Object</a>, return a <a>WebDriver
error</a> with <a>WebDriver error code</a> <a>invalid argument</a>.

2. Let |message| be the result of <a>trying</a> to get |parameters|'s
{{GenerateTestReportParameters/message}} property.

3. If |message| is null, return a <a>WebDriver error</a> with <a>WebDriver
error code</a> <a>invalid argument</a>.

4. Let |group| be the result of <a>trying</a> to get |parameters|'s
{{GenerateTestReportParameters/group}} property.

5. If |group| is null, set |group| to "default".

6. Let |body| be a new object that can be serialized into a <a>JSON text</a>,
containing a single string field, |body_message|.

7. Set |body_message| to |message|.

8. Let |settings| be the <a>environment settings object</a> of the
<a>current browsing context</a>'s <a>active document</a>.

9. Execute [[#queue-report]] with |body|, "test", |group|, and |settings|.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if this fails/throws an error?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're okay here. The #queue-report algorithm always succeeds — it just adds the report to a queue and immediately returns.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Though that does mean we might need another extension command that lets you trigger the #send-reports algorithm, and that one does have interesting failure modes]

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there is some unforeseen memory error? Or it's implemented in some more complicated way in a different browser?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd argue that there's not much that you can do in that case. Report delivery is already best-effort, not guaranteed (see §1.1). A report can be dropped at any point down the line (by being evicted when a later report is queued, for instance, or during a GC run), and there's no way to signal that back to whatever originally generated the report.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh I see what you are saying now. In the scope of just the reporting API that makes sense to me, but I think I would still prefer to have some sort of catch for an error. Maybe WebDriver is doing this by invoking a call to another system and that system changes the name of the method WebDriver usually calls. In that case we never reach the queue but we still error out. I think Unknown Error in the WebDriver spec could be used for this case.

WYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give an example of a specific error condition that you'd want to write a test for?

Copy link

@kereliuk kereliuk Jun 5, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If ChromeDriver implements this it will probably use the DevTools Protocol to dispatch the queue-report. Would want to have an error catch error in case there is an error thrown in the protocol code.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a conversation offline with Paul I've changed my opinion and I think this is good to go as is. For further reference this can be compared to step 6 in the maximize window command in the WebDriver (spec)[https://w3c.github.io/webdriver/#maximize-window] which is handled in a similar way to this.


10. Return <a>success</a> with data null.
</section>

<section>
<h2 id="security">Security Considerations</h2>

Expand Down