Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve service worker script caching and update #1283

Merged
merged 4 commits into from
Mar 13, 2018
Merged

Improve service worker script caching and update #1283

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fcabe9c
Improve service worker script caching and update
jungkees Mar 5, 2018
ea84047
Clarify a reference to the script resource map
jungkees Mar 7, 2018
e2c9db2
Do all byte-check before starting the service worker
jungkees Mar 8, 2018
c34985f
Apply the changes to V1 and add html files
jungkees Mar 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 50 additions & 31 deletions docs/index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ spec: webappsec-referrer-policy; urlPrefix: https://w3c.github.io/webappsec-refe

A [=/service worker=] has an associated <dfn export id="dfn-skip-waiting-flag">skip waiting flag</dfn>. Unless stated otherwise it is unset.

A [=/service worker=] has an associated <dfn export id="dfn-imported-scripts-updated-flag">imported scripts updated flag</dfn>. It is initially unset.
A [=/service worker=] has an associated <dfn export id="dfn-classic-scripts-imported-flag">classic scripts imported flag</dfn>. It is initially unset.

A [=/service worker=] has an associated <dfn export id="dfn-set-of-event-types-to-handle">set of event types to handle</dfn> (a [=ordered set|set=]) whose [=list/item=] is an <a>event listener</a>'s event type. It is initially an empty set.

Expand Down Expand Up @@ -2027,22 +2027,22 @@ spec: webappsec-referrer-policy; urlPrefix: https://w3c.github.io/webappsec-refe
When the <dfn method for="ServiceWorkerGlobalScope" id="importscripts-method"><code>importScripts(|urls|)</code></dfn> method is called on a {{ServiceWorkerGlobalScope}} object, the user agent *must* <a>import scripts into worker global scope</a>, given this {{ServiceWorkerGlobalScope}} object and |urls|, and with the following steps to [=fetching scripts/perform the fetch=] given the [=/request=] |request|:

1. Let |serviceWorker| be |request|'s [=request/client=]'s [=environment settings object/global object=]'s [=ServiceWorkerGlobalScope/service worker=].
1. If |serviceWorker|'s <a>imported scripts updated flag</a> is unset, then:
1. Let |registration| be |serviceWorker|'s [=containing service worker registration=].
1. Set |request|'s [=service-workers mode=] to "`none`".
1. Set |request|'s [=request/cache mode=] to "<code>no-cache</code>" if any of the following are true:
* |registration|'s [=service worker registration/update via cache mode=] is "`none`".
* The [=current global object=]'s [=force bypass cache for importscripts flag=] is set.
* |registration|'s [=last update check time=] is not null and the time difference in seconds calculated by the current time minus |registration|’s [=last update check time=] is greater than 86400.
1. Let |response| be the result of <a lt="fetch">fetching</a> |request|.
1. If |response|’s <a for="response" href="https://github.com/whatwg/fetch/issues/376">cache state</a> is not "<code>local</code>", set |registration|’s [=service worker registration/last update check time=] to the current time.
1. [=Extract a MIME type=] from the |response|'s [=unsafe response=]'s [=response/header list=]. If this MIME type (ignoring parameters) is not a [=JavaScript MIME type=], return a [=network error=].
1. If |response|'s <a>unsafe response</a>'s [=response/type=] is not "<code>error</code>", and |response|'s [=response/status=] is an <a>ok status</a>, then:
1. [=map/Set=] <a>script resource map</a>[|request|'s [=request/url=]] to |response|.
1. Return |response|.
1. Else:
1. If <a>script resource map</a>[|url|] [=map/exists=], return <a>script resource map</a>[|url|].
1. Else, return a <a>network error</a>.
1. If |serviceWorker|'s [=service worker/script resource map=][|request|'s [=request/url=]] [=map/exists=], return the [=map/entry=]'s [=map/value=].
1. If |serviceWorker|'s [=state=] is not *parsed* or *installing*, return a [=network error=].
1. Let |registration| be |serviceWorker|'s [=containing service worker registration=].
1. Set |request|'s [=service-workers mode=] to "`none`".
1. Set |request|'s [=request/cache mode=] to "`no-cache`" if any of the following are true:
* |registration|'s [=service worker registration/update via cache mode=] is "`none`".
* The [=current global object=]'s [=force bypass cache for importscripts flag=] is set.
* |registration|'s [=last update check time=] is not null and the time difference in seconds calculated by the current time minus |registration|’s [=last update check time=] is greater than 86400.
1. Let |response| be the result of [=fetch|fetching=] |request|.
1. Set |response| to |response|'s [=unsafe response=].
1. If |response|’s <a for="response" href="https://github.com/whatwg/fetch/issues/376">cache state</a> is not "`local`", set |registration|’s [=service worker registration/last update check time=] to the current time.
1. [=Extract a MIME type=] from the |response|'s [=response/header list=]. If this MIME type (ignoring parameters) is not a [=JavaScript MIME type=], return a [=network error=].
1. If |response|'s [=response/type=] is not "`error`", and |response|'s [=response/status=] is an <a>ok status</a>, then:
1. [=map/Set=] |serviceWorker|'s [=service worker/script resource map=][|request|'s [=request/url=]] to |response|.
1. Set |serviceWorker|'s [=classic scripts imported flag=].
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I'm a bit confused what about the meaning of include updated resources flag and classic scripts imported flag. Could you help me understand what they are doing?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The classic scripts imported flag is used to indicate the main script has some imported scripts so Update must not return early before actually checking the imported scripts even if the main scripts are byte identical. I named it classic .. because module scripts don't support importScripts() in the first place.

The job's potentially include updated resources flag is used to check if we can return early even without evaluating the script. That is, the script received from the network is byte identical and doesn't import any script. For module scripts, it checks all the responses gotten during fetching the module worker script graph.

The service worker's include updated resources flag is checked when Update couldn't determined from the earlier steps whether it could stop early.

1. Return |response|.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To aid my understanding, what exactly happens in the error response case? Does it result in an error so the installation/update fails, or does it mean we basically treat it like the importScripts() call never happened?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An error response makes it throw a "NetworkError" DOMException in fetch a classic worker-imported script step 4. That again makes Update reject the job promise in https://github.com/w3c/ServiceWorker/pull/1283/files#diff-27b79860afe28f01aed4f1f6228367faR2467. This particular behavior hasn't been changed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. That means if you do try { importScripts('fail.js'); } and catch the exception, you can later try to do importScripts('fail.js') and it'll retry again. I think that matches our implementation too.

(Acknowledged that this hasn't changed, just confirming my understanding.)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if you catch the exception, it just installs it without populating the cache. When the service worker starts up later, importScript('fail.js') will get a network error again that throws and be caught again.

</section>
</section>

Expand All @@ -2069,7 +2069,7 @@ spec: webappsec-referrer-policy; urlPrefix: https://w3c.github.io/webappsec-refe
<section>
<h3 id="privacy">Privacy</h3>

[=/Service workers=] introduce new persistent storage features including <a>scope to registration map</a> (for [=/service worker registrations=] and their [=/service workers=]), [=request response list=] and <a>name to cache map</a> (for caches), and <a>script resource map</a> (for script resources). In order to protect users from any potential <a biblio data-biblio-type="informative" lt="unsanctioned-tracking">unsanctioned tracking</a> threat, these persistent storages *should* be cleared when users intend to clear them and *should* maintain and interoperate with existing user controls e.g. purging all existing persistent storages.
[=/Service workers=] introduce new persistent storage features including <a>scope to registration map</a> (for [=/service worker registrations=] and their [=/service workers=]), [=request response list=] and <a>name to cache map</a> (for caches), and [=service worker/script resource map=] (for script resources). In order to protect users from any potential <a biblio data-biblio-type="informative" lt="unsanctioned-tracking">unsanctioned tracking</a> threat, these persistent storages *should* be cleared when users intend to clear them and *should* maintain and interoperate with existing user controls e.g. purging all existing persistent storages.
</section>
</section>

Expand Down Expand Up @@ -2371,6 +2371,8 @@ spec: webappsec-referrer-policy; urlPrefix: https://w3c.github.io/webappsec-refe
1. Invoke <a>Finish Job</a> with |job| and abort these steps.
1. Let |httpsState| be "<code>none</code>".
1. Let |referrerPolicy| be the empty string.
1. Let |hasUpdatedResources| be false.
1. Let |updatedResourceMap| be an [=ordered map=] where the keys are [=/URLs=] and the values are [=/responses=].
1. Switching on |job|'s <a>worker type</a>, run these substeps with the following options:
<!-- TODO: reorganize algorithm so that the worker environment is created before fetching happens -->
: "<code>classic</code>"
Expand Down Expand Up @@ -2418,10 +2420,28 @@ spec: webappsec-referrer-policy; urlPrefix: https://w3c.github.io/webappsec-refe
1. Else:
1. Invoke [=Reject Job Promise=] with |job| and "{{SecurityError}}" {{DOMException}}.
1. Asynchronously complete these steps with a <a>network error</a>.
1. Set |updatedResourceMap|[|request|'s [=request/url=]] to |response|.
1. If |response|'s <a for="response" href="https://github.com/whatwg/fetch/issues/376">cache state</a> is not "<code>local</code>", set |registration|'s <a>last update check time</a> to the current time.

Issue: The response's cache state concept had been removed from fetch. The fetch issue <a href="https://github.com/whatwg/fetch/issues/376">#376</a> tracks the request to restore the concept or add some similar way to check this state.

1. If |newestWorker| is null, or |newestWorker|'s [=service worker/script resource map=][|request|'s [=request/url=]]'s [=response/body=] is not byte-for-byte identical with |response|'s [=response/body=], set |hasUpdatedResources| to true.
1. Else if |newestWorker|'s [=classic scripts imported flag=] is set, then:
1. [=map/For each=] |url| → |storedResponse| of |newestWorker|'s [=service worker/script resource map=]:
1. Let |request| be a new [=/request=] whose [=request/url=] is |url|, [=request/client=] is |job|'s [=job/client=], [=request/destination=] is "`script`", [=request/parser metadata=] is "`not parser-inserted`", [=request/synchronous flag=] is set, and whose [=request/use-URL-credentials flag=] is set.
1. Set |request|'s [=request/cache mode=] to "`no-cache`" if any of the following are true:
* |registration|'s [=service worker registration/update via cache mode=] is "`none`".
* |job|'s [=force bypass cache flag=] is set.
* |registration|'s [=last update check time=] is not null and the time difference in seconds calculated by the current time minus |registration|’s [=last update check time=] is greater than 86400.
1. Let |fetchedResponse| be the result of [=fetch|fetching=] |request|.
1. Set |fetchedResponse| to |fetchedResponse|'s [=unsafe response=].
1. Set |updatedResourceMap|[|request|'s [=request/url=]] to |fetchedResponse|.
1. If |fetchedResponse|'s <a for="response" href="https://github.com/whatwg/fetch/issues/376">cache state</a> is not "<code>local</code>", set |registration|’s [=last update check time=] to the current time.
1. [=Extract a MIME type=] from the |fetchedResponse|'s [=response/header list=]. If this MIME type (ignoring parameters) is not a [=JavaScript MIME type=], asynchronously complete these steps with a [=network error=].
1. If |fetchedResponse|'s [=response/type=] is "`error`", or |fetchedResponse|'s [=response/status=] is not an [=ok status=], asynchronously complete these steps with a [=network error=].
1. If |fetchedResponse|'s [=response/body=] is not byte-for-byte identical with |storedResponse|'s [=response/body=], set |hasUpdatedResources| to true.

Note: The control does not break the loop in this step to continue with all the imported scripts to populate the cache.
1. Return true.

If the algorithm asynchronously completes with null, then:
Expand All @@ -2434,20 +2454,20 @@ spec: webappsec-referrer-policy; urlPrefix: https://w3c.github.io/webappsec-refe
1. Invoke <a>Finish Job</a> with |job| and abort these steps.

Else, continue the rest of these steps after the algorithm's asynchronous completion, with |script| being the asynchronous completion value.

1. If |newestWorker| is not null, |newestWorker|'s [=service worker/script url=] [=url/equals=] |job|'s [=job/script url=], and |script|'s [=source text=] is a byte-for-byte match with |newestWorker|'s [=script resource=]'s [=source text=], if |script| is a [=classic script=], and |script|'s [=module script/module record=]'s \[[ECMAScriptCode]] is a byte-for-byte match with |newestWorker|'s [=script resource=]'s [=module script/module record=]'s \[[ECMAScriptCode]] otherwise, then:
1. If |hasUpdatedResources| is false, then:
1. Invoke [=Resolve Job Promise=] with |job| and |registration|.
1. Invoke [=Finish Job=] with |job| and abort these steps.
1. Let |worker| be a new [=/service worker=].
1. Set |worker|'s [=service worker/script url=] to |job|'s [=job/script url=], |worker|'s <a>script resource</a> to |script|, and |worker|'s [=service worker/type=] to |job|'s <a>worker type</a>.
1. [=map/For each=] |url| → |response| of |updatedResourceMap|:
1. Set |worker|'s [=service worker/script resource map=][|url|] to |response|.
1. Set |worker|'s <a>script resource</a>'s <a>HTTPS state</a> to |httpsState|.
1. Set |worker|'s <a>script resource</a>'s [=script resource/referrer policy=] to |referrerPolicy|.
1. Invoke <a>Run Service Worker</a> algorithm given |worker|, and with the *force bypass cache for importscripts flag* set if |job|'s [=job/force bypass cache flag=] is set.
1. If an uncaught runtime script error occurs during the above step, then:
1. Invoke [=Reject Job Promise=] with |job| and `TypeError`.
1. If |newestWorker| is null, invoke <a>Clear Registration</a> algorithm passing |registration| as its argument.
1. Invoke <a>Finish Job</a> with |job| and abort these steps.
1. Else:
1. Let |worker| be a new [=/service worker=].
1. Set |worker|'s [=service worker/script url=] to |job|'s [=job/script url=], |worker|'s <a>script resource</a> to |script|, and |worker|'s [=service worker/type=] to |job|'s <a>worker type</a>.
1. Set |worker|'s <a>script resource</a>'s <a>HTTPS state</a> to |httpsState|.
1. Set |worker|'s <a>script resource</a>'s [=script resource/referrer policy=] to |referrerPolicy|.
1. Invoke <a>Run Service Worker</a> algorithm given |worker|, and with the *force bypass cache for importscripts flag* set if |job|'s [=job/force bypass cache flag=] is set.
1. If an uncaught runtime script error occurs during the above step, then:
1. Invoke [=Reject Job Promise=] with |job| and `TypeError`.
1. If |newestWorker| is null, invoke <a>Clear Registration</a> algorithm passing |registration| as its argument.
1. Invoke <a>Finish Job</a> with |job| and abort these steps.
1. Invoke <a>Install</a> algorithm with |job|, |worker|, and |registration| as its arguments.
</section>

Expand Down Expand Up @@ -2509,7 +2529,6 @@ spec: webappsec-referrer-policy; urlPrefix: https://w3c.github.io/webappsec-refe
1. Run the <a>Update Registration State</a> algorithm passing |registration|, "<code>installing</code>" and null as the arguments.
1. If |newestWorker| is null, invoke <a>Clear Registration</a> algorithm passing |registration| as its argument.
1. Invoke <a>Finish Job</a> with |job| and abort these steps.
1. Set |registration|'s <a>installing worker</a>'s <a>imported scripts updated flag</a>.
1. If |registration|'s <a>waiting worker</a> is not null, then:
1. [=Terminate Service Worker|Terminate=] |registration|'s [=waiting worker=].
1. Run the [=Update Worker State=] algorithm passing |registration|'s [=waiting worker=] and *redundant* as the arguments.
Expand Down