Storage Types Covered

[johnwilander]

Mozilla's documentation on differences between Safari's and Firefox's implementation covers the difference explained below.

As mentioned in the original WHATWG issue on the Storage Access API, the naming implies that not only cookie access can be managed by this API but also other storage APIs such as LocalStorage and IndexedDB.

Safari: The Storage Access API only controls cookie access. All other storage APIs are either permanently partitioned (double-keyed) and ephemeral or blocked in third-party contexts.

Firefox: The Storage Access API controls all means of storage.

One of the main reasons for not covering more as of now in WebKit is that we have a shipping, privacy-preserving solution with our partitioning and switching it through the Storage Access API may cause breakage without progressing privacy. We need compelling use cases to consider effectively regressing these 7-year old privacy protections. Browser interoperability could be compelling, especially if we could get agreement across the board.

[Brandr0id]

Edge has seen that with other storage restriction options if they are not uniformly applied (or removed) to all storage types it can cause compatibility issues with sites. The issue we have seen is that some sites would check access to one type of storage (e.g. cookies or localstorage) and assume that all other storage had the same access/state. If this was not the case the site can throw errors or perform operations assuming they are able to access storage.

I think ensuring that the API uniformly applies to all storage makes sense from a web compatibility POV.

[johnwilander]

Edge has seen that with other storage restriction options if they are not uniformly applied (or removed) to all storage types it can cause compatibility issues with sites. The issue we have seen is that some sites would check access to one type of storage (e.g. cookies or localstorage) and assume that all other storage had the same access/state. If this was not the case the site can throw errors or perform operations assuming they are able to access storage.

I see. I would not be surprised if we saw the opposite since partitioned storage is so well established in Safari.

I think ensuring that the API uniformly applies to all storage makes sense from a web compatibility POV.

Yeah, this is the good argument for a change on our part.

[othermaciej]

The name certainly suggests it will affect all storage, so it's surprising that it doesn't. And it doesn't seem like user-granted access to the other storages creates extra risk. So I'd suggest affecting all storage types. (Or perhaps an enumerated set of explicit storage types? I'm not sure we need or want this to undo http case partitioning for instance. And it might be ambiguous what "all" means without enumerating.)

[annevk]

In Firefox it's definitely not all, to be clear. We want to have a distinction between what's traditionally been origin-based storage and, for lack of a better word, network caches (which would not have a way to use the first-party key).

Storage access would affect what's labeled Site at https://storage.spec.whatwg.org/#infrastructure.

Cookies are the oddball here and require special consideration.

[annevk]

The Firefox approach does mean that a largish number of features are affected (localStorage, BroadcastChannel, Indexed DB, ...) for which transition behavior needs to be defined (from "blocking" to "first-party" and from "third-party" to "first-party", aiui). The transition in Firefox is one-way and global (modulo Firefox Containers) at the moment so there shouldn't be privacy implications.

[othermaciej]

A one-way global transition from a state of partitioned storage to unpartitioned still has a privacy implication: it allows IDs from each partition to be copied intounpartitioned state and associated server-side. (Simply store in a variable before requesting storage access, then read it, then tell the server those two IDs are the same.)

If cookies are blocked rather than partitioned in a third-party context (as WebKit currently does), this same risk doesn't arise for data of the same type if only cookies are granted first party access. However, the same trick could be played by combining partitioned LocalStorage with unpartitioned cookies.

It seems like the only way to avoid this dilemma is to make the partitioned versions of non-cookie storages either blocked or both partitioned and ephemeral.

[annevk]

That sounds correct. "third-party" is scoped by the top-level, which helps in terms of exposure, but making it more ephemeral and recommending it to be cleared more often would be good.

(I don't think there's anything you can do during the transition to prevent this, even reloading and hoping there's not enough fingerprintable bits and the reload itself is not enough of a tell does not seem impactful and would give a poor experience.)

[johnwilander]

If we are considering more than cookies to be mandatory to change with granted storage access, we need to have a plan for what to do for engines that don’t block third-party storage by default but instead partition it.

In the case of IndexedDB, there may be open connections to partitioned IDB when storage access is granted. Do we then sever those connections? With some kind of error? Or do we allow concurrent connections to partitioned and non-partitioned IDB?

And it gets worse if we also consider storage access to be granted for all matching iframes on the page since the other iframes will typically have no clue that one of them is requesting storage access. This would mean all iframes have to be ready at all times for storage access to change underneath them. At that point we’re almost into broadcasting territory.

[annevk]

Yeah, that's transitioning from "third-party" to "first-party" above. Firefox's implementation provides possible answers here, but we're open to changing the details. (Those frames all share an event loop, so it's not that bad I think.)

cc @ehsan @bakulf

[ehsan]

In Firefox in order to avoid having to answer such tricky questions we currently don't have a transition from partitioned third-party storage to first-party. The only transition supported is from blocked third-party storage to granted first-party storage.

[johnwilander]

In Firefox in order to avoid having to answer such tricky questions we currently don't have a transition from partitioned third-party storage to first-party. The only transition supported is from blocked third-party storage to granted first-party storage.

Thanks, Ehsan. That was my suspicion since it is indeed a tricky situation.

[johnwilander]

The reason why I said "effectively regressing these 7-year old privacy protections" is that WebKit's third-party LocalStorage is not only partitioned but also ephemeral, i.e. third-party LocalStorage goes away on browser quit. Allowing access to first-party LocalStorage means third-parties get access to persistent LocalStorage and since LocalStorage has no expiry function, it is "forever" => worse for privacy.

(Some layman comments on legal things: We were told that some developers are prohibited from using LocalStorage because you can't have a reasonable privacy policy tied to client-side storage that doesn't have an expiry mechanism. Cookies may also have more legal restrictions on them than other means of storage by virtue of their long legacy.)

[hober]

Spitballing here. Suppose requestStorageAccess simply rejects if the calling document has already accessed any of its storage. This lets you avoid a partitioned → unpartitioned transition. If you're okay with partitioned storage, you just go ahead and access it. If you want unpartitioned, you have to hold off on accessing storage until the promise resolves before accessing it.

[johnwilander]

Spitballing here. Suppose requestStorageAccess simply rejects if the calling document has already accessed any of its storage. This lets you avoid a partitioned → unpartitioned transition. If you're okay with partitioned storage, you just go ahead and access it. If you want unpartitioned, you have to hold off on accessing storage until the promise resolves before accessing it.

I could see that working for per-frame storage access. But if we want granted storage access to apply to all matching subresources for the whole page, I don't know if we can require them all to hold off touching partitioned storage because one of them might request access to first-party storage.

Also, I have a concern with blocking instead of partitioning. I believe the way Firefox is doing that is based off of a list of "bad actors" that Firefox has decided it can subject to storage blocking. While that works for tracking prevention as we know it today, I don't think we can base a standard off of the likes of such a list because it keeps us in the weird world of some third-parties having storage access by default and some not.

Blocking would have to be done for all third parties if it's to be a solution to anything here. If we block all third-party storage by default, storage access across all matching subresources becomes easy.

[michael-oneill]

I agree, block all third-party storage access unless the user has been properly informed of who wants it and why, and has indicated their consent. Nobody is in a position to second-guess that.

I also think all storage, partitioned or not, should have an expiry with a reasonable maximum enforced by browsers, i.e. an automatic virtual Clear-Site-Data after a period of no use.

[johnwilander]

I agree, block all third-party storage access unless the user has been properly informed of who wants it and why, and has indicated their consent. Nobody is in a position to second-guess that.

I don't think we're agreeing on doing that, rather just saying 1) that blocking is Firefox's current solution based on a blocklist, and 2) that blocking of all third-party storage is a general solution to the transition problem. So it's an option.

The second option was described by @hober, namely letting the requesting third-party iframe voluntarily hold off touching partitioned storage until it has been granted (first-party) storage access. This one works with per-frame storage access but potentially gets hard for developers with per-page storage access.

The third option is some kind of canceling of all ongoing storage activities which I believe is worst for IndexedDB with its connections since all other storage forms are synchronous and atomic from a JavaScript standpoint (true?). This again should work for per-frame storage access but could get complex for developers in a per-page scenario.

More options?

For all of the above, we must recognize that the Storage Access API itself is asynchronous which means we have to handle attempts at partitioned storage access while the Storage Access API is processing. 😒

I also think all storage, partitioned or not, should have an expiry with a reasonable maximum enforced by browsers, i.e. an automatic virtual Clear-Site-Data after a period of no use.

Personally, I agree. But that's out of scope for the Storage Access API. 🙂

[jackfrankland]

As another option, the granting of access could be limited to only network requests. Non-expired cookies which were set by the server with the flags HttpOnly, and not SameSite=Strict/Lax, are now allowed to be sent with subsequent requests. Any new cookies being set by the server (also HttpOnly and not SameSite) will be allowed to be set. This fulfills the need to allow user-granted sessions to continue in a third-party context.

Any script-accessible storage state does not need to be affected, and the browser's normal rules (partitioned/blocked/otherwise) can continue to apply.

[othermaciej]

I think current users of Storage Access API need access to state from script, so they can dynamically change the behavior in an already-loaded embedded widget.

[jackfrankland]

In order to retrieve the session data, would you not need to make a subsequent request after the user has granted access? The initial request to the server would not have included the session cookie.

[othermaciej]

The cookie may already contain everything you need to know, but this is a good point. I guess we should research how existing Storage Access API adopters use it.

[johnwilander]

As another option, the granting of access could be limited to only network requests. Non-expired cookies which were set by the server with the flags HttpOnly, and not SameSite=Strict/Lax, are now allowed to be sent with subsequent requests. Any new cookies being set by the server (also HttpOnly and not SameSite) will be allowed to be set. This fulfills the need to allow user-granted sessions to continue in a third-party context.

Any script-accessible storage state does not need to be affected, and the browser's normal rules (partitioned/blocked/otherwise) can continue to apply.

Let's see if I understand. For WebKit this would mean …

Before granted storage access: Blocked cookies and partitoned web storage.

After granted storage access: Send cookies in HTTP requests, blocked document.cookie, and partitioned web storage.

Correct?

[jackfrankland]

Yes, that's correct

[johnwilander]

That's very easy to implement, that's for sure. And it locks in even further on the authenticated embeds case, i.e. granted storage access opens up for auth cookies (or HTTP State Tokens) to be sent but doesn't change anything for scripts.

[othermaciej]

I interpreted the suggestion to mean only HTTPOnly cookies are unblocked (for both sending and receiving) rather than ask cookied.

[jackfrankland]

I interpreted the suggestion to mean only HTTPOnly cookies are unblocked (for both sending and receiving) rather than ask cookied.

I was thinking of a situation where document.cookie could be set by a script in different contexts, like:

• social.example first-party cookie
• social.example cookie on blog1.example
• social.example cookie on blog2.example

In this case, which cookies would get sent after granting access? By only allowing HttpOnly cookies, it's a way of guaranteeing that the cookie was set by the server, rather than by document.cookie, and avoids having to make this decision. In the WebKit example above, where document.cookie remains blocked in a third-party context, it's maybe not much of a concern - the only question would be whether or not the first-party document.cookie would be sent with the request.

[jackfrankland]

As an alternative, perhaps document.cookie could become read-only in a third-party context instead?

[ehsan]

I agree, block all third-party storage access unless the user has been properly informed of who wants it and why, and has indicated their consent. Nobody is in a position to second-guess that.

I don't think we're agreeing on doing that, rather just saying 1) that blocking is Firefox's current solution based on a blocklist, and 2) that blocking of all third-party storage is a general solution to the transition problem. So it's an option.

It is definitely an option, though for Firefox I doubt it is going to be web compatible in the form of just outright blocking all third-party storage, based on the results of the last study we ran on it.

The second option was described by @hober, namely letting the requesting third-party iframe voluntarily hold off touching partitioned storage until it has been granted (first-party) storage access. This one works with per-frame storage access but potentially gets hard for developers with per-page storage access.

I think this is pretty difficult for Gecko to implement as things are right now, though that is something that we can potentially bite the bullet on if we need to. In Gecko we define "storage" (for the purposes here) as accessing anything that offers you some type of potential persistence or communication with other instances of your origin, which can include a bunch of internal implementation details. For example if you open an HTTP connection that accesses the HTTP cache, you're accessing "storage". We also currently partition cookies. So practically it may be hard for pages to avoid accessing storage very early on.

But more importantly for web developers I wonder how realistic it is to assume they can ensure no library or third-party code would run and try to access a storage API before some code of their own finishes running asynchronously?

The third option is some kind of canceling of all ongoing storage activities which I believe is worst for IndexedDB with its connections since all other storage forms are synchronous and atomic from a JavaScript standpoint (true?).

Yes, also for the Cache API which is also asynchronous, though in Gecko AFAIK DOM storage is asynchronous behind the hoods too.

This again should work for per-frame storage access but could get complex for developers in a per-page scenario.

Ideally we would need to define how this all would serialize in relation to each other, noting that these pages may also run in different content processes.

[jameshartig]

It is definitely an option, though for Firefox I doubt it is going to be web compatible in the form of just outright blocking all third-party storage, based on the results of the last study we ran on it.

@ehsan Can you share the results of that study? As a developer, I'd find the blocking option to be the easiest to understand. It appears Webkit already has been doing partitioned storage so there might be implementations assuming that exists which would break but it seems like blocking access until granted seems like the clearest path forward. It would be very difficult to prevent anything from touching storage before requesting access since there are many different parties (different internal teams or third-parties) on the page at the same time. Without a standardized way to synchronize all of those scripts it feels like there's going to be a lot of mistaken accesses that now locked you out of the intended path.

As another option, the granting of access could be limited to only network requests. Non-expired cookies which were set by the server with the flags HttpOnly, and not SameSite=Strict/Lax, are now allowed to be sent with subsequent requests. Any new cookies being set by the server (also HttpOnly and not SameSite) will be allowed to be set. This fulfills the need to allow user-granted sessions to continue in a third-party context.

Any script-accessible storage state does not need to be affected, and the browser's normal rules (partitioned/blocked/otherwise) can continue to apply.

This seems more confusing to a developer than what currently exists in my opinion. If a service doesn't/can't use header-sent cookies presently but do use the Storage Acesss API to read the cookie in JS this change would break and potentially require re-architecting in order to switch to server-side reading of cookies.

[johannhof]

An update on Firefox: Our implementation of storage partitioning plus Storage Access API (called dFPI) has been enabled in Nightly for a while and we're planning to ship to more users soon (starting with Strict Mode). In our current implementation, granting storage access will simply switch storage to its non-partitioned ("first party") version.

As Ehsan mentioned above this covers (sans bugs) "anything that offers you some type of potential persistence or communication with other instances of your origin". As such things like BroadcastChannel and ServiceWorkers are partitioned and will be un-partitioned on storage access grant, though we might revisit our strategy here based on ongoing discussions. Trying to use existing partitioned IndexedDB connections after a storage access grant will, as far as I understand, throw an error because the connection is severed at the time of the switch.

When storage access is granted in our implementation, frames of the same origin in the same tab will also immediately be switched to non-partitioned storage, which (as folks here mentioned) can lead to subtle racing bugs in certain cases though we haven't really seen those happening so far.

Besides privacy concerns we are mostly interested in what causes least confusion for web developers and/or the least web breakage, and are happy to iterate on our approach based on that.

For this specific issue to be actionable, I think we have to figure out how it relates to https://github.com/privacycg/storage-partitioning. Maybe most of the discussion needs to be had there, including how the transition from partitioned to non-partitioned should work exactly.

[jackfrankland]

I'd like to re-suggest that perhaps only HttpOnly cookies (and HTTP State Tokens) should be in scope. I think it greatly simplifies the problem of transitioning between partitioned and unpartitioned state (by almost avoiding it altogether), and as mentioned above, locks in further the use-case of allowing storage access for authenticated embeds.

Maybe another argument to not have JS accessible storage in scope is that it would give scripts from other parties access to storage that they may be using to correlate user ids with.

[annevk]

I certainly see the appeal, but at that point you essentially require server-side synchronization of storage and server-side management of user identity for embeddable applications. I.e., embeddable applications will need a login function (or get a token from the embedder) and then download any relevant user data for them to function.

[jackfrankland]

Perhaps that's OK?

It would mean that the scope of behaviour of third party embeds would be closer to that of a cross-browser or cross-device user, where of course the third party could not get access to non-server-side storage. The main benefit of requestStorageAccess, then, would be that it gives the user the ability to be authenticated in the embedded content (or recognised as the same user as a server-stored profile).

If there are further legitimate use cases for access to JS storage, perhaps requestStorageAccess could be extended with options? Or, if a document hasStorageAccess, it can electively switch to the unpartitioned JS storage for that document only via a different API.

[dlongley]

@jackfrankland,

Preventing the third party from getting access to non-server-side storage would be harmful to our use case:

https://github.com/digitalbazaar/credential-handler-polyfill/blob/master/README.md

We prefer the credential mediator component of the polyfill to be implemented as a static website -- where the mediator application (and its storage) runs entirely in the browser. Requiring server-side infrastructure for it would cause an unnecessary increase in cost and attack surface.

[johannhof]

We will update the storage section to explicitly list the ways that cookies and/or storage may be affected by the storage access API and that the exact mechanism and type of storage is implementation-defined. This will happen in #31 so I'll close this issue. I know that we're not entirely done with the conversation around storage types but this issue has become a bit of a mess, so please file new issues for individual follow-up topics.