one-shots via background Fetch

[jkarlin]

This is a continuation of #18 but focused on the one-shot case.

What if Fetch had a 'background' option that fired a service worker event when complete (if one is registered). There would be some way to bundles fetches together (e.g., by tag) so that only one event would fire per given bundle of fetches.

There are some advantages to this:

1. You could perform a background Fetch without having a registered Service Worker at all if you didn't need to perform any action post-fetch. This is useful for simply uploading changes to the server. You could then query the server on next page-load to see if it received the change.
2. A service worker doesn't need to be open while the fetch is occurring. This is great because:
   • This reduces resource usage during download
   • The SW is only spun up on final success or failure. This avoids potential thrashing of starting/stopping service workers as the network connectivity goes up and down.
   • The UA knows exactly what resources it needs to download in the background and can display them in UX if necessary
3. The API is a bit cleaner as it removes the registration step.

For privacy you'd want to restrict background fetches to be HTTPS. You would still need to put time and retry caps on them for privacy sake. For longer downloads you might require a permission or a notification (the notification could be another option to the background fetch).

Open questions:

1. How long would the backgroundFetchFinished event get to run for?
2. How would you perform a series of transactions in the background?
3. Would/Should there be a way to terminate an ongoing background fetch?

A simple example:

Document:

fetch('https://example.com/upload', {
  method: 'post', 
  body: 'foo', 
  background: true, 
  backgroundTag: 'outbox'
})
.then(function(response) {
  console.log("Outbox sent in the foreground");
})
.catch(function(err) {
  console.log("Outbox failed to send in the foreground");
});

Service Worker:

self.addEventListener('backgroundFetchFinished', function(event) {
  // event.response has the response if needed
  var mode = event.finishedInForeground ? 'foreground' : 'background';
  if (event.backgroundTag == 'outbox') {
    if (event.err == null) 
      console.log('Outbox sent in the ' + mode);
    else
      console.log('Outbox failed to send in the ' + mode);
  }
});

[jkarlin]

@jakearchibald noted offline that cache.backgroundAdd makes more sense as that way the response has somewhere to go without an extra copy. And you can check the cache to see if the fetch succeeded yet on the next page load. If there is no cache entry then the background fetch is still trying. This seems like a useful API for large downloads.

My question is, is background fetch/cache.backgroundAdd useful enough that we don't actually need background sync?

[jakearchibald]

Here's how a chat app might use sync:

1. User offline
2. User sends chat message
3. Sync registered "chat"
4. User sends chat message
5. Sync registered "chat", deduped
6. User sends chat message
7. Sync registered "chat", deduped
8. User changes avatar photo
9. Sync registered "profile"
10. User online
11. Sync event for "profile", get profile data from IDB, send data using fetch
12. Sync event for "chat", get unsend chat messages, send them in single fetch

A higher level API wouldn't know the "profile" sync and "chat" sync are unrelated to the point where order doesn't matter. A higher level API wouldn't be able to coalesce the "chat" sends into a single send (if we fix #104).

[jakearchibald]

My wikipedia example requires fetching data, then reading it to find more things to fetch (the images). This would be messy without cache.backgroundAddAll to do batch atomic fetching.

Aside from my previous post, I don't have much counter-argument aside from chanting "extensible web".

Maybe @slightlyoff can talk us down?

[mkruisselbrink]

A higher level API wouldn't be able to coalesce the "chat" sends into a single send

Why not? If we also provide some API to look at and cancel pending background fetches, the chat app could still do the legwork to coalesce the chat sends. Some care should be taken to make sure updates to this background fetch happen in a safe way without race conditions, but that seems like it wouldn't be too hard.

[jkarlin]

My wikipedia example requires fetching data, then reading it to find more things to fetch (the images). This would be messy without cache.backgroundAddAll to do batch atomic fetching.

cache.backgroundAddAll nicely takes care of the 'bundle fetches together' issue.

But how does cache.backgroundAddAll help? Don't you still need to download the html to figure out what images to load? We would need to be able to create more fetches in the event handler, which is problematic from a privacy perspective.

[jakearchibald]

We would need to be able to create more fetches in the event handler, which is problematic from a privacy perspective.

Yeah, I guess I didn't think it'd be a problem since you can create requests from a SW anyway. I guess the delayed nature makes this a problem.

[jakearchibald]

If we also provide some API to look at and cancel pending background fetches

Good point

[clelland]

I really like this, at least in the abstract :)

@jkarlin, for your questions:

1. I suspect that in Chrome, we could easily start by applying the 3 or 5 minute timeout that we're using for onsync events right now. It should be specced as 'UA-defined', to give browsers the freedom to use whatever heuristics are available to them. (Site trustworthiness, user engagement, or other crowdsourced wisdom about how long sites should get here)
2. I like the idea of parallelizing them the transactions by tag -- I'm having a really hard time thinking of a way to make them serial, or add any additional logic, if there's no way to launch a new background fetch from an onBackgroundFetchFinished handler running in the background (which I think is sensible)
3. If it is possible to terminate a fetch in the foreground, then I don't see why we couldn't do it in the background -- but I think it would require a site to do something like this:

/* Document */
fetch('https://example.com/download', {
  method: 'get', 
  background: true, 
  backgroundTag: 'dotcom'
});
fetch('https://example.net/download', {
  method: 'get', 
  background: true, 
  backgroundTag: 'dotnet'
});

/* Service Worker */
self.addEventListener('backgroundFetchFinished', function(event) {
  // whichever one finished, cancel the other
  if (event.err == null) 
    if (event.backgroundTag == 'dotcom') {
      registration.sync.getFetch('dotnet').then(function(pendingFetch) {
        pendingFetch.cancel();
      });
    } else if (event.backgroundTag == 'dotnet') {
      registration.sync.getFetch('dotcom').then(function(pendingFetch) {
        pendingFetch.cancel();
      });
    }
  }
});

My question is, is background fetch/cache.backgroundAdd useful enough that we don't actually need background sync?

That's an excellent question :) It takes care of probably 90% of the use cases that we've seen. They're all around uploading/downloading data gracefully in the offine case.
Are there other cases that we've neglected? minRequiredPower (or avoidDraining or however we're spelling it these days) enables another class of use that I don't think would be covered by this (as does periodic, but that's looking more and more like another API at this point).

[clelland]

A couple of other questions:

1. Given that we're expecting this to handle an unreliable network, is the network stack expected to be able to handle recovery of partially complete requests on its own, or is some guidance from the app or assistance from the service worker required in that case?

   (I'm thinking of a long image upload on a slow network that gets cut off right near the end -- can we avoid restarting the entire transfer?)

2. Can POST or PUT requests with large bodies survive, say, a device restart, with this model? Or do we need the app to use IDB to store the body, and have something like an onBackgroundFetchFailedCatastrophically event to tell the service worker to retry the request?

[ithinkihaveacat]

1. Is it possible to change cache.add() to work like the suggested cache.backgroundAdd()? I suspect this changes the expected behavior too much, but not adding to the API surface (and not needing to use the misleading word sync) would be awesome.
2. Would there be any way to check for sync completion aside from essentially polling the cache via Cache.match()? This makes things a bit more awkward, especially in cases where the endpoint is dynamic. (Upload to a fixed endpoint would be fine, but if the sync is ultimately a request for something like http://foo.com/search?q=pizza or http://bar.com/posts/comments/33241 the caller needs to separately keep track of the active requests.)

[wibblymat]

A thought about non-network use-cases for sync.

I have a Cache that I would like to limit to holding only the 50 most recent items. Every time I add something to the cache while handling a fetch event, I also run some code that updates IDB to keep track of what was cached (and when), and if the count of items is > 50, remove the oldest items.

This has to happen as part of the promise returned to event.respondWith, otherwise the service worker can be killed before the cache invalidation code has run.

You could use sync to make this a little more efficient - request a sync each time you add to the cache and then use the sync handler to do the cache invalidation. This would, in a hacky way, allow some batching of the invalidation code if several fetch handlers are run before the first sync fires. It also means that the fetch events aren't being blocked by it.

This seems like the sort of thing people might try to use sync for that wouldn't be possible with cache. backgroundAddAll. Perhaps if the current sync spec goes away there would be room for a slightly different API that enables these use-cases. Something like requestIdleCallback for service workers.

I'm sort of imagining that it might be an event that fires when a running service worker is about to be killed (or similar), rather than ever waking a stopped service worker.

[mkruisselbrink]

@wibblymat in the spec FetchEvent has a waitUntil, exactly for these kinds of things where you want to do work after responding (at least that's why I think it's there). But yeah, that's currently not implemented in chrome.
It also seems that for your usecase you might not actually want the current behavior of background sync where it (tries to) only call the sync event when it thinks there is a network connection?

[wibblymat]

@mkruisselbrink Yes, I guess I've sort of conflated a few things here.

I wanted some API that lets me say "I have some work to do. Possibly there is a batch of similar things. Please give me an opportunity to do some background work in the future. Preferably not right away, but rather when things are quiet."

Background sync doesn't give me that, but it is close, so could be used to fake it. For the particular use case I mentioned the network availability isn't even that much of a problem - the task is only triggered by a successful network request being stored in the cache, so 99% of the time the sync will fire without much delay.

But that isn't really relevant. My point, which I failed to make well, was that any non-network use-cases for background sync would probably be better served by some other API around scheduling background work. fetch(r, {background: true}) or cache.backgroundAddAll(r) should be able to handle the network part just fine.

[jkarlin]

Let's move the non-network processing to issue #77 .

I'm closing this issue as there are some major concerns that Background Fetch (or cache) failed to address.

1. How would background fetch run requests sequentially?

• e.g., If fetch 3 out of 4 failed to send, how should that be handled? Should it continue to send 4?
  2) How would background fetch handle resumable uploading?
• There isn't a resumable upload standard that servers implement. Background Sync lets you create your own protocol with JavaScript.

There are likely many other use cases that a low-level API like Background Sync can address that the background fetch API can't.