Add support for manually triggering service workers being suspended to allow automated testing

[JWorthe]

Background

We have a fairly large browser extension which we are migrating from supporting only MV2 to supporting both MV2 and MV3.

We already have a fairly extensive set of automated tests that use Selenium Webdriver to launch different browsers with our extension installed and verify that our functionality works correctly.

One of the changes we need to make is that, in MV3, the background page is being replaced with a service worker. As the documentation says, service workers may be suspended by the browser under various circumstances, such as when it is inactive. When it is suspended, the service worker still maintains its event listeners, and may be woken up by an event. This detail requires many changes in our code base because, when a service worker is suspended, in-memory storage like variables are cleared. Some of these changes are subtle, and so need to be supported by good automated tests.

Missing Functionality / Required Changes

I have been trying to write automated tests to demonstrate bugs where we do not handle the service worker being suspended correctly, and verify the fixes when I implement them. On a high level, the tests that I want to write follow this structure:

1. The service worker starts up, adds event listeners, and does some work
2. The service worker is suspended
3. The service worker is sent an event, and responds correctly

There does not currently appear to be a reliable cross-browser way to do step 2, suspending the service worker on command.

Example Use Case

As an example use case, this is a snippet of code from Chrome's Migrating from background pages to service workers guide

// background.js

// Don't do this! The service worker will be created and destroyed over the lifetime of your
// exension, and this variable will be reset.
let savedName = undefined;

chrome.runtime.onMessage.addListener(({ type, name }) => {
  if (type === "set-name") {
    savedName = name;
  }
});

chrome.browserAction.onClicked.addListener((tab) => {
  chrome.tabs.sendMessage(tab.id, { name: savedName });
});

As the snippet indicates, using variables to persist state may be unreliable. I would like to be able to write a unit test which demonstrates this not working.

[dotproto]

My first thought is that this seems like something that will need to be adresse by Selenium/WebDriver, though obviously this group has a vested interest in pursuing these capabilities. The WebDriver standard is authored by the Browser Testing and Tools Working Group. This seems like as good to reach out to them in order to figure out how our respective groups can collaborate to improve browser extension testing.

[dotproto]

Last Friday I posted to the public-browser-tools-testing mailing list to start discussing cross-group collaboration.

[JWorthe]

I think the suggestion to include this in broader extension support for webdriver is the right direction for a long term fix, but it doesn't help me right now today with writing tests to explore how our extension currently responds to the service worker being suspended at various points.

What are my options for a hacky way to test this functionality, or quick and dirty additions the that could be added to the API to make testing this possible? For example, maybe a function like chrome.runtime.suspendServiceWorker could be added?

Regarding workarounds, on another thread, I saw someone suggest using chrome://serviceworker-internals/ to stop the service worker. If I use webdriver to open a tab to chrome://serviceworker-internals/ and click the 'stop' button, is that equivalent to the service worker being suspended or is there some nuance that I should be aware of? My understanding is that 'suspended' and 'terminated' aren't quite the same thing, so this might not actually be testing the right thing.

[tophf]

Service workers are simply terminated, not suspended.

[JWorthe]

I've tried to create a test case that uses Webdriver to click the 'stop' button on chrome://serviceworker-internals/, but this does not work as expected. After the service worker has been stopped, it does not start again when my extension page calls chrome.runtime.sendMessage.

This repo is a minimal reproduction of the issue I found in testing: https://gitlab.com/j.wernick/service-worker-webdriver-test/-/tree/main.

[JWorthe]

I've noticed that my test does what I expect if I go through the steps manually, but the service worker never reactivates if I'm running my test with Webdriver. That has lead me to believe that this is actually a Chrome bug, and so I've opened a bug in Chrome's issue tracker: https://bugs.chromium.org/p/chromium/issues/detail?id=1325792