Support mocking WebSocket APIs

[Doesntmeananything]

Is it possible to use msw to mock server-sent events (SSE) and WebSocket (WS) connections?

My use case is to have a somewhat full client-side mocking story (including REST, SSE, and WS functionality), and since msw is such a joy to use when mocking out REST APIs, I was wondering if it makes sense to use it to mock more specialised server interactions.

Have you thought about this? I admit that I haven't looked much into whether it's possible just to use custom request handlers to add this functionality, emulating SSE and WS behaviour in some way. I wanted to know if you already had something in mind regarding this question. Thanks!

[kettanaito]

Hey, @Doesntmeananything, thanks for bringing this topic up. I'd love to bring SSE and WS support to the users of MSW. I admit I haven't researched the topic yet, but would use this thread for this.

Technically, it comes down to the ability of Service Worker to intercept those requests/events. If the spec supports it, there shouldn't be much changes needed on the MSW side.

Here's some useful resources:

• should EventSource bypass service worker interception? w3c/ServiceWorker#885, EventSource interception in Service Worker.
• Should EventSource and WebSocket be exposed in service workers? w3c/ServiceWorker#947, a discussion whether Service Worker should access WebSocket events.

Could you please try to set up a proof of concept, if those events can be intercepted in the worker's fetch event?

You're right about the custom request handler, we can use it to log all intercepted requests:

setupWorker(
  {
    predicate(req) {
      console.log(req)
      // log captured requests, but bypass them
      return false
    },
    resolver: () => null
  }
)

If we confirm it working, I'd be glad to discuss the API for a dedicated request handler for WebSocket/SSE. I can read on their specification meanwhile.

[Doesntmeananything]

Sounds like a plan! At a cursory glance, it does indeed seem quite doable. Let me PoC this, and I'll get back to you with my results as soon as I can.

[Doesntmeananything]

Hi, @kettanaito! I've set up a (very quick and dirty) repository to test these interactions over at https://github.com/Doesntmeananything/msw-sse-ws.

My initial findings are the following:

• I am able to intercept SSE connections, however this occurs only when a stream ends. For example, if a stream consists of 3 messages, logging happens only after all 3 messages have been received, including the final end event.
• I am not able to intercept WS connections. I tried to make sure that the WS connection is established from the client only after the mock service worker had been initialised, but it didn't seem to help. I want to investigate this further.

I'm a bit concerned about WS events, although I hope that with some additional work it'd possible to intercept them.

[kettanaito]

@Doesntmeananything, thank you for the investigation! I'm excited to hear that SSE can be intercepted! Wonder if there's anything we can do it intercept events as they go.

I'm currently working on a NodeJS support, but can switch to this issue to help you once I'm done. I'm always open to questions or discussions, so please don't hesitate to raise those here.

Also, if you don't mind, we could then move your proof of concept repo under "msw" to serve as an example how to work with SSE/WS. That'd be awesome.

[kettanaito]

I'm trying to get my head around the SSE example. It seems MSW should intercept the hi from client event, so then it can mock the server response to it. I can see the once the WS connection is established, all the messages are inspectable live in DevTools. However, the webSocket.send("hi from client") is not intercepted by the Service Worker. I'm reading through w3c/ServiceWorker#947, trying to figure out if it's technically possible to access WS communication in a service worker.

API-wise, I think there should be at least two types of request handlers: event-based handler, and persistent handler (pulsing back messages to the client, like you have in your example using AsyncIterator).

[kettanaito]

One of the most useful pieces of code I've found in the w3c discussion (w3c/ServiceWorker#947 (comment)) was that the Service Worker file can establish a WebSocket connection. It appears that the WS events are not subjected to be intercepted in the fetch event, but one can establish a socket connection an (?) intercept events that way.

If it comes down to the manual WS connection, I'd suggest to do that on the client's side, not in the worker file. There's no technical reason to move this logic to worker, at least as of how I understand such implementation now.

[Doesntmeananything]

Thanks very much for taking the time to look further into this!

Since I've hit the wall in regards to intercepting WS connections, your suggestions come in handy. Will definitely look into this.

To be clear, are you saying that mocking WS connections falls strictly outside of MSW concerns? My investigations lead me to believe this, and I would certainly not want to push for something that doesn't make sense neither on technical nor on conceptual level.

[kettanaito]

Not necessarily. What I was trying to say is that a WebSocket event is not intercepted by the fetch event in a Service Worker. That's a per-spec behavior. However, I've mentioned an example above, that creates a WS connection within the worker file, which I suppose allows to intervene the communication in some way. I haven't tried that approach out, whether it's actually possible to mock the response of an event.

[kettanaito]

I've received a suggestion to look at mock-socket. We may get some inspiration from how it's implemented, and see if a similar approach can be done in MSW.

[kettanaito]

Update: I've started with the WebSocket support and will keep you updated as I progress. For those interested I will post some technical insights into what that support means, what technical challenges I've faced, and what API to expect as the result.

Session 1: It's all about sockets

No service for the worker

Unfortunately, WebSocket events cannot be intercepted in the fetch event of the Service Worker. That is an intentional limitation and there's no way to circumvent it. This means a few things:

• WebSocket events won't be visible in the "Network" tab;
• WebSocket support does not require the worker and can live outside of setupWorker context.
• WebSocket events interception should be done by patching the underlying logic (i.e. a WebSocket class).

Goodbye, handlers!

WebSocket operates with events, not requests, making the concept of request handler in this context redundant. Instead, you should be able to receive and send messages from ws anywhere in your app, including your mock definition.

import { rest, ws, setupWorker } from 'msw'

// Create an interception "server" at the given URL.
const todos = ws.link('wss://api.github.com/todos')

setupWorker(
  rest.put('/todo', (req, res, ctx) => {
    const nextTodos = prevTodos.concat(req.body)
    
    // Send the data to all WebSocket clients,
    // for example from within a request handler.
    todos.send(nextTodos)
    return res(ctx.json(nextTodos))
  })
)

// Or as a part of arbitrary logic.
setInterval(() => todos.send(Math.random()), 5000)

URL that got away

When constructing a WebSocket instance you must provide a URL that points to a WebSocket server. Providing a URL to a non-existing server yields and exception that you cannot circumvent by patching WebSocket class, as the URL validation lives in its constructor.

I've ended up re-implementing a WebSocket class, effectively making a polyfill out of it. That way it can have its custom constructor that would tolerate non-existing URLs if there is a ws.link interception server declared for that URL.

Persisting WebSocket clients

The entire idea of WebSocket is to sync data between multiple clients in real time. When you dispatch a mocked ws.send event to send some data to all clients, you need to let all the clients know they should receive the data (trigger their message event listener). However, there's no way to know and persist a list of WebSocket clients on runtime, since each page has its own runtime.

Usually a solution to this kind of problems is to lift the state up and maintain a record of clients in the upper context shared with all the clients (pages). However, in JavaScript there isn't that may ways to share and persist data between clients. In case of WebSocket clients one needs to store references to WebSocket instances—basically, object references. I've considered:

• localStorage/sessionStorage. Good for sharing textual data, not that suitable for storing objects with prototypes. Objects flushed here effectively lose their references, making them completely different objects.
• Web Worker/Service Worker. A great way to have a detached process in a browser that can control and communicate with multiple pages. However, as stated before, worker API cannot intercept WebSocket events, and using it only for the sake of storing some records is utterly redundant (I don't wish to ask people to copy any more worker scripts). Utilizing an existing mock worker may be an option, however, I'm afraid it would put too many logic into it, increasing its maintenance, and violating its purpose of containing a bare minimum logic that you seldom need to update.
• BroadcastChannel. Turns out the API that allows workers to communicate with clients exists standalone and it's awesome. You can create a broadcast channel as a part of page's runtime, and as long as another page on the same host creates a channel with the same name they can send data between them.

const channel = new BroadcastChannel('ws-support')

// One client sends a data.
channel.send('some-data')

// All clients can react to it.
channel.addEventListener('message', (event) => {
  event.data // "some-data"
})

I find BroadcastChannel a great choice to mimic the real time data synchronization functionality of WebSocket. I've chosen it to spawn a single channel between all clients and notify them when they should trigger their message event listeners.

[Sun-2]

@kettanaito

URL that got away
When constructing a WebSocket instance you must provide a URL that points to a WebSocket server. Providing a URL to a non-existing server yields and exception that you cannot circumvent by patching WebSocket class, as the URL validation lives in its constructor.
I've ended up re-implementing a WebSocket class, effectively making a polyfill out of it. That way it can have its custom constructor that would tolerate non-existing URLs if there is a ws.link interception server declared for that URL.

You could use an ES6 Proxy. It can mess with ctors.

Link.

[BlackGlory]

SSE and WebSockets are different issues.
If msw supports response streams (such as ReadableStream), it can support SSE.

[kettanaito]

@BlackGlory, MSW should support ReadableStream as the mocked response body. Would you have some time to try to which extent that's true, and whether SSE would be supported now?

[BlackGlory]

@kettanaito Although ctx.body supports ReadableStream, it does not seem to work.

export const worker = setupWorker(
  rest.get('/sse', (req, res, ctx) => {
    return res(
      ctx.status(200)
    , ctx.set('Content-Type', 'text/event-stream')
    , ctx.body(sse(function* () {
        yield 'message1'
        yield 'message2'
      }))
    )
  })
)

function sse(gfn) {
  let iter
  return new ReadableStream({
    start() {
      iter = gfn()
    }
  , pull(controller) {
      controller.enqueue(`data: ${iter.next().value}\n\n`)
    }
  })
}

Browser:

[MSW] Request handler function for "GET http://localhost:8080/sse" has thrown the following exception:

DOMException: Failed to execute 'postMessage' on 'MessagePort': ReadableStream object could not be cloned.
(see more detailed error stack trace in the mocked response body)

Node.js:

TypeError: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of ReadableStreamTypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of ReadableStream

      at ClientRequestOverride.<anonymous> (node_modules/node-request-interceptor/src/interceptors/ClientRequest/ClientRequestOverride.ts:216:34)
      at step (node_modules/node-request-interceptor/lib/interceptors/ClientRequest/ClientRequestOverride.js:33:23)
      at Object.next (node_modules/node-request-interceptor/lib/interceptors/ClientRequest/ClientRequestOverride.js:14:53)
      at fulfilled (node_modules/node-request-interceptor/lib/interceptors/ClientRequest/ClientRequestOverride.js:5:58)

[kettanaito]

Hey, @BlackGlory. Could you please report this as a separate issue? Thanks.

[kettanaito]

@SerkanSipahi, the issue is that only browser-side interception is not a finite feature. I wouldn't merge things that don't make sense on their own into main. With MSW (and with the Interceptors), we're establishing across-environment support, so it's a requirement to guarantee compatibility with both browsers and at least a limited set of Node.js versions. The latter is missing in the current implementation of the WebSocket support so it doesn't belong in the main.

The best I can do is release the browser implementation under a beta flag for Interceptors but it'd still require a bump in MSW, including the release of the ws() API which may not be fully ready at this point.

Honestly, this makes little sense to me, and if you wish to give this a try, consider using GitHub pull requests as dependencies in your project. Add the @mswjs/interceptors resolution on top of MSW and you should get the things installed correctly. But before that, check on the status of the #396 API because I don't remember if it's in working condition at the moment.

Contributing to the ws API is also something I'd much appreciate, as that's a wrapper around the Interceptors and doesn't concern itself much with how to intercept requests, rather how to consume this new interceptor from MSW.

[wade-gooch-kr]

Hello @kettanaito! My team and I have been following this thread, and we saw that the change has been merged in so that msw will support EventSource. I was hoping to inquire when an official release might be happening that includes this update? Thank you in advance!

[kettanaito]

Hey, @wade-gooch-kr. Excited to hear that. I'm in the middle of some test rewrites, I will publish that branch when I have a minute. Meanwhile, you can specify that PR as your dependency in package.json and should be able to try it out.

[Stackustack]

Hey, whats the status on this? It seems like some part of the job was done in this merged PR 🤔 Should this be marked closed / done or is this still WIP?

[kettanaito]

September 2023 Status Update

mswjs/interceptors#236 (comment)

@Stackustack, supporting SSE is unfortunately not enough to ship WebSocket support. See the status update in the linked comment above.

[kettanaito]

Update

I've had some success implementing a WebSocket class-based interceptor (mswjs/interceptors#501). This means that the WebSocket support is coming to MSW rather soon! The browser test suite is currently passing. The Node.js test suite using Undici's WebSocket as a global is also passing!

Now, before anyone gets overly excited about this, let me clarify a few things.

1. MSW will only support intercepting a WebSocket communication established by using the global WebSocket class (i.e. the WHATWG WebSocket standard present globally). This means that third-party libraries that implement WebSockets by other means (e.g. using custom transports) will not be supported. I see no future in supporting contrived transports from third-parties—that is an unreliable strategy. I'd much rather see (and even help) those third-parties migrate to use the standard, as it's also landing in Node.js later this year.
2. The API to intercept and work with WebSockets will respect the WHATWG WebSocket Standard. This means you will receive outgoing MessageEvent and send back data that will be translated to an incoming MessageEvent.

What's left?

Feedback. You can help land this API sooner by helping with the following:

• Read the WebSocket Interceptor API and usage examples. Note that you are reading about the API in the underlying interception library that MSW depends on. It will, however, land in a similar form in MSW itself, perhaps wrapped in a ws.link() for convenience.
• Share your thoughts on the implementation at feat: add WebSocket class interceptor interceptors#501

Meanwhile, I will improve the test coverage of the interceptor to make sure it's fully compatible with the standard when you're using it.

[kettanaito]

Turns out that the initial WebSocket implementation will support SocketIO also!

If you want to be able to mock SocketIO with MSW, please read and upvote this:
socketio/socket.io-parser#129

Thank you.

[kettanaito]

WebSocket Support Beta

Please participate in the discussion about the upcoming WebSocket API to help us shape it and ship it faster:

• WebSocket Support Beta #2010

Thank you!

[kettanaito]

I'm renaming this issue so it focuses on the WebSocket API mocking exclusively.

Server-Sent Events (SSE) are quite different, and to my best knowledge, they can be intercepted by the Service Worker. Their implementation will be different. If someone needs it, I encourage you to open a new feature proposal and describe it in more detail (e.g. how you are using SSE in your application, how you imagine mocking to look like, etc).

[kettanaito]

Update: Give the RC a try!

You can install the RC with the WebSocket support today: npm i msw@next.

• WebSocket Support Beta #2010

Please participate and share your feedback! The more feedback we get, the faster and better the end API will be released. Thank you!

[95th]

@kettanaito I am getting when using websocket mocks:

ReferenceError: BroadcastChannel is not defined
    at file:///<project>/node_modules/.pnpm/[email protected][email protected]/node_modules/msw/src/core/ws/ws.ts:14:28
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)

Code I am trying:

    const api = ws.link("wss://foo.bar.com/baz");
    const apiHandler = api.on("connection", ({ client }) => {
        client.addEventListener("message", event => {
            // Echo the message back to the client
            client.send(event.data);
        });
    });

Platform: Node

[kettanaito]

@95th, hi! What version of Node.js are you running? It looks like it's older than v18.

MSW itself doesn't support Node.js <18. Please update and have the global BroadcastChannel API available!

[johnhunter]

I have a regression with 2.3.0-ws.rc-9 moving from 2.3.0-ws.rc-8. The socket returned by ws.link has no on method.

Resolved: looks like I can listen for events directly on the WebSocketLink, e.g.

const socket = ws.link('wss://chat.example.com');
socket.addEventListener("connection", ...)

[kettanaito]

@johnhunter, hi. That is not a regression but a change in the API. I've removed .on in favor of consistent .addEventListener. My bad, I forgot to include the change log (fixed here).

[kettanaito]

Released: v2.6.0 🎉

This has been released in v2.6.0!

• 📄 Release notes
• 📦 npm package

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.

---

Predictable release automation by @ossjs/release.

[AdrienFromToulouse]

👋

Is it expected to have to mock BroadcastChannel even for "classic" HTTP requests using msw/node?
With 2.6.0 I get this error with my Jest mocks: ReferenceError: BroadcastChannel is not defined

http.get(myMockedEndpoint, () => {
    return HttpResponse.json(foo);
}),

using the node setupServer like so with the brand new type WebSocketHandler

import { setupServer } from 'msw/node';

export const server = setupServer(...handlers);

Since the setupServer is now having union type like so: declare const setupServer: (...handlers: Array<RequestHandler | WebSocketHandler>) => SetupServerApi;

Cheers,

[kettanaito]

Hi, @AdrienFromToulouse. Thanks for reporting his. Looks like the same issue as mswjs/data#306 (comment). Please use the supported version range of Node.js and stay away from browser-like environments that meddle with your environment.

Is it expected to have to mock BroadcastChannel

No. BroadcastChannel is a global Node.js API. If it's not there, your test environment is taking it away from you. Don't use such environments.

[AdrienFromToulouse]

Hi, @AdrienFromToulouse. Thanks for reporting his. Looks like the same issue as mswjs/data#306 (comment). Please use the supported version range of Node.js and stay away from browser-like environments that meddle with your environment.

Is it expected to have to mock BroadcastChannel

No. BroadcastChannel is a global Node.js API. If it's not there, your test environment is taking it away from you. Don't use such environments.

Thank you for you detailed answer and thank you for maintaining msw.
Cheers

[kettanaito]

I will do my best to make in-browser automation accessible and clear in the months to come. This experience has to go, it's absurd. Sorry you have to go through this. It will get better.

[AdrienFromToulouse]

I am planning to migrate to vitest.
Jest was great for a while but the amount of shenanigans with typescript and nextJs etc to run tests is just too much and unproductive at some point.
Thank you again.