[RFC] useSWRList

[mAAdhaTTah]

Issue #1041 opened a request for a version of useQueries for SWR. useQueries, from react-query, enables a developer to make multiple API calls from a single hook call when the number of requests is going to be dynamic, a feature currently missing from SWR. This RFC will lay out the motivation for including this feature in SWR, a proposal for how that feature should behave when integrated with SWR, and some potential alternatives & open questions for the proposal.

Motivation

In SWR, if multiple API calls need to be made, this is easy to do with multiple SWR hooks:

const { data: user1 } = useSWR("/users/1");
const { data: user2 } = useSWR("/users/2");
const { data: user3 } = useSWR("/users/3");

However, in the above example, the developer needs to know in advance the number of users the component needs to fetch. If the number of users to fetch is variable, there isn't a good way to do that currently¹. Due to the Rules of Hooks, you can't map over an array to call useSWR. While this can be solved in userland with a custom fetcher, it is currently difficult for userland developers to correctly implement the cache management, a core benefit of SWR. A naive implementation would push the whole array into the cache, require a full refetch of every item should an item be added to or removed from the key array. Providing a shared, stable, and efficient solution for all consumers of SWR would be incredibly helpful.

Proposals

Adopting this proposal would add a new hook, useSWRList, that would be exported at swr/list, matching the current subfolder export of useSWRInfinite. This would ensure that useSWRList would only be included in a consumer's bundle if they actually use it in their application.

import useSWRList from "swr/list";

This applies to either of the proposals below, as minimizing bundle size is a common value across both.

Additionally, both proposals have the same input API:

useSWRList<Data = any, Error = any>(
  keys: Key[],
  fetcher: Fetcher<Data> | null,
  config: typeof defaultConfig & SWRConfiguration<Data, Error>
);

This mirrors the input API of useSWR itself. Instead of a single Key as the first param, it instead provides an array of Keys. The two proposals differ in their return types.

Proposal A: "Array of SWRs"

The return type of useSWRList will be an array of SWR Responses, indicating their status in the standard SWR fashion.

useSWRList<Data = any, Error = any>(
  keys: Key[],
  fetcher: Fetcher<Data> | null,
  config: typeof defaultConfig & SWRConfiguration<Data, Error>
): SWRResponse<Data, Error>[];

As each request completes, the hook will trigger a rerender. In this approach, each request is effectively treated as a separate SWR hook call. An error thrown by a single hook will not affect the others. Individual requests can be refetched by calling their respective mutate functions.

Advantages

• Can render successful requests as they come in
• Can aggregate into something resembling Proposal B in userland
• Caching behavior for individual requests is clear
  • Interaction with mutate is clear, because it's bound to individual keys

Disadvantages

• Performance - can trigger a lot of rerenders as data comes in
• Interaction with Suspense could be complicated

Proposal B: Aggregated Return

The return type of useSWRList will a normal SWR Response with an array of Data.

useSWRList<Data = any, Error = any>(
  keys: Key[],
  fetcher: Fetcher<Data> | null,
  config: typeof defaultConfig & SWRConfiguration<Data, Error>
): SWRResponse<Data[], Error>;

After all requests are complete, the hook rerenders with data being an array of the response. If any of the responses fail, error is populated and data remains undefined. Any successful requests before or after the thrown error will be cached and reused if the failed request is retried. This behavior is similar to Promise.all resolving after all promises resolve.

Advantages

• Performance - single rerender after all requests resolve
• Simple mental model based off familiar semantics
• Easy interaction with Suspense (suspend until all resolved; throw if any throw)

Disadvantages

• All-or-nothing means successful requests can't be rendered
• Behavior of mutate currently unspecced
  • Any behavior of mutate is going to have new semantics

Alternatives

Some potential areas to consider:

• Developer chooses between A & B based on some configuration option
• Some combo of both with aggregate status + individual results

Open Questions

1. How does this interact with Suspense?
2. How should mutate work in Proposal B?

Footnotes

1. With Suspense, one option could be to map over the users & render an array of components that fetch each user, wrapping the render in a Suspense boundary. However, this doesn't help if the goal is to combine the results in some way, e.g. if you want to add up the number of points scored by all the users on a single team. ↩

[gfox1984]

Proposal A is the react-query way (which was the initial request). Plus as you said, you can still implement B on top of A shall you need it. So A for me!

[lg]

having used the React Query method (proposal A), all i'd say is easy access to the Key when looping over the responses would be greatly appreciated! additionally see what's possible to not require users utilize a "deep" object comparison when using hooks like useEffect on these re-renders. people were resulting to things like JSON.stringify() to get useQueries stable...

[mAAdhaTTah]

easy access to the Key when looping over the responses would be greatly appreciated!

This should be doable without changes to the API. You have access to the array of keys you passed in. In Proposal A, you'll have access to them in the resulting array in the same order. So you could do:

const keys = ['/users/1', '/users/2'];
const results = useSWRList(keys);
const mapped = results.map((result, i) => {
  const keyForResult = keys[i]
  // do something with key
});

additionally see what's possible to not require users utilize a "deep" object comparison when using hooks like useEffect on these re-renders.

I'm not sure I understand this comment. I don't think this will be any different that SWR normally. Is there something specific to the list that applies here?

[lg]

i think the issue was around that when you have an array of arrays, at least in ReactQuery's useQueries function it would constantly change causing useEffect to keep triggering

[mAAdhaTTah]

Ah ok. We'll have an array of objects with SWR, so maybe this won't be an issue?

[shuding]

Indeed, it won't be an issue with SWR as it has deep comparison built-in for object keys.

[shuding]

Thank you for putting together this RFC! I vote for proposal A because it’s simply “a list of useSWR hooks”, and the behavior is pretty clear. Hopefully it can be implemented with very little change to the core.

[mAAdhaTTah]

@shuding Is this approval enough for me to get started or do we need more people to chime in?

[shuding]

@mAAdhaTTah Yes I think the design is good and an implementation can be started!

@promer94 made a quick prototype of this and showed me recently, we can have him sharing it and have further discussions based on that. Because one thing that isn't clear to me is if we should add it to the core to reuse existing logic, or start a completely new hook?

How does this interact with Suspense?

Also for this question, I'd disallow suspense in this new hook for now until we figure out a programmatic way to do prefetching for SWR hooks. Otherwise they can't be triggered in parallel.

[mAAdhaTTah]

@shuding

Because one thing that isn't clear to me is if we should add it to the core to reuse existing logic, or start a completely new hook?

I'm not sure I follow. What is "core" vs "new hook" in this context?

@promer94 Would love to see what you've started! Feel free to email me if you need (email is in my GitHub profile).

[n7ck]

Any Update on if useSWRList or useSWRListImmutable is in the works currently? @mAAdhaTTah @shuding @promer94 ?
I'm realizing that is exactly what I could use... was asking for something like this in other thread here: #2124

[mAAdhaTTah]

I opened an initial PR but got some pushback on the semantics of the hook. Since then, I haven't gotten a clear direction from the maintainers as to the preferred direction. I'm still open & interested in contributing this hook but without that clarity, there isn't much I can do.

[summer-cook]

would also love if this was working! following

[harrismendell]

Any update on this work? I'd like to be able to do the following:

1. Query a list of URLs
2. Store each response in its own cache (have its own key)

I think this is essentially what useSWRList would do?

[s-fletcher]

Hello, it looks like not much progress has been made on this request.

As a workaround, I'm using useSWRInfinite with the option parallel: true.

• Where I originally found this workaround
• Documentation on parallel.

For instance, this was how I covered my use case:

import useSWRInfinite from 'swr/infinite';

const keys = ['user-1', 'user-2', 'user-3'];
const { data } = useSWRInfinite((index) => `/users/${keys[index]}`, fetcher, {
	initialSize: keys.length,
	parallel: true,
});

[RobPruzan]

Doesn't work if the keys are dynamically calculated

[jalooc]

Doesn't dedupe the requests either.

[nickmarca]

While this feature remains unreleased (if it ever will be), I've taken the initiative to code a solution that achieves (more or less) the same goal. I'm curious to receive feedback in case there's a better approach.

Let's imagine an e-commerce website where we're working on the basket page. Our aim is to display the total basket value alongside the added products.

We're adopting a loading strategy where each product loads individually. This approach ensures that each product request can be cached, enhancing the page's resilience to errors. For instance, if product A encounters an issue loading, it won't impede the loading of product B. While it might not secure the full sale, it allows customers to proceed to checkout with available products.

Developing a component for each product in the basket page and fetching their information within these components is straightforward.

The challenge arises in consolidating all these product details, loaded within their respective components, at the parent-level component responsible for rendering these product nodes.

One feasible method involves establishing a list in the parent component and enabling the product components to update this list using useEffect as their loading status changes. This approach accumulates an updated list of all loaded products by the completion of the last product load.

However, this approach tends to generate verbose, repetitive, and aesthetically unpleasing code rapidly.

To address this, I crafted a solution: a generic component that tracks the loaded products and subsequently passes them to a child component via a function as a prop. This approach aims to streamline the code and prevent it from becoming overly convoluted.

I acknowledge this is less than ideal, but I cannot visualise a better approach till we get this RFC released.

This is essentially how we would use it:

// page.tsx

type Product = {
  price: number;
  id: number;
  name: string;
};

export function BasketPage() {
  return (
    <GetDataList
      keys={['/test/1', '/test/2', '/test/3']}
      render={(entries: Entry<Product>[]) => {
        if (entries.some(entry => entry.isLoading)) {
          return <div>Loading...</div>;
        }

        return (
          <div>{sum(entries.map(({ data }) => ({ price: data.price })))}</div>
        );
      }}
    />
  );
}

And these are the supporting components and functions:

// get-data-list.tsx
'use client';

export default function GetDataList<T extends {}>({
  keys,
  render,
}: {
  keys: string[];
  render: (entries: Entry<T>[]) => React.ReactNode;
}) {
  const [entries, setEntries] = useState<Entry<T>[]>([]);

  return (
    <>
      {keys.map(key => (
        <GetDataListItem key={key} setEntries={setEntries} />
      ))}
      {render(entries)}
    </>
  );
}

// get-data-list-item.tsx
'use client';

import { useCallback, useEffect } from 'react';
import { addOrUpdateItem, fetcher, generateId, removeOnce } from './utils';
import useSWRImmutable from 'swr/immutable';

export type Entry<T> = {
  _id: string;
  isLoading: boolean;
  error: any;
  data: T;
};

export default function GetDataListItem<T extends {}>({
  key,
  setEntries,
}: {
  key: string;
  setEntries: (prev: (value: Entry<T>[]) => Entry<T>[]) => void;
}) {
  const { data, isLoading, error } = useSWRImmutable(key, fetcher);
  const getInternalId = useCallback(generateId, []);

  useEffect(() => {
    const _id = getInternalId();

    setEntries(items => {
      return addOrUpdateItem(items, { _id, data, isLoading, error });
    });

    return () => {
      setEntries(items => {
        return removeOnce(items, _id);
      });
    };
  }, [getInternalId, data, isLoading, error, setEntries]);

  return <></>;
}

// utils.ts

export function generateId() {
  return Math.random().toString(36).slice(2, 9);
}

export function addOrUpdateItem<T extends { _id: string }>(
  items: T[],
  item: T,
) {
  const index = items.findIndex(i => i._id === item._id);
  if (index === -1) {
    return [...items, item];
  }
  return items.map(i => (i._id === item._id ? item : i));
}

export function fetcher(url: string) {
  return fetch(url)
    .then(r => r.json())
    .catch(console.error);
}

export function removeOnce<T extends { _id: string }>(items: T[], _id: string) {
  const entryIndex = items.findIndex(entry => entry._id === _id);
  return items.filter((_, index) => entryIndex !== index);
}

export function sum<T extends { price: number }>(items: T[]) {
  return items.reduce((acc, { price }) => acc + price, 0);
}

[dizzyjaguar]

any update on this?