feat: add createSitemapItems hook

[johnnyreilly]

Pre-flight checklist

• I have read the Contributing Guidelines on pull requests.
• If this is a code change: I have written unit tests and/or added dogfooding pages to fully verify the new behavior.
• If this is a new API or substantial change: the PR has an accompanying issue (closes Proposal: createSitemapItems hook - a sitemap equivalent to createFeedItems #10081) and the maintainers have approved on my working plan.

Motivation

I presently mutate my sitemap manually on each build as a post processing step. I've written about it here and I have historically done this for two reasons:

1. To add lastmod to sitemap entries (something that is no longer necessary as of 3.2)
2. To trim pagination, tags pages and programmatically determined canonicals from the sitemap - this is still necessary

If there was a hook that allowed control of the sitemap, I would no longer need to do 2 as a separate post processing step

Test Plan

Automated tests, plus look at the generated sitemap for the preview site: https://deploy-preview-10083--docusaurus-2.netlify.app/sitemap.xml

The screenshot below was taken from the current live Docusaurus sitemap: https://docusaurus.io/sitemap.xml

Please note the URLs above which feature page. Now look at the sitemap generated using this PR which filters those sitemap items out out using the new hook: https://deploy-preview-10083--docusaurus-2.netlify.app/sitemap.xml

Test links

Deploy preview sitemap: https://deploy-preview-10083--docusaurus-2.netlify.app/sitemap.xml
Updated docs: https://deploy-preview-10083--docusaurus-2.netlify.app/docs/api/plugins/@docusaurus/plugin-sitemap

Related issues/PRs

#10081

[netlify]

✅ [V2]

Name	Link
🔨 Latest commit	61ae607
🔍 Latest deploy log	https://app.netlify.com/sites/docusaurus-2/deploys/6631323dbce6020008440554
😎 Deploy Preview	https://deploy-preview-10083--docusaurus-2.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[github-actions]

⚡️ Lighthouse report for the deploy preview of this PR

URL	Performance	Accessibility	Best Practices	SEO	PWA	Report
/	🟠 60	🟢 98	🟢 96	🟢 100	🟠 88	Report
/docs/installation	🟠 58	🟢 96	🟢 100	🟢 100	🟠 88	Report
/docs/category/getting-started	🟠 75	🟢 100	🟢 100	🟢 90	🟠 88	Report
/blog	🟠 70	🟢 100	🟢 100	🟢 90	🟠 88	Report
/blog/preparing-your-site-for-docusaurus-v3	🟠 62	🟢 96	🟢 100	🟢 100	🟠 88	Report
/blog/tags/release	🟠 70	🟢 100	🟢 100	🟠 80	🟠 88	Report
/blog/tags	🟠 74	🟢 100	🟢 100	🟢 90	🟠 88	Report

[slorber]

Overall that looks good, will review in depth later

[slorber]

I know postBuild already receives this, but IMHO this was a mistake and we shouldn't expose this "messy" data structure as part of our public API.

Unless it's really needed I'd prefer to remove it for now, or provide it with a clear Docusaurus public API.

Also React 19 has metadata apis now, so not sure it's a good time to provide such API. We can introduce it later once things get clearer.

[johnnyreilly]

The reason the API is this shape is because it is modelled on the createFeedItems API which allows the caller to call the underlying default implementation and provides the necessary inputs to do so. We could have a different style of API here which doesn't work that way. Instead, perhaps, we could expose a "default" implementation which wraps the underlying dependencies of the actual implementation and just exposes a parameterless function that could be called. That would likely feel cleaner.

What do you think? The downside of this is inconsistency. The createFeedItems and createSitemapItems APIs will be similarly named but slightly different to use from one another. Not necessarily a big issue - but worth considering

[slorber]

I'm not sure we talk about the same thing. What annoys me most is this {[location: string]: HelmetServerState};

And annoying code like this:

// Maybe we want to add a routeConfig.metadata.noIndex instead?
// But using Helmet is more reliable for third-party plugins...
function isNoIndexMetaRoute({
  head,
  route,
}: {
  head: {[location: string]: HelmetServerState};
  route: string;
}) {
  const isNoIndexMetaTag = ({
    name,
    content,
  }: {
    name?: string;
    content?: string;
  }): boolean => {
    if (!name || !content) {
      return false;
    }
    return (
      // meta name is not case-sensitive
      name.toLowerCase() === 'robots' &&
      // Robots directives are not case-sensitive
      content.toLowerCase().includes('noindex')
    );
  };

  // https://github.com/staylor/react-helmet-async/pull/167
  const meta = head[route]?.meta.toComponent() as unknown as
    | ReactElement<{name?: string; content?: string}>[]
    | undefined;
  return meta?.some((tag) =>
    isNoIndexMetaTag({name: tag.props.name, content: tag.props.content}),
  );
}

I'd prefer if we didn't expose HelmetServerState to the user. I don't think createFeedItems exposes it so far.

[slorber]

wonder if this one is really useful

I know createFeedItem takes the config (which IMHO is already not super useful), but it doesn't take the option so maybe we shouldn't add this here?

[johnnyreilly]

Just pushed a change that omits head and options from params - is this what you had in mind?

[slorber]

LGTM thanks 👍

Did some minor code/docs/type refactors but overall it works!