chore: fix broken docs build / remove unused tag files

[thomasheartman]

Issue

So, we've got an issue with our docs build not working. When building for production, we get an error that looks a little bit like this:

$ docusaurus build
[INFO] [en] Creating an optimized production build...

✔ Client


✖ Server
  Compiled with some errors in 40.85s



TypeError: source_default(...).bold is not a function
TypeError: source_default(...).bold is not a function
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
    at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
    at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
    at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
    at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.

Which isn't very helpful at all. If you go into /node_modules/@docusaurus/core/lib/client/serverEntry.js and modify the render function to log the actual error and remove anything chalk-related, you get this instead:

$ docusaurus build
[INFO] [en] Creating an optimized production build...

✔ Client


✖ Server
  Compiled with some errors in 44.62s

Actual error: Error: Unexpected: cant find current sidebar in context
    at useCurrentSidebarCategory (main:11618:247)
    at MDXContent (main:38139:1593)
    at Fb (main:149154:44)
    at Ib (main:149156:254)
    at W (main:149162:89)
    at Jb (main:149165:98)
    at Ib (main:149157:145)
    at W (main:149162:89)
    at Jb (main:149165:98)
    at Ib (main:149157:145)
Actual error: Error: Unexpected: cant find current sidebar in context
    at useCurrentSidebarCategory (main:11618:247)
    at MDXContent (main:38513:1469)
    at Fb (main:149154:44)
    at Ib (main:149156:254)
    at W (main:149162:89)
    at Jb (main:149165:98)
    at Ib (main:149157:145)
    at W (main:149162:89)
    at Jb (main:149165:98)
    at Ib (main:149157:145)


Error: Unexpected: cant find current sidebar in context
Error: Unexpected: cant find current sidebar in context
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
    at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
    at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
    at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
    at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.

That's better, but it's still not very clear.

Getting more info

We've had problems with a similar error message before. Last time it was caused by an empty file that docusaurus couldn't process.

A similar issue has also been described in this docusaurus GitHub issue . That's also what gave me the idea of changing the logging in the dependency.

I'm currently unsure whether this is caused by the openapi docs or something else. I've been in touch with the openapi plugin maintainer and he has been able to see the same error when building for prod locally, but it was due to some old generated files.

Worth noting: this only seems to affect the prod build. Building for dev (yarn docusaurus start) works just fine. It also fails locally and in CI, so it is an issue.

Updating the logging

To get better logging, you can go into the /node_modules/@docusaurus/core/lib/client/serverEntry.js file and update the render function from

export default async function render(locals) {
    try {
        return await doRender(locals);
    }
    catch (err) {
        // We are not using logger in this file, because it seems to fail with some
        // compilers / some polyfill methods. This is very likely a bug, but in the
        // long term, when we output native ES modules in SSR, the bug will be gone.
        // prettier-ignore
        console.error(chalk.red(`${chalk.bold('[ERROR]')} Docusaurus server-side rendering could not render static page with path ${chalk.cyan.underline(locals.path)}.`));
        const isNotDefinedErrorRegex = /(?:window|document|localStorage|navigator|alert|location|buffer|self) is not defined/i;
        if (isNotDefinedErrorRegex.test(err.message)) {
            // prettier-ignore
            console.info(`${chalk.cyan.bold('[INFO]')} It looks like you are using code that should run on the client-side only.
To get around it, try using ${chalk.cyan('`<BrowserOnly>`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#browseronly')}) or ${chalk.cyan('`ExecutionEnvironment`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#executionenvironment')}).
It might also require to wrap your client code in ${chalk.cyan('`useEffect`')} hook and/or import a third-party library dynamically (if any).`);
        }
        throw err;
    }
}

to

export default async function render(locals) {
    try {
        return await doRender(locals);
    }
    catch (err) {
        console.error(err)
        throw err;
    }
}

That'll yield the errors about the current sidebar in context.

Root cause

Found the issue! 🙋🏼 It's explained in this comment on the openapi docs integration for now, but in short: we have tags defined that we don't use. They're being picked up by docusaurus, but don't have the proper context. That's causing this.

The previously mentioned comment is included here for easy finding in the future:

Root cause explanation

The OpenAPI spec we use to generate the docs has a number of tags listed at the root level. This is necessary for this plugin to pick up tag categories and is, as far as I can tell, also how it should be done.

However, not all of those tags are in use. Specifically, there's 2 tags that are not.

When the plugin generates docs from the spec, it generates pages for all endpoints and all tags and groups them by tag. However, it seems likely that if a tag doesn't have any associated endpoints, then it won't get added to the sidebar because there's no endpoint that references it.

But the doc files for these tags do end up lying around in the directory regardless, and when docusaurus tries to pick up the files in the generated directory, it also tries to pick up the unused tag files. But because they're not part of a sidebar, they end up throwing errors because they can't find the sidebar context.

How I found it

The fact that I got more instances of the error message without the sidebar ref than with it made me pay more attention to the number of errors. I decided to check how many files were in the generated directory and how many files were referenced from the generated sidebar. Turns out the difference there was 2: there were two generated files in the directory that the sidebar didn't reference.

At this point, it was easy enough to try and delete those files before rebuilding, and wouldn't you know: it worked!

Our use case

Now, why do we have tags that are unused in the root spec? Can't we just remove them?

That's a good question with a bit of a complicated answer. Unleash uses an open core model and the OpenAPI integration is part of that open core. The closed-source parts of Unleash are located in another repo and extend the open-source distribution.

Because the OpenAPI spec is configured in the open-source part, enterprise-only tags etc also need to be configured there. Then, when the changes are absorbed into enterprise, we can use the tags there.

It gets more complicated because we use an enterprise instance to generate the docs (because we want enterprise-endpoints to be listed too). The instance uses the latest released instance of Unleash to have the docs most accurately reflect the current state of things.

So, in this case, the tags have been added, but not yet used by any endpoints, which suddenly causes this build failure. We can add the tags to the enterprise-version, but the spec wouldn't be updated before the next release regardless, which will probably be in a week or so.

This isn't an ideal setup, but .. it is what it is.

Solutions and workarounds

As mentioned in the previous section, the reason the build was failing was that there were unused tag files that docusaurus tried to include in the build. Because they don't belong to a sidebar, the compilation failed.

I've reported the issue to the openapi plugin maintainers and am waiting for a response.

However, it seems that having unused root tags declared is invalid according to the spec, so it's something we should look into fixing in the future.

Current workaround: cleaning script

The current workaround is to extend the api cleaning script to manually remove the unused tag files.

Ideal solution: filter root-level tags

Ideally, we shouldn't list unused OpenAPI tags on the root level at all. However, because of the way we add root-level tags (as a predefined, static lists, refer to src/lib/openapi/index.ts) and endpoints (dynamically added at runtime) today, we don't really have a clear way to filter the list of tags. This gets even more complicated when taking the enterprise functionality and the potential extra tags they must have.

This is, however, something that should definitely be looked into. Working with OpenAPI across multipile repos is already troublesome, so this is just yet another thing to look into.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
unleash-docs	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Nov 11, 2022 at 0:48AM (UTC)
unleash-monorepo-frontend	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Nov 11, 2022 at 0:48AM (UTC)

[sighphyre]

You're a ninja for finding this, nice one

[chriswk]

Moving from a released version (1.4.1) to 0.0.0-<> ?

[thomasheartman]

Canary release that includes fixes for 2.2.0. Should be in the next release, but this fixes the issue for now. We may be able to roll back to 1.4.3 with docusaurus 2.1.0, however. I can look into it on monday unless we just want to wait it out.