nav: fix problem with non-existing pages in sidebar

[iAdramelk]

In the old sidebar.json config it was possible to create page with source: false and then user requested such a page, it's first child was rendered instead.

For example then you enter in browser /doc/changelog/ it will show content of /doc/changelog/0.18. As a result we have two pages with the exactly same content on different url. It is bad for SEO and it is bad for search with docsearch because we can have duplicate results (enter changelog in the search for example).

Right now we made a temporal solution:

1. Top menu always render url of the first article.
2. Sidebar always render child's url (but now we have two links with the same url in sidebar).
3. For old pages with source: false we added temporary 307 redirects to the redirects-list.json file.

But it is a bad long-term solution and also requires us to manually check redirects-list.json file every time we want to add page to see if there is a redirect that prevents it. We want to only have permanent 301 redirect in this file.

There is multiple ways how can we fix this problem:

1. Write missing pages and forbid folders without index.md files (they should throw errors on build).
2. Make separate middleware for the temporary app redirects and generate it's config at the build time from missing imdex.md files in doc folder. (Possible conflicts between 2 middlewares for redirects)
3. Create two copies of the same page with the same <link rel="canonical" />.
4. Something else.

But as result we want to remove temporary redirects from redirects-list.json and make this logic the part of the app.

[jorgeorpinel]

two pages with the exactly same content on different url. It is bad for SEO and it is bad for search with docsearch...
Top menu always render url of the first article.

OK so these are fixed by "temporary" redirects right? 307 status code for example. (Keeping in mind our docs structure keeps changing so for example the top item in /changelog may go from /0.1 to /0.2, etc.)

requires us to manually check redirects-list.json file every time we want to add page to see if there is a redirect that prevents it. We want to only have permanent 301 redirect in this file.

Yes this would be terrible, but why do we only want 301s there? I'm not sure about that part. Each redirect type serves different purposes, see discussion in #757 (comment)

---

2. Make separate middleware for the temporary app redirects and generate it's config at the build time from missing imdex.md files in doc folder

I vote for this. And it could use sidebar.json as base instead of browsing the directories.

[shcheklein]

I vote for this. And it could use sidebar.json as base instead of browsing the directories.

It's done now.

[jorgeorpinel]

Amazing 🎉