Improve resolution of relative links to files not in the website

[felipecrs]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Motivation

Refs #6460 (reply in thread).

The proposal is to make Docusaurus smarter on how it handles relative links to files which are not included on the website itself. To be clear about what I'm referring to:

Let's say you have a docs/docker-images/index.md with something like:

The source code is available at [Dockerfile](../../Dockerfile).

Docusaurus will try to link to this file in my website itself (http://localhost:3000/docs/../../Dockerfile -> ?). Which, in turn, is broken link. Instead, for such situations, Docusaurus could link to the file in the source code browser.

Technically, I'm talking about something like below:

// docusaurus.config.js
module.exports = {
  presets: [
    [
      "classic",
      /** @type {import('@docusaurus/preset-classic').Options} */
      ({
        docs: {
          path: "docs",
          routeBasePath: "docs",
          editUrl:
            "https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD",
          relativeFilesBaseUrl:
            "https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD"
        },
      }),
    ],
  ],
};

Namely look at relativeFilesBaseUrl. This would tell Docusaurus to use this base URL instead of the website itself to link relative files (which otherwise would be broken links). Some examples:

Considering that this file is at docs/index.md:

- [Dockerfile](../Dockerfile) links to https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD/Dockerfile
- Also [.gitignore](.gitignore) links to https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD/docs/.gitignore
- Or, [Jenkinsfile](./Jenkinsfile) links to https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD/docs/Jenkinsfile 

Of course, one possible workaround is to do something like:

- [Dockerfile](https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD/Dockerfile)
- [.gitignore](https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD/docs/.gitignore)
- [Jenkinsfile](https://review.gerrithub.io/plugins/gitiles/felipecrs/gerrit-jenkins/+/HEAD/docs/Jenkinsfile)

But the proposed solution would have the benefit of:

1. Work in both the website and in Gitiles (or whatever markdown renderer provided by the Git project)
2. No need to write the full URL link, which
   • Hostname may change
   • Project name may change
   • And most importantly: the branch does change
     • Docusaurus would still point to "HEAD" as the example below, but at least in Gitiles it would follow the branches accordingly.

About how to determine whether a file should be linked using relativeFilesBaseUrl or the normal website linking, I have some ideas:

1. Try to link on the website. If it turns out to be a broken link, try to check if the file exists in local folder hierarchy, and if it does, link using relativeFilesBaseUrl. If not, throw broken link as before.
2. Always link using relativeFilesBaseUrl unless the link is a .md file which is known to be valid and included in the website. But, of course, check if the file exists before, and if not, throw a broken link as before.
   • For this approach, there is the situation where one could use docs/foo/bar (no .md extensions). I would suggest that, if one enabled relativeFilesBaseUrl, Docusaurus won't try to add .md proactively and instead would treat as a full file name directly. Linking to .md files without using their extensions is already discouraged by Docusaurus anyway.

Self-service

• I'd be willing to do some initial work on this proposal myself.

[Josh-Cena]

Try to link on the website. If it turns out to be a broken link, try to check if the file exists in local folder hierarchy, and if it does, link using relativeFilesBaseUrl. If not, throw broken link as before.

The broken links check happens at a much later stage than the Markdown link resolution, so we unfortunately can't do that. You still have to be explicit in telling Docusaurus that it's a file path, e.g. by prepending file://

[felipecrs]

@Josh-Cena what about idea number 2 then? Always link with relativeFilesBaseUrl unless the file is a .md.

[Josh-Cena]

Mmm, that sounds like a problem very close to #6210. TL;DR: ../../Dockerfile is a very, very common way to indicate a relative URL link rather than a file path, and we can't always assume that only .md paths are page links, because to link pages across plugins (or to non-MD pages), you always have to use URL links instead of file paths.

[felipecrs]

Ok. If there was a way to check for broken links earlier... something like:

1. Parse all the .md files for their frontmatter before (not parse or convert their contents), the objective is to construct a collection of links (all the .md pages) following their correct slug and so.
2. When parsing the contents of the file, use the collection built in step 1 to check whether a link is broken or not, and apply the idea number 1.

But as you said, pages are not only made of .md files. If there was a way to get a collection of links in beforehand for all the sources (not only .md)...

[Josh-Cena]

The problem is that there are a bunch of other pages that are not generated with Markdown, e.g. React pages, plugin-created dynamic pages, that can only be reliably detected once we build the site, which happens after the Markdown transformation is done.

No problem if we can't think of a viable solution yet. This could be interesting to work on, maybe we can check in the FS for a corresponding file first.

About making the site work in the GitHub interface though, that has never been our priority, although it's usually a nice thing to stick to. So if we decided it can only be reliably done by sacrificing usability on GitHub, we will do it.

[felipecrs]

About making the site work in the GitHub interface though, that has never been our priority, although it's usually a nice thing to stick to. So if we decided it can only be reliably done by sacrificing usability on GitHub, we will do it.

Totally understood.

[slorber]

The source code is available at [Dockerfile](../../Dockerfile).

We don't to anything special with such link, we just render <a href="../../Dockerfile"/> which is something that browsers understand, now it's your responsibility to use the right path

Note that when you write: [Dockerfile](../../Dockerfile) we don't to anything, we just render <a href="../../Dockerfile"/> which is something that browsers understand, now it's your responsibility to use the right path

---

@felipecrs what I understand is that you want to create a download link to the Dockerfile right?

How can Docusaurus easily understand that you want to download a file, instead of navigating? You could have a page with slug: /Dockerfile too for example. I don't think we want to do anything magic here.

Here /Dockerfile is not a Docusaurus route, so you'd have to tell Docusaurus what to do exactly, because it's clearly not a SPA navigation (the default behavior of a link)

Some things worth giving a try:

This should create a download link:

The source code is available at [Dockerfile](@site/../Dockerfile)

This should navigate the current page to the Dockerfile, with regular HTML navigation (no SPA 404/broken link reported):

The source code is available at [Dockerfile](pathname://../../Dockerfile)

[slorber]

Note you may also like to display the dockerfile content directly as a code block in your docs:

https://docusaurus.io/docs/markdown-features/react#importing-code-snippets

[felipecrs]

@felipecrs what I understand is that you want to create a download link to the Dockerfile right?

Not really, I just want to link to the regular file in GitHub (or whatever other "baseUrl" I select). Example:

https://github.com/felipecrs/jenkins-agent-dind/blob/master/Dockerfile

(it's not meant to be downloaded, but to be viewed)

---

Regardless, I'm not saying Docusaurus is doing anything wrong. I'm just proposing something that in my view would be nice.

Perhaps, as @Josh-Cena, we can use a prefix to tell Docusaurus when to apply the relativeFilesBaseUrl, like:

The source code is available at [Dockerfile](file://../../Dockerfile). 

[slorber]

What I suggest is that you build (or find an existing one) a remark plugin to rewrite the links to whatever you want

Here's an inspiration to process md links:

https://github.com/remarkjs/remark-external-links/blob/main/index.js

You could replace:

The source code is available at [Dockerfile](../../Dockerfile). 

with

The source code is available at [Dockerfile](https://xyz.com/Dockerfile). 

Now it's up to you to provide your own conversion logic

[Josh-Cena]

Oh, that makes sense! I think we don't need to implement this in the core, since making a Remark plugin is pretty easy and flexible