nixos-render-docs: cannot render commonmark

[hsjobeki]

I hope for some help from the community here ❤️

Describe the bug

If you feed the nixos-render-doc with regular commonmark; for example:

Tagline

# Example

Foo content

This won't render because the tool expects an id for each heading.

This is a problem with doc-comments and documentation that originates from the nix source code. Nobody really wants to write heading ids for each heading in a doc-comment. which should also be unique across whole nixpkgs.

Expected behavior

Commonmark as of RFC72 is the official format for documentation.
Also with RFC145 we agreed that the Commonmark format is a valid format for documentation in nix doc-comments.

A related discussion about this was in one of the recent documentation team meetings.
https://discourse.nixos.org/t/2024-01-04-documentation-team-meeting-notes-102/37754/1

There are some different perspectives on how you could perceive this kind of problem.

1. I'd expect the "lint stage" (error for missing IDs) for handwritten stuff in (/doc) to be separate from the rendering tooling itself.
2. I'd not want to write and invent headings ids in doc-comments.

bad.nix

{
/**

Tagline

# Example {#section-example-2309}

foo
*/
foo = id;
}

Those heading ids could be optional / and or autogenerated for doc-comments.

Notify maintainers

@pennae @SuperSandro2000

What do you think

I looked for quite some hours into nixos-render-docs. But I couldn't manage to find a proper solution for this.

I prepared these PRs:

• Automigrate nixpkgs to commonmark comments: treewide: migrate doc comments #262987
• Shift heading ids and support commonmark in nixdoc: init support for doc-comments nix-community/nixdoc#91

nixos-render-docs is the last piece that is missing to clear the migration path.

It would be so nice if you can help me tackle the problem. ❤️

--

Add a 👍 reaction to issues you find important.

[pennae]

• I'd expect the "lint stage" (error for missing IDs) for handwritten stuff in (/doc) to be separate from the rendering tooling itself.

it's not just a lint. headings always need ids to be linked from a TOC, and those ids should be stable so you can pass those links around to other people. in the general case we want missing ids to be a hard error instead of generating something because we can't trust any generation method to remain stable without keeping a bunch of conflict-ridden state in nixpkgs.

• I'd not want to write and invent headings ids in doc-comments.

and that's totally fair! for nixdoc comments it'd be fine to enable auto-generated heading ids since nixdoc is a trustworthy tool. why not add an auto-id-prefix="string" tag for {=include=} blocks that can be used for nixdoc-generated documentation? could be as simple as another phase prior to id collection that generateds ids from heading text where they aren't given explicitly (taking into account all parent headings to the nearest auto-id-prefix tag).

[hsjobeki]

@pennae what do you mean by tag?

I found this (not sure if it is the right file)

nixpkgs/doc/functions.md

Line 5 in 3763c9f

```{=include=} sections

Would that become something like that?

# Functions reference {#chap-functions}

The nixpkgs repository has several utility functions to manipulate Nix expressions.

```{=include=} sections auto-id-prefix="sec-functions"
functions/library.md
functions/generators.section.md
functions/debug.section.md
functions/prefer-remote-fetch.section.md
functions/nix-gitignore.section.md
```

Could you point me to the right file where the id generation would happen?

[pennae]

Would that become something like that?

pretty much, yep. we'd suggest using sec-functions- as the id prefix though ;)

Could you point me to the right file where the id generation would happen?

you'll probably want to expand on this infrastructure with a global namespaces that's handled in the base converter _parse_include_blocks (in this case). might be sufficient to do some postprocessing right there and assign ids where none are given explicitly.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-01-25-documentation-team-meeting-notes-106/38792/1