[RFC 0064] New Documentation Format for nixpkgs and NixOS

[infinisil]

Edit: See #64 (comment) for the state of this RFC

Many people from the Nix community are interested in evaluating alternatives to DocBook as the documentation format for nixpkgs/NixOS.
Through the process of this RFC, a potentially new doc format will be decided.

In short, with this RFC a set of requirements for a doc format is decided, after which candidates that fulfil them are collected and evaluated. A public poll is then held allowing everybody to vote which formats they'd prefer. The result of this poll is then to be used by the shepherds to come to a final decision.

Rendered

Pinging some people who have shown interest in this topic and/or are knowledgeable in certain formats: @edolstra @grahamc @domenkozar @zimbatm @alyssais @peterhoeg @danbst @chreekat

[edolstra]

FWIW, I don't want to move away from DocBook; at least not to the listed alternatives, which are all worse than DocBook for technical documentation.

[infinisil]

@edolstra As mentioned in the first paragraph, this is not the place for such discussions. If you have another format you'd prefer, feel free to suggest it, or feel free to write an objective overview of docbook for the RFC.

[grahamc]

Thank you Silvan for moving forward on this project. However, I feel this RFC is too early: This RFC is jumping to solutions, when we don't have a handle on (or agreement on) the problem's requirements. I feel this RFC is more akin to a technical survey of available documentation technology, potentially answering many questions about each individual tool -- without an idea of how to concretely determine if they meet our requirements. The RFC I would hope to see is an exploration and documentation of what our project requires out of its documentation tooling.

[infinisil]

I think all formats listed here are easily up to the task. If you have a concrete reason to believe one of them is infeasible, feel free to elaborate and we can remove it from the candidates if it's really a complete no-go. If in the end the chosen format is found to be infeasible, we will use the one on second place, and so on, as described in the RFC.

[grahamc]

I'm glad you think they are all up to the task! If we started first by evaluating requirements, we might agree. Evidently, Eelco strongly disagrees. I think this project is much more likely to succeed if we start with some requirements first.

Edit: I would not expect you to put any in the list which you didn't think were up to the task. However, we've discussed the problems with asciidoctor's closure size and it remains in this list. We cannot remove candidates if we don't have agreement on what the requirements are. We cannot evaluate candidates any more than superficially if we don't have agreement on what the requirements are, either.

[FRidh]

I agree with @grahamc we should list requirements first. Additionally, an RFC should specify which projects' documentation it would cover because the different projects have, although there's overlap, their own communities.

[zimbatm]

Just a reminder that a RFC's value is also in the associated discussion. It doesn't have to be perfect and can also dramatically change in it's form. @infinisil thanks for raising a subject which has been a recurring topic in the community.

[infinisil]

Sounds good, let's start with requirements and decide on candidates based on those. I'll start with some:

• Supports standard stuff like bold, italics, code, links, references (all formats support this)
• Inter-file references (I'm not sure if all formats support this)

I personally don't think small closure size is a hard requirement, as there's no way to define "small", and it can change over time too. I also think all formats have a passable closure size as of now. Asciidoctor is about 1GB, but Antora which can also handle Asciidoc is only like 300MB.

Edit: Also note that nixpkgs currently uses Pandoc to process markdown, which is over 2GB in closure size.

[grahamc]

I think the requirement gathering is significant enough to warrant its own RFC (and as @zimbatm pointed out, that could be this one), and quite probably a community video call about it.

In terms of closure size, I think the closest we can get to 50MB the better. For one datapoint, using jing in the XML build process was no good at its existing size, and I'm pretty sure that was quite a bit smaller than 300MB.

My list of requirements would be something like:

• errors while authoring documentation are trivial to spot, and don't require a debugger
• has an existing ecosystem of tools which already:
  • produces searchable, multi-page documentation as HTML
  • produces man pages
  • supports translations at some level (ie: internationalization)
  • takes 10 seconds or less to generate the full documentation set on a laptop, 1 minute or less on a raspberri pi 3b+
    • ideally, supports partial documentation rebuilding or even live-reloading for faster iteration

A nice-to-have would be: rendered well in GitHub's "visual diff".

[jtojnar]

Some features I like to use in our current toolchain:

• Annotations/Links inside code (e.g. linking options in configuration snippets)
• Ability to make arbitrary command line prompt in code listings non-copyable (see docbook.rocks example, I always hate it when I coppy multiline bash snippet and then have to delete $)
• Create anchors anywhere (if not inside paragraphs, at least for list items)
• Automatic link labels (again, for options)

[FRidh]

Additionally, an RFC should specify which projects' documentation it would cover because the different projects have, although there's overlap, their own communities.

To clarify my point here. I don't think the whole Nix/NixOS/Nixpkgs community should decide for e.g. for Nix or Hydra or NixOps what the docs should be like. We have different projects composed of different contributors and within those groups of contributors you have a subset that writes docs. Let the projects decide this for themselves.

Furthermore, I think that if we would decide on a process for the format, we should also reconsider exactly what manuals we want to have. There have often been talk about user manuals, tutorials, reference manuals. This is something that I think adds value.

[infinisil]

@FRidh I think we should at least have the same format for all of nixpkgs including NixOS, otherwise we won't get much out of it, since nixpkgs gets the majority of use and contributions.

Nix itself would benefit from the same format as nixpkgs too, at least for the builtins.* docs defined in it, which we could then easily include in the lib.* nixpkgs docs, which is where a lot of builtins.* are reexported.

Also to consider is that many projects define NixOS options, which are all processed and rendered in a single evaluation, so it would simplify things if all had the same format. However I think with some better doc tooling we should also be able to allow projects to choose their own format for options, even if they're rendered at the same time as the NixOS ones.

[infinisil]

For now I think keeping it to nixpkgs only (including NixOS) should be good.

[Shados]

@infinisil For what it's worth, markdown does have a standardized specification and test suite in CommonMark, which is fairly near a finalised 1.0 release.

[infinisil]

@Shados However even mentioned in the page you linked, the original Markdown does not have a standard. But yeah, CommonMark would almost certainly be the standardized markdown to use because of what you said, and I also just learned that it's what Sphinx supports (in addition to reST), and that GitHub's markdown is a strict superset of CommonMark and they are intending to have full 1.0 conformance. I'll change the RFC to reflect that CommonMark would be the Markdown to use for these reasons.

[zimbatm]

A few links to existing discussions so we don't go over the same arguments again:

• https://discourse.nixos.org/t/documentation-improvements/3111
• https://discourse.nixos.org/t/documentation-format/4650

[domenkozar]

• supports syntax highlighting (of Nix)
• wide editor integration (this could be added to the errors bullet point)
• active community supporting the tooling infrastructure
• good conversion story from docbook
• ideally a good search integration
• low work cost for integration (this one is really important as docbook fails hard)

[infinisil]

Nice, what do you mean by the last point though? That we don't need to write too many customizations and supporting code to make it work?

[domenkozar]

I think poll is too vague, people should discuss on this RFC exactly what they'd like and then the shepherd team makes a decision.

[infinisil]

I think it's important to involve as many people from the community as possible, because they will be the ones writing and reading the docs. Unfortunately RFC's are known to be bad places for involving many people, since they tend to get long quickly and less-active people get overshadowed by very vocal people.

As you suggested to me though, having the poll happen before the RFC is accepted and using it as an input for the shepherd team might be a good idea.

[infinisil]

I changed the RFC to reflect this

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/documentation-format/4650/14

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/marketing-team-meeting-minutes-5/7303/6

[danielstaleiny]

I would love to see org-mode used as suggested by @oxij. I think we should think about maintainability of docs, if they are hard to maintain and hard to navigate, it will be much harder to contribute. I would also like to suggest support for on page linking to documentation file, so when somebody decide to contribute to docs, it would automatically point to correct file instead of searching for it in doc repo. (I get that could be hard, but I would love to see it, if possible)

one more suggestion I would like to see, be able to use git-based CMS to modify docs. if it is possible technically.

and here is taste of how could workflow look like. technical documentation with org-mode

I could also make demo for you, if you would like that.

[jtojnar]

@danielstaleiny Deadline for demos is June 30, feel free to send it if you want it considered.

Only doc formats that have a demo until the deadline of June 30 will be included in the survey and can be considered candidates. We can have a decision for the next NixOS release.

[domenkozar]

I've been playing around with Sphinx at https://github.com/domenkozar/nix.dev

And I'm really happy with results:

• there's ./live script that watches local changes and auto-reloads your browser (<1s)
• netlify builds PRs and live site and deploys it to https://nix.dev

I did also find https://myst-parser.readthedocs.io/en/latest/, which would really allow to join the two:

• sphinx amazing tooling + documentation directives
• markdown as a basic format

But that's something to further explore.

We have a meeting in 15min!

[asymmetric]

Update on the RFC

The shepherd team has concluded that this RFC, as originally proposed, has
little chances of arriving at a satisfying outcome in a timely manner. The
formulation is too open, leading to a confusing discussion, and the process the
shepherd team itself proposed (a survey, followed by a decision) has several
downsides too:

• it would drag on the decision, whereas we think the most likely candidate is
  already clear
• it asks everybody for their opinion, independent of how much time they would
  be actually willing to invest in writing documentation and tooling around it

In our discussion, it seemed clear that the strongest candidate is ReST, paired
with Sphinx, both for its technical merits and for the amount of resources within the community that would be available to make the integration work.

A new RFC is being drafted, which proposes changing the documentation format to
ReST. That RFC will be exclusively used to discuss the benefits of the one
format.

This RFC will stay open, in case someone wants to discuss (or propose) other
options further, but at this point, we don't plan on running the survey anymore.

If the new, yet-to-be-published RFC will be accepted, this RFC will be
superseded. If it won't, then we will revisit this RFC.

We (@asymmetric, @infinisil, @jtojnar and @domenkozar) will nominate ourselves as
shepherds for the new RFC, but anyone else is also able to nominate themselves
and be picked. We just think that, given the time we have already spent on the
topic, it makes sense for us to continue until the end.

We are aware that some people will be disappointed about the change in course,
but we think this is in the best interest of the project and its users :)

[lheckemann]

For clarity's sake, we decided in the RFCSC that it would be best to close this RFC PR to show that it will not be progressing anytime soon. We are of course happy to reopen it should the need arise.

[Ekleog]

I'm far from being against the choice to use ReST, or to come to the end of this discussion rapidly… but I must say that it looks not-nice, to me, to close this RFC less than a week after saying that there is one more month to send in demos, when people wanted to suggest other documentation formats that haven't been considered yet.

Nixpkgs has been using docbook for I don't know how many years, surely the switch could wait just one more month to avoid hurting people that had potentially been working towards the deadline that was expected to hold? There must definitely be a deadline, but one had already been set, and closing this ahead of time… I'm not sure I understand how the speed benefits outweigh the human drawbacks.

[domenkozar]

@Ekleog we've tried to communicate that this is not the end of the discussion. There's also clarification why we've changed the course, given the downside.

The new RFC still needs to be written and go through the process, which I don't expect to happen in less than a month at all.

Meanwhile anyone is welcome to open a new RFC with proposed changes to documentation. It's going to be more focused discussion on each RFC between the current state and the proposed solution, rather than debating all at once.

Nixpkgs has been using docbook for I don't know how many years, surely the switch could wait just one more month to avoid hurting people that had potentially been working towards the deadline that was expected to hold?

There's a lot of damage done with not doing this sooner. Sticking with docbook is a choice. We had a number of meetings on marketing team where it was unclear how to continue with tutorials, etc because we're waiting on the resolution of this RFC. And note that only accepting the RFC just means we can then start all the work that's needed. So ultimately months to just make a call.

The main reason is that we'd like to have something ready for 20.09 so that documentation is a bit better in next release rather than waiting another full year.

To avoid debating possibilities, If someone has been writing a proposal for a different tool, please share what you have here and I'll help crafting ReST and your proposal.

[davidak]

@domenkozar was Markdown using MyST considered? Do you still plan to try it out? It looks nice.

[domenkozar]

@davidak I plan to try it out, but it does seem very early stage. We did discuss it and markdown compiles to ReST which then Sphinx takes to output format.

That means users have to write Markdown and Sphinx, which is something we are trying to avoid to have two formats.

Still need to dive into it though to form an opinion.

[oxij]

@domenkozar How are you going to check documentation against the code (i.e. Literate Programming) with ReST and Sphinx? AFAIK (after reading https://docutils.sourceforge.io/docs/user/rst/quickref.html), ReST and, thus, Sphinx only do "doctest" blocks, not proper literate code blocks, which, I think, is not what you (would probably eventually) want with NixOS, since, I expect, you would eventually want to extract multiple `.nix` files from a single document section and then test them against the rest of Nixpkgs (which, I'm pretty sure, is #1 thing you would want after banishing XML editing from the doc writing process). For a typical example: Emacs and GHC sections of the manual provide code samples for both `configuration.nix` and `config.nix` files, how are you going to describe and test those with ReST and Sphinx?

[domenkozar]

@FRidh already figured that out, see as an example https://fridh.github.io/nix-tutorials/tutorials/01-nix/02-using-experimental-nix-command.html#Building-a-package-with-nix-build

[FRidh]

@domenkozar need to be careful here. Those tutorials are written as Jupyter notebooks and then exported with Sphinx. Because nix is used the docs are generated outside of a nix-build.

Until we can call nix-build inside a nix-build I don't think there is any other way for literate programming.

Note there is an extension for Sphinx that allows you to create executable sections backed by Jupyter. But again, if you want to invoke a nix-build, then generation of the docs needs to be impure.

[domenkozar]

@FRidh thanks for jumping in - my point being that this is doable. It's not the immediate goal of the RFC to implement that, but rather to pick an ecosystem that will allow such things.

Sphinx is quite extensive and things like https://sphinx-litprog.readthedocs.io/en/stable/ are already available for Python.

[FRidh]

Definitely. There is good support for extending and already a rich ecosystem of extensions. I can imagine creating a Nix domain for Sphinx for documenting libraries, and a Nix kernel for Jupyter for evaluating Nix expressions directly in the Sphinx docs.

[FRidh]

So an interesting follow-up question could be. Do we find literate programming so important we are willing to perform impure builds of the docs or not?

Note we can do both. With a switch we can simply not evaluate those cells.

[domenkozar]

Nix 2.4 will ship recursive Nix that might help with that - but I suggest we leave discussion for when the new RFC is written and accepted. The goal of the RFC should be to pick something with which we can get very far in the next few years.

[bhipple]

I do not consider literate programming to be a must-have. The Python and Rust chapters of the manual today are written in Markdown, and examples are just copy-pasted into the docs. It's not ideal, perhaps; but it works OK with 0 overhead and no tooling complexity.

[Mic92]

Who is writing the new RFC?

[asymmetric]

There's a new RFC proposing CommonMark.

[domenkozar]

As I said privately to some people, I was waiting for this RFC to be published (and didn't want to spread rumors before it was made):

I'm not going to make the RST proposal as I have no intention of competing two formats, I'm happy we just get it done. (although again the RFC misses the whole point of choosing tooling as way more important than the format).

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-04-13-documentation-team-meeting-notes-41/27264/1