Building the manual requires building all of Nix

[fricklerhandwerk]

Problem

The build process for the manual is both cognitively involved and technically bottle-necked on first building the Nix binary. That is quite some overhead in terms of contributor onboarding time and turnaround, especially when jumping between PRs.

Proposal

Splice out the parts that dump the JSON with CLI definitions to be built separately just for the manual. Remove those hidden helper commands from the Nix binaries.

Priorities

Add 👍 to issues you find important.

[thufschmitt]

Splice out the parts that dump the JSON with CLI definitions to be built separately just for the manual

That looks very hard to do without significantly refactor the codebase (I mean, we could do it somewhat easily, but these parts would still depend on most of the codebase, so it wouldn't be useful at all). Still valuable, but by no mean an easy task.

[fricklerhandwerk]

• @edolstra: not in favor of this
  • sounds like large refactoring
  • would break separation of concerns if CLI and builtins were defined in an independent component
• @roberth: could do a light version. the goal is to improve the rebuild time
  • @edolstra: could do that directly. once did an experiment concatenating all source files and compiling without optimisation, which was a lot faster
    • this could work for just rendering the manual
• @Ericson2314: it's problematic that the same Nix code is processing the markdown outside and inside the executable
  • would be better if markdown is processed once before the build, and imported in to the executable
  • @fricklerhandwerk: agreed, removing that spaghetti would ease contributions
  • @roberth: there should be a way without a lot of code changes
    • @edolstra: not sure how that would work
• @roberth and @fricklerhandwerk will look into this

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-06-16-nix-team-meeting-minutes-63/29316/1

[fricklerhandwerk]

Discussed in the Nix team meeting:

• @roberth came up with a very non-invasive change, but still stuck on some linker issues
  • essentially building intermediate results as separate derivations
  • doesn't remove the complexity of the existing solution though
  • would be nice to have separate derivations for individual libraries
  • current approach adds some incrementaly at low cost, and is opt-in

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-06-30-nix-team-meeting-minutes-67/29835/1

[fricklerhandwerk]

Related: #9817

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-06-05-nix-team-meeting-minutes-150/46583/1