document nix-env bug relating to multiple output installation

[dudebout]

Document the issue and workaround discussed in https://discourse.nixos.org/t/cant-find-a-way-to-install-gnumake-info-files-to-user-environment/3534 @doronbehar @lilyball @lheckemann

cc @dtzWill @matthewbauer as you both worked in that space in #35884

I encountered the issue trying to install netcat from nix-env, which is not easy anymore since it is provided via the nc output of libressl which is not part of outputsToInstall (#14708, #39634)

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[lheckemann]

still valid

[dudebout]

@lilyball, I don't remember much about this PR. I skimmed through your recommendations, and they make sense. At this point, you know more about the topic than I do, so you'd probably be the most appropriate person to take this forward.

…

On Mon, Jul 27, 2020, 4:26 PM Lily Ballard ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In doc/stdenv/multiple-output.xml <#76794 (comment)>: > @@ -27,34 +27,59 @@ <title>Installing a split package</title> <para> - When installing a package via <varname>systemPackages</varname> or <command>nix-env</command> you have several options: + When installing a package with multiple outputs, the package's <varname>meta.outputsToInstall</varname> attribute determines which outputs are actually installed. <varname>meta.outputsToInstall</varname> is a list whose <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix#L289">default installs binaries and the associated man pages</link>. The following sections describe ways to install different outputs. Linking to a particular line of the master ref risks the reference going stale. You can press y on GitHub to canonicalize the current URL, which will change it to a blob reference so the line number will always be accurate, at the cost of the linked file not necessarily reflecting what's in master anymore. We should probably canonicalize this. Note that since this PR was submitted, the referenced line number is in fact no longer accurate. The new correspond line number is 310, though we can go ahead and select the whole range too. ⬇️ Suggested change - When installing a package with multiple outputs, the package's <varname>meta.outputsToInstall</varname> attribute determines which outputs are actually installed. <varname>meta.outputsToInstall</varname> is a list whose <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix#L289">default installs binaries and the associated man pages</link>. The following sections describe ways to install different outputs. + When installing a package with multiple outputs, the package's <varname>meta.outputsToInstall</varname> attribute determines which outputs are actually installed. <varname>meta.outputsToInstall</varname> is a list whose <link xlink:href="https://github.com/NixOS/nixpkgs/blob/f1680774340d5443a1409c3421ced84ac1163ba9/pkgs/stdenv/generic/make-derivation.nix#L310-L320">default installs binaries and the associated man pages</link>. The following sections describe ways to install different outputs. ------------------------------ In doc/stdenv/multiple-output.xml <#76794 (comment)>: > </para> - <itemizedlist> - <listitem> - <para> - You can install particular outputs explicitly, as each is available in the Nix language as an attribute of the package. The <varname>outputs</varname> attribute contains a list of output names. This sentence seems useful, especially since this is the introduction to split packages ------------------------------ In doc/stdenv/multiple-output.xml <#76794 (comment)>: > - </listitem> - <listitem> - <para> - You can let it use the default outputs. These are handled by <varname>meta.outputsToInstall</varname> attribute that contains a list of output names. - </para> + <section xml:id="sec-multiple-outputs-installing-nixos"> + <title>Selecting outputs to install via NixOS</title> + + <para> + NixOS provides two ways to select the outputs to install for packages listed in <varname>environment.systemPackages</varname>: + </para> + + <itemizedlist> + <listitem> + <para> + The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install for <literal>info</literal> documentation or debug symbols for all packages. ⬇️ Suggested change - The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install for <literal>info</literal> documentation or debug symbols for all packages. + The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install <literal>info</literal> documentation or debug symbols for all packages. ------------------------------ In doc/stdenv/multiple-output.xml <#76794 (comment)>: > + <section xml:id="sec-multiple-outputs-installing-nixos"> + <title>Selecting outputs to install via NixOS</title> + + <para> + NixOS provides two ways to select the outputs to install for packages listed in <varname>environment.systemPackages</varname>: + </para> + + <itemizedlist> + <listitem> + <para> + The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install for <literal>info</literal> documentation or debug symbols for all packages. + </para> + </listitem> + <listitem> + <para> + The outputs can be listed as packages in <varname>environment.systemPackages</varname>. For example, the <literal>"out"</literal> and <literal>"info"</literal> outputs for the <varname>coreutils</varname> package can be installed by including <varname>coreutils</varname> and <varname>coreutils.info</varname> in <varname>environment.systemPackages</varname>. The fact that outputs are vended as attributes on the package hasn't been explained yet. That happens in the next section *Using a split package*. We may need to pull that first sentence from that section up to this section (not inside this list item, but in the intro paragraph, perhaps right after explaining the outputs attribute). ------------------------------ In doc/stdenv/multiple-output.xml <#76794 (comment)>: > <para> - NixOS provides configuration option <varname>environment.extraOutputsToInstall</varname> that allows adding extra outputs of <varname>environment.systemPackages</varname> atop the default ones. It's mainly meant for documentation and debug symbols, and it's also modified by specific options. + installs the <literal>"out"</literal> output (<varname>coreutils.meta.outputsToInstal</varname> is <literal>[ "out" ]</literal>) instead of the requested <literal>"info"</literal>. ⬇️ Suggested change - installs the <literal>"out"</literal> output (<varname>coreutils.meta.outputsToInstal</varname> is <literal>[ "out" ]</literal>) instead of the requested <literal>"info"</literal>. + installs the <literal>"out"</literal> output (<varname>coreutils.meta.outputsToInstall</varname> is <literal>[ "out" ]</literal>) instead of the requested <literal>"info"</literal>. ------------------------------ In doc/stdenv/multiple-output.xml <#76794 (comment)>: > + The only recourse to select an output with <command>nix-env</command> is to override the package's <varname>meta.outputsToInstall</varname>, using the functions described in <xref linkend="chap-overrides" />. For example, the following overlay installs the <literal>"out"</literal> and <literal>"info"</literal> outputs for the <varname>coreutils</varname> package: + </para> + +<programlisting>self: super: +{ + coreutils = super.coreutils.overrideAttrs (oldAttrs: { + meta = oldAttrs.meta // { outputsToInstall = [ "out" "info" ]; }; + }); +} It would be nice to change this to showing how to add info instead of overriding the existing outputs. coreutils doesn't currently have a man output (which is a shame, I'd like to be able to get the GNU coreutils manpages since darwin has the FreeBSD ones), but many other packages do so we shouldn't accidentally instruct the user to throw that away. ⬇️ Suggested change - The only recourse to select an output with <command>nix-env</command> is to override the package's <varname>meta.outputsToInstall</varname>, using the functions described in <xref linkend="chap-overrides" />. For example, the following overlay installs the <literal>"out"</literal> and <literal>"info"</literal> outputs for the <varname>coreutils</varname> package: - </para> - -<programlisting>self: super: -{ - coreutils = super.coreutils.overrideAttrs (oldAttrs: { - meta = oldAttrs.meta // { outputsToInstall = [ "out" "info" ]; }; - }); -} + The only recourse to select an output with <command>nix-env</command> is to override the package's <varname>meta.outputsToInstall</varname>, using the functions described in <xref linkend="chap-overrides" />. For example, the following overlay adds the <literal>"info"</literal> output for the <varname>coreutils</varname> package: + </para> + +<programlisting>self: super: +{ + coreutils = super.coreutils.overrideAttrs (oldAttrs: { + meta = oldAttrs.meta // { outputsToInstall = oldAttrs.meta.outputsToInstall or [ "out" ] ++ [ "info" ]; }; + }); +} ------------------------------ In pkgs/stdenv/generic/make-derivation.nix <#76794 (comment)>: > @@ -287,7 +287,8 @@ in rec { name = attrs.name or "${attrs.pname}-${attrs.version}"; # If the packager hasn't specified `outputsToInstall`, choose a default, - # which is the name of `p.bin or p.out or p`; + # which is the name of `p.bin or p.out or p` along with `p.man` when + # present; # if he has specified it, it will be overridden below in `// meta`. Since you're editing this, can we remove the unnecessarily gendered pronoun here? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#76794 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC5LGX77T3EDDUNTBEQRUTR5XPFTANCNFSM4KB4LVKQ> .

[dudebout]

@lilyball I have attempted to address all your concerns. Let me know if you need anything more.

[infinisil]

This is already mentioned in the paragraph above. Can probably remove the warning section.

[infinisil]

Looking good otherwise! @lilyball Can you take another look too?

[dudebout]

I see what you are saying, but I think this warrants a warning box. This is a very confusing bug from a user perspective and having a self-contained warning box describing the problem can save time for the reader. If we want to remove the warning box, the section needs to be tweaked because the warning box contains the example that is used in the following paragraph. On a related note, any idea on what can be done to push this PR past the finish line? This bug has been known for a long time, there was a TODO in the doc, I offered some content almost 8 months ago and I cannot seem to get traction to get this merged in.

…

On Sat, Aug 22, 2020 at 10:50 AM Silvan Mosberger ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In doc/stdenv/multiple-output.xml <#76794 (comment)>: > <para> - TODO: more about tweaking the attribute, etc. + <command>nix-env</command> silenty disregards the outputs selected by the user, and instead installs the outputs from <varname>meta.outputsToInstall</varname>. For example, This is already mentioned in the paragraph above. Can probably remove the warning section. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#76794 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC5LGTA4DMMHGK5OHZUDTLSB7LKTANCNFSM4KB4LVKQ> .

[infinisil]

I see what you are saying, but I think this warrants a warning box. This is a very confusing bug from a user perspective and having a self-contained warning box describing the problem can save time for the reader. If we want to remove the warning box, the section needs to be tweaked because the warning box contains the example that is used in the following paragraph.

Alright I see. I guess it's fine, even though the warning box contains part of what was said just before it.

On a related note, any idea on what can be done to push this PR past the finish line? This bug has been known for a long time, there was a TODO in the doc, I offered some content almost 8 months ago and I cannot seem to get traction to get this merged in.

That's what I'm here for :). I can merge this PR after the commits are squashed!

[dudebout]

@infinisil I have squashed the commits

[dudebout]

And re-squashed with trailing whitespaces removed