Pinning nixpkgs in configuration.nix?

[jluttine]

Edit by @nh2: I turned this into an issue to track missing manual documentation, with acceptance criteria for closing in #62832 (comment)

---

Issue description

I searched for information on how to pin system-wide nixpkgs in configuration.nix, but I couldn't find. I found a lot of information on how to pin nixpkgs for nix-shell or nix-build though. I'm probably just missing something obvious because I'm expecting this to be something very trivial..

The point is that I want to specify nixpkgs commit in configuration.nix because the configuration is quite tightly tied to nixpkgs version. For instance, module options might change, packages might be removed/added/renamed. Also, for easier reproducability, it'd be great if configuration.nix contained all the relevant information so that when passing the file to someone, I don't need to also specify the nixpkgs version and the receiver doesn't have to figure out how to get that nixpkgs version. They would just put the file in place and run nixos-rebuild switch, and that's it.

I tried:

  nix.nixPath = [
    "nixpkgs=https://github.com/NixOS/nixpkgs/archive/4ab1c14714fc97a27655f3a6877386da3cb237bc.tar.gz"
    "nixos-config=/etc/nixos/configuration.nix"
  ];

But this has two shortcomings:

• There's no hash for verifyingthe content of the URL. I guess this could be easily solved by using builtins.fetchTarball, so no real problem here.
• More importantly, when I modify nixpkgs version, I need to run nixos-rebuild switch twice for it to take effect. The first call updates my system wide nixpkgs and the second call updates my system with this new nixpkgs. As far as I understand, this is a bit problematic:
  • First, what if configuration.nix isn't compatible with the nixpkgs version that is currently active in the system? Then the build fails and one cannot upgrade the system by just calling nixos-rebuild switch although configuration.nix would build just fine with the nixpkgs it specifies itself.
  • Second, IIUC, nixos-rebuild boot or nixos-rebuild build wouldn't use nixpkgs specified in configuration.nix but what is currently active in the system.

So what is the correct way of specifying nixpkgs commit in configuration.nix?

[Ma27]

You probably want to use nixpkgs.pkgs as well which sets _module.args.pkgs, so each module will use the pkgs value declared here. Or in other words whenever you have { pkgs, ... }: in a module, pkgs will be equal to the value of nixpkgs.pkgs.

On my local setup the pinning looks like this:

# ~/nixos-config/nixos.nix
{ config, ... }:
let
  nixpkgs = builtins.fetchTarball {
    url = https://github.com/NixOS/nixpkgs/archive/hash-of-the-commit-to-pin.tar.gz;
    sha256 = "checksum";
  };
in {
  # no need to inject `nixpkgs.overlays` here, this will be done by NixOS
  nixpkgs.pkgs = import "${nixpkgs}" {
    inherit (config.nixpkgs) config;
  };

  nix.nixPath = [ "nixpkgs=${nixpkgs}" ];
}

More information can be found at <nixpkgs/nixos/modules/misc/nixpkgs.nix>.

Does this help?

[jluttine]

Thanks! That'll probably improve the solution, but as far as I understand, the latter shortcoming still exists.. That is, for instance, nixos-rebuild switch needs to be run twice for configuration.nix to use the changed nixpkgs for modules and module options. For pkgs, that should work. So I don't still see this as a "correct" solution.

[Ma27]

That is, for instance, nixos-rebuild switch needs to be run twice for configuration.nix to use the changed nixpkgs for modules and module options. For pkgs, that should work. So I don't still see this as a "correct" solution.

Yeah, forgot to mention that, sorry. The problem is that we now need some sort of option which overrides the entire module system including the code which evaluates the module system and I'm not sure if that's even possible atm.

I'm pretty sure that right now you have to (1) incorporate a new nixpkgs rev and rebuild and (2) modify the config accordingly and rebuild. In case you run into some kind of edge case where even that breaks, you could still work around with imports and disabledModules.

[jtojnar]

There might be some unsafe eval hacks but I think creating your own script that will call nixos-rebuild with the correct -I nixpkgs=… flag is cleaner.

Related: #35411 (comment)

[jluttine]

According to man page, nixos-rebuild doesn't take -I flag.

If pinning nixpkgs in configuration.nix isn't currently possible, I think it is something that should be implemented. Is there something that makes it difficult/impossible or is it just something nobody has bothered to do yet (or is there some solution already)?

[jtojnar]

It does support the flag. I guess the following line refers to that:

In addition, nixos-rebuild accepts various Nix-related flags

---

The issue with defining nixpkgs in configuration.nix is that by the time we evaluate the expression, we already need to know NIX_PATH to be able to use angle bracket paths. We would need to split the config parsing to two stages similarly to what we already do with virtualization options.

[Ma27]

It does support the flag. I guess the following line refers to that:

I'm pretty sure as well. But it seems as this sentence can be missed easily, so I'll open a PR later which adds some more helpful nix flags to the man page.

There might be some unsafe eval hacks but I think creating your own script that will call nixos-rebuild with the correct -I nixpkgs=… flag is cleaner.

That's an interesting approach! But may I ask how you update your Nix path then?

The issue with defining nixpkgs in configuration.nix is that by the time we evaluate the expression, we already need to know NIX_PATH to be able to use angle bracket paths. We would need to split the config parsing to two stages similarly to what we already do with virtualization options.

That's basically what I tried to explain in #62832 (comment).

From my point of view it's not really worth to make the module system even more complex to achieve this, I'm using the approach with two rebuilds for a new nixpkgs for at least a year and it works pretty great here, however we should improve the documentation (either in the manual or in the wiki) before closing this issue.

[jtojnar]

That's an interesting approach! But may I ask how you update your Nix path then?

You would update it in the file as outlined in #35411 (comment)

Or you can combine it with the hashing:

#!/bin/sh
nixpkgs=$(nix-build --no-out-link -E '(builtins.fetchTarball {
    url = https://github.com/NixOS/nixpkgs/archive/hash-of-the-commit-to-pin.tar.gz;
    sha256 = "checksum";
  })')

nixos-rebuild -I nixos-config=configuration.nix -I nixpkgs=$nixpkgs "$@"

or store the hash and url in a separate file and import it to the script through builtins.fromJSON (builtins.readFile ./nixpkgs.json).

[Ma27]

Oh... of course... with -I the path is available as <nixpkgs> 😅

But @jluttine is this sufficient for you? In that case I'd prepare a PR to update the docs accordingly next week.

[jluttine]

I suppose that'll work, thanks! But I would still love to see some better built-in support for pinning the nixpkgs commit..

In that example:

• do I need to specify -I nixos-config=...?
• do I need to specify nix.nixPath in configuration.nix?
• is that -I nixpkgs=... in nixos-rebuild sufficient for setting the nixpkgs for all subsequent calls to nix-shell, nixos-rebuild etc or do I need to set nix.nixPath or something else for that?

How about this: Could I pin nixpkgs with nix-channel somehow? Some very sloppy pseudo sketching:

nix-channel --add mynixpkgs.nix nixos

And mynixpkgs.nix would be something like:

builtins.fetchTarball {....}

Then, changing nixpkgs commit would require modiyfying mynixpkgs.nix, calling nix-channel and then I can run any nixos-rebuild or nix-shell or anything consistently. Does this make sense?

[jtojnar]

• do I need to specify -I nixos-config=...?

Only if you set nix.nixPath. Otherwise it defaults to /etc/nixos/configuration.nix. I would do it though, so that you can checkout the repo anywhere you want.

• do I need to specify nix.nixPath in configuration.nix?

You might want that if you want other Nix commands to use that pinned repo. Probably something like the following might work.

nix.nixPath = [ "nixpkgs=${<nixpkgs>}" ];

• is that -I nixpkgs=... in nixos-rebuild sufficient for setting the nixpkgs for all subsequent calls to nix-shell, nixos-rebuild etc or do I need to set nix.nixPath or something else for that?

No, -I flags only apply to the command they are passed to. They do not even apply recursively (e.g. if you run nix-build in nix-shell with -I you will want to pass it to nix-build as well). So setting nix.nixPath is probably desirable, see the previous point.

How about this: Could I pin nixpkgs with nix-channel somehow? Some very sloppy pseudo sketching:

nix-channel --add mynixpkgs.nix nixos

And mynixpkgs.nix would be something like:

builtins.fetchTarball {....}

Then, changing nixpkgs commit would require modiyfying mynixpkgs.nix, calling nix-channel and then I can run any nixos-rebuild or nix-shell or anything consistently. Does this make sense?

You could use nix-channel --add https://github.com/NixOS/nixpkgs/archive/hash-of-the-commit-to-pin.tar.gz nixos but Nix would not know the URI is immutable and would try to re-download it every once in a while. Also there would not be an hash check.

Your solution might work but I am not familiar enough with nix-channel to says so conclusively. You would need to try it or read the source code to know for sure.

[lheckemann]

(I've only skimmed the other comments, sorry if I'm overlooking something)

What about this?

{ pkgs, ... }: {
  nixpkgs.pkgs = import (builtins.fetchTarball https://nixos.org/channels/nixos-19.03/nixexprs.tar.xz);
  nix.nixPath = ["nixpkgs=${pkgs.path}"];
}

Replacing the builtins.fetchTarball (…) with a commit-specific URL and specifying the hash for pinning purposes?

Just realised that this has the same problem as mentioned before.

[lheckemann]

mumble, mumble, cough, NixOS/rfcs#49

[zimbatm]

I think it was a mistake to make nixpkgs configurable from NixOS (eg: nixpkgs.overlays).

Without that option it would be possible to spawn multiple nixos configurations with a nixpkgs function: pkgs.mkNixOS ./configuration.nix.

But since we make it configurable from inside the configuration.nix, it means that nixpkgs is forced to be re-evaluated entirely for each machine. This means that deploying multiple machines becomes a O(N) operation in terms of time and memory instead of something closer to O(log(N)).

[Ma27]

Just wondering if it would still be possible to add overlays (I mean, the package set consists of overlays with a fixpoint, so in theory it would be possible to allow something like this, however I'm not sure if that's actually a good idea and even possible as it's a while ago since I had to dig into this part of nixpkgs internals :) ). I mainly bring this up because I actually consider it a good thing to allow configurations to add e.g. patches (not all modules have a .package option) to existing packages that are needed to get a working system in the end.

But do you have any idea/plan on how it would be possible to migrate to such an approach? I'm not sure if such a change is possible atm, but I guess I'd be interested in something like this :)

But unless there's anything further to discuss, I'd propose to close this as this issue is merely a question and it seems as everything needed was said :)

[zimbatm]

Ok we're getting out of topic :)

Just to answer your question, one possibility might be to introduce a /etc/nixos/default.nix:

let
  nixpkgs = <nixpkgs>; # pin nixpkgs here if you want
  pkgs = import nixpkgs {
    config = {}; # potentially override the system config here
    overlays = []; # <- set your overlays here
  };
in
  pkgs.mkNixOS (import ./configuration.nix)

The overlays are not configurable from nixos modules anymore but that might be a bad idea anyways.

[colemickens]

I have sort of hacked my way though this. It's more complicated than "just pinning nixpkgs in nixos-configuration.nix" but it works. https://github.com/colemickens/nixcfg I actually have my imports and overlays effectively pinned as well, along with a update.sh that updates those references. See lib.nix for how I eval nixpkgs with overlays applied, and follow default.nix to see how it all works together. Finally, nixup.sh builds the system and activates it without using nixos-rebuild.

(It looks like I still have nixPath set inside, but it's effectively not used, anywhere I do rebuilds you'll see that I unset NIX_PATH.)

[davidak]

@zimbatm i want to build an ISO using the configuration. When i use pinning like you described, i get this error:

[davidak@ethmoid:~/code/nixos-config-greenbone/machines/targets-image]$ time nix-build -A config.system.build.isoImage
error: attribute 'mkNixOS' missing, at /home/davidak/code/nixos-config-greenbone/machines/targets-image/default.nix:8:3

Do i have to do it differently?

It would be great to have a working example documented!

[zimbatm]

Sorry that was a hypothetical example! Your best bet for now is to use the nixos-generators project right now.

[davidak]

@zimbatm how does it help with pinning? I see no option for that.

When it's just a hypothetical example, can the issue be reopened until a solution is implemented?

[davidak]

It is now possible after this is implemented: nix-community/nixos-generators#31

Using -I nixpkgs=URL

For example i build an ISO image using an unmerged PR (branch on my fork):

nixos-generate -I nixpkgs=https://api.github.com/repos/davidak/nixpkgs/tarball/backport_nixos-containers-TimeoutStartSec -c /home/davidak/code/nixos-config-greenbone/machines/targets-image/configuration.nix -f iso

[fricklerhandwerk]

one possibility might be to introduce a /etc/nixos/default.nix:

let
  nixpkgs = <nixpkgs>; # pin nixpkgs here if you want
  pkgs = import nixpkgs {
    config = {}; # potentially override the system config here
    overlays = []; # <- set your overlays here
  };
in
  pkgs.mkNixOS (import ./configuration.nix)

The overlays are not configurable from nixos modules anymore but that might be a bad idea anyways.

This example is not hypothetical, since mkNixOS already exists in the form of (import <nixpkgs/nixos> {}).system. In practice it could look like this:

$PREFIX/nixos/default.nix:

let
  nixos = <nixpkgs/nixos>; # pin NixOS here if you want
  configuration = /path/to/config.nix; # no need for `NIXOS_CONFIG`
in
import nixos { inherit configuration; }

Then call nixos-rebuild -I nixpkgs=$PREFIX to build the system using configuration modules and nixos-rebuild from the pinned version.

Note though that it will break the fallback cases when nix cannot build a newer version of itself.

For convenience I would add a wrapper to environment.systemPackages:

{ config, pkgs, ...}:
let
  nixos-rebuild = pkgs.writeScriptBin "nixos-rebuild" ''
    #!${pkgs.stdenv.shell}
    exec ${config.system.build.nixos-rebuild}/bin/nixos-rebuild -I $PREFIX $@
  '';
in
{
  config.environment.systemPackages = [ nixos-rebuild ];
}

When using autoUpgrade you have to add autoUpgrade.flags = [ "-I" "$PREFIX" ];. Yes, auto-upgrading makes sense even in this setting, for example if you want to impurely track the unstable branch.

I would like to stress for clarification that we really want to pin a nixos version, which in turn specifies the nixpkgs version the configuration modules depend on.

While the OS distribution and the package collection are two distinct things, right now they live in the same repository "for consistency" (whatever that means). So the way of specifying the dependency is a bit obfuscated, and although that is not essential, it was misleading me when working this out. In hindsight, the warning on nixpkgs.pkgs totally makes sense. Overriding that option is what we usually do not want.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/pinning-nixpkgs-for-nixos-rebuild/6473/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/pinning-nixpkgs-for-nixos-rebuild/6473/3

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/pinning-nixpkgs-for-nixos-rebuild/6473/5

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/build-nixos-config-without-environment-dependencies-and-have-nixos-option-nixos-rebuild-support/6940/3

[toraritte]

From the Nix flakes proposal:

The idea is that your NixOS system is itself a flake, so its flake.lock pins the exact version of Nixpkgs.
See example code there.

[nh2]

When it's just a hypothetical example, can the issue be reopened until a solution is implemented?

I am reopening this issue because this question continues to be asked by new NixOS users.

Acceptance criteria to close:

• The NixOS manual describes sensible methods how to pin nixpkgs.
  • Currently, searching pin does not even produce any related results.
  • It only mentiones the -I approach in a side note of "Getting the Sources" ("If you want to rebuild your system using your (modified) sources"), but does not give recommendation of how to then udpated those sources or that this can be a sensible way to pin your system.
• It is made very clear whether nixpkgs can be pinned from configuration.nix without drawbacks mentioned in this issue, such as having to nixos-rebuild twice (nix.nixPath), or some subcommands of nixos-rebuild failing to work, and without having to resort to currently-experimental features that you cannot get with current NixOS default configuration (such as flakes).
• The manual explains that configuration.nix is included by nixpkgs, not the other way around, which many NixOS users don't know for a while, and which is the reason why you cannot easily override nixpkgs from configuration.nix.

---

Current methods that I personally know work, which should be documented with pros and cons in the manual:

• -I flag with git checkout of nixpkgs at /etc/nixos/nixpkgs, and running

  sudo nixos-rebuild -I nixpkgs=/etc/nixos/nixpkgs switch

  • To update, you use git fetch and git checkout/git rebase on your checkout
  • Benefit: No unnecessary downloading.
  • Benefit: Makes hacking nixpkgs very easy, because you already have a git checkout.
• -I flag with URL pinning the commit:

  sudo nixos-rebuild -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/6dc03726f61868c0b8020e9ca98ac71972528d8f.tar.gz switch

  • Drawback: Does not put the commit in a file, need to write a wrapper script to achieve that.
  • Drawback: Nix will re-download that tarball every now and then (depending on cache setting), because it is not pinned with a hash (even though there's a hash in the URL, nix doesn't know that this is a stable link).
  • Passing -I nixpkgs=... will make that the nixpkgs or nixos channel in sudo nix-channel --list is ignored; you might as well sudo nix-channel --remove them. However, if you do this, then sudo nix-shell -p to quickly try out packages will not work unless you also give it your pin with -I (without the -I it would pull from that channel, not your pin).

[lheckemann]

We have a module that includes the nixpkgs used to build the system in the system's closure, links it to /run/nixpkgs, and sets NIX_PATH to nixpkgs=/run/nixpkgs: https://github.com/mayflower/nixexprs/blob/8d3cc13295a310cb7af7762723b806d3e6bdace4/modules/copy-nixpkgs.nix

• Advantage: Provides the exact nixpkgs used to build the system
• Advantage: Uses ambient nixpkgs while allowing overriding using -I
• Advantage: Provides a consistent nixpkgs for use outside building the system (i.e. nix-shell -p will use the same nixpkgs)
• Advantage: No downloads required
• Advantage: Does not require multiple rebuilds or rebooting/relogging for environment changes to take effect, after initial setup
• Disadvantage: uses more space, especially if rebuilding frequently with small changes (this is also very relevant if deploying to remote hosts via the internet with slow upload rates)
• Disadvantage: does not include git metadata
• Disadvantage: clobbers NIX_PATH
• Disadvantage: no declarative pin written to a file

[NorfairKing]

@nh2 asked me to contribute what I use:

This is in the shell.nix in my config repo:

  shellHook = ''
    rebuild () {
      sudo nixos-rebuild -I "nixpkgs=${sources.nixpkgs}" $@
    }
  '';

And this is in my aliases file:

rebuild () {
  cd ~/src/nix-depot && sudo echo "getting sudo early" && nix-shell --run "rebuild $@"
}

[cxandru]

@nh2 asked me to contribute what I use:

This is in the shell.nix in my config repo:

  shellHook = ''
    rebuild () {
      sudo nixos-rebuild -I "nixpkgs=${sources.nixpkgs}" $@
    }
  '';

what functions return a set accepted as an entry in sources set? This is why it sucks that nix doesn't have a type system. I find the docs difficult to navigate so I have to ask. And what is the advantage over just storing a reference as a reference to the tar archive on github and using that in "nixpkgs=${nixpkgsGitHubArchiveUrl}"?

And this is in my aliases file:

rebuild () {
  cd ~/src/nix-depot && sudo echo "getting sudo early" && nix-shell --run "rebuild $@"
}

[cxandru]

@cxandru replying this myself: sources refers to a file generated by niv.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/confused-about-how-vscode-extensions-versioning-works-in-nixos/17608/6

[jluttine]

Could this be made much simpler if nixos-rebuild would start from configuration.nix and that file would need to import whatever it wants to import, typically nixpkgs somewhere? I mean, running nixos-rebuild switch would read /etc/nixos/configuration.nix and that file would need to import, for instance, nixpkgs. This would make it trivial to use whatever nixpkgs. I don't actually understand why it isn't like this, but there's probably a very good reason why. I'd like to understand that.

[lheckemann]

@jluttine the current Flakes support of nixos-rebuild works kind of like that, practically mandating a pin (via flake.lock): the flake has an output named nixosConfigurations which contains named fully-evaluated NixOS configs, and they are applied by nixos-rebuild --flake /path/to/flake#configName. What do you think of that?

[jluttine]

Haven't really used flakes, I have preferred using plain nix/NixOS stuff..

For the record, my current solution in configuration.nix is:

let
  nixpkgs = builtins.fetchTarball {
    name = "nixpkgs-2023-01-10";
    # nixos-unstable branch Jan 10, 2023
    url = "https://github.com/NixOS/nixpkgs/archive/c07552f6f7d4eead7806645ec03f7f1eb71ba6bd.tar.gz";
    sha256 = "sha256:1lw6wa6akj16ia6jjqb9dhq0yx6rkxzzqsb2bjh3r3f7s138bl0q";
  };
in {
  environment.etc = {
    nixpkgs = {
      source = nixpkgs;
    };
  };
  nix.nixPath = ["nixpkgs=/etc/nixpkgs"];
}

Then, whenever I need to modify nixpkgs version, I do it in two steps:

1. "Install nixpkgs": Update nixpkgs and run nixos-rebuild switch once. This will "install" the new nixpkgs in /etc/nixpkgs. Note that the system was still built with the old nixpkgs, so don't make any changes to configuration.nix at the same time.
2. "Build the system": Update configuration.nix if you want/need to and run nixos-rebuild switch. The system is now built with the new nixpkgs.

So, instead of thinking that "I need to run nixos-rebuild switch twice", I like to think that I just first need to "install" the new nixpkgs as a separate step and then continue normally. Otherwise you'd might bump into an issue that your configuration.nix has settings that won't build with the old nixpkgs and your nixos-rebuild switch will fail. So, just don't put changes in both configuration.nix and nixpkgs at the same time, but "install" the new nixpkgs first and then build your system (with optionally changes in configuration.nix). I find this a decent workaround for me though not really a solution.