Title	Author	Discussions-To	Status	Type	Topic	Created
Pass language ecosystem overlays as pkgs.config options	jonringer	NixOS/nixpkgs#44426	Draft	Standards Track	Packaging	2024-09-17

Summary

Extending language package sets (e.g. python3Packages) is notoriously difficult as each ecosystem is created differently. This proposal attempts to provide a series of config.<language>Overlays options in which overlays can be passed to the respective package set(s).

Detailed Implementation

This is using python as an example, but applicable to almost all package sets.

Config support:

  # pkgs/top-level/config.nix

  overlayType = lib.mkOptionType {
    name = "overlay";
    description = "overlay";
    check = lib.isFunction;
    merge = lib.mergeOneOption;
  };

  ...

  options.pythonOverlays = mkOption {
    type = types.listOf overlayType;
    default = [];
    description = ''
      Overlays which will be applied to every python interpreter package set.
    '';
  };

Integration with python package sets, in this case it would replace the need for the pythonPackagesExtensions you can define at the top-level:

--- a/pkgs/development/interpreters/python/passthrufun.nix
+++ b/pkgs/development/interpreters/python/passthrufun.nix
@@ -56,7 +56,7 @@
         pythonExtension
       ] ++ (optionalExtensions (!self.isPy3k) [
         python2Extension
-      ]) ++ pythonPackagesExtensions ++ [
+      ]) ++ config.pythonOverlays ++ [
         overrides
       ]);

Example python use case

In a downstream overlay:

# flake.nix or default.nix
import poly-repo {
  ...
  config.pythonOverlays = [ (final: _: {
    myPythonPackage = final.callPackage ./package.nix { };
  )}];
}

Then building for any python interpreter would just be:

$ nix-build -A python311.pkgs.myPythonPackage
/nix/store/<hash>-python3.11-mypythonpackage-1.0
$ nix-build -A python312.pkgs.myPythonPackage
/nix/store/<hash>-python3.12-mypythonpackage-1.0

Migration plan

• If a package set does expose a way to inject overlays (e.g. python + pythonPackagesExtensions), then deprecate old usage.

Future work

• Apply overlay options for every language ecosystem and package set.

Relevant topics

• Recursive Overlays: NixOS/nixpkgs#54266
• Common Override Interface: NixOS/rfcs#67
• "Better extensible" attrsets: NixOS/nixpkgs#51213