test: add test for grpcio wheel usage

[cpcloud]

Adds a test to demonstrate failure to use the grpcio 1.51.1 osx 12.0 wheel available on PyPI wheel at all.

Fixes #842.

[cpcloud]

@adisbladis I could really use your help here, it seems like preferWheel has no effect anymore. #842 is likely the same issue demonstrated in this PR.

Curiously, throwing maturin in as a dependency the preferWheel attribute is evaluated.

Any idea why this would be the case?

[cpcloud]

It seems like reordering the overlays in overrides.withDefaults put the user overlay before the default poetry overrides overlay achieves the desired behavior. Admittedly, this is not a solution that I arrived at from thorough analysis.

My rationale was that it seemed like eval isn't happening at all on many packages, so I tried "forcing" eval by putting user overrides first in the list of overlays.

[cpcloud]

@asymmetric Can you try this PR out on your panel use case? I've incorporated a test for it here, and I think I've addressed the issue of preferWheel not ever being evaluated.

[asymmetric]

@cpcloud thanks for looking into this!

We actually don't use withDefaults, we do:

      mkPoetryApplication = {
        projectDir = ./.;
        python = pythonVersion;
        overrides = [
          pkgs.poetry2nix.defaultPoetryOverrides
          (final: prev: {
            panel = prev.panel.override { preferWheel = true; };
            bokeh = prev.bokeh.override { preferWheel = true; };
          })
          # more custom overlays
        ];
      };

I moved the inline overlay above pkgs.poetry2nix.defaultPoetryOverrides, and it indeed solves the issue!

One thing I don't understand though is why the -A preferWheel test passes even without this PR.

[cpcloud]

@adisbladis Any chance you can take a look at this PR?

[asymmetric]

One thing I don't understand though is why the -A preferWheel test passes even without this PR.

@cpcloud do you have any ideas about this?

[asymmetric]

One more thing: what does this test catch that the panel/bokeh one doesn't?

Could we just limit ourselves to one of the two? If not, then maybe adding an explanation of what exactly is being tested could be helpful.

[cpcloud]

Sure, grpcio fails because of a missing wheel on the latest GitHub macos runners.

[yorickvP]

Applying defaults after user overlays feels wrong to me, and it does in fact break my code. It prevents users from undoing changes made in the default overlays.
If preferWheel doesn't compose right that's a deeper issue that we should fix.

Please revert this.

Code that breaks:

poetry2nix.overrides.withDefaults (super: self: {
  pytest = null;
})

[cpcloud]

@adisbladis @yorickvP's request seems reasonable. Shall we revert?

[adisbladis]

I'm OK with a revert.

[asymmetric]

Any ideas about what the underlying cause of the lack of composability of preferWheel?

[yorickvP]

Yes, .override is from the callPackage call in mk-poetry-dep.nix, while overridePythonAttrs comes from its call to buildPythonPackage, which doesn't know about the outer callPackage.

Options:

• override overridePythonAttrs to know about the outer callPackage.
• change buildPythonPackage to take a function, just like mkDerivation these days, and specify preferWheel through overridePythonAttrs
• remove preferWheel somehow
• ???

[iwanb]

Applying defaults after user overlays feels wrong to me, and it does in fact break my code. It prevents users from undoing changes made in the default overlays. If preferWheel doesn't compose right that's a deeper issue that we should fix.

Please revert this.

Code that breaks:

poetry2nix.overrides.withDefaults (super: self: {
  pytest = null;
})

I encountered the same issue. Will the change be reverted? As a workaround I did:

overrides = [ poetry2nix.defaultPoetryOverrides  ...];

[sigprof]

This also breaks in cases like #1519, or when trying to apply a local override for rpds-py while #1502 is still pending. The workaround in #899 (comment) helps, but why then have overrides.withDefaults if it can't be used sanely?