xrootd: provide fuse support (xrootdfs) for Darwin and fix the tests #206969

ShamrockLee · 2022-12-20T06:41:35Z

Description of changes

Pass fuse into buildInputs by defaults and use fuse = osxfuse on Darwin.

This should bring no change to the Linux package. However, the Linux package also rebuilds due to the change of the buildInputs order.

Things done

ShamrockLee · 2022-12-20T15:05:20Z

@veprbl Could you help take a look?

wegank · 2023-02-02T03:23:23Z

@ofborg build xrootd xrootd.passthru.tests

gador · 2023-02-03T07:18:07Z

This seems to break nixpkgs-review (from an unrelated PR):

$ nix-env --option system x86_64-linux -f /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs -qaP --xml --out-path --show-trace --no-allow-import-from-derivation
error:
       … while querying the derivation named 'hepmc3-3.2.5'

       … while evaluating the attribute 'out.outPath'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/customisation.nix:215:13:

          214|             drvPath = assert condition; drv.${outputName}.drvPath;
          215|             outPath = assert condition; drv.${outputName}.outPath;
             |             ^
          216|           };

       … while evaluating the attribute 'buildInputs' of the derivation 'hepmc3-3.2.5'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating the attribute 'buildInputs' of the derivation 'root-6.26.10'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating the attribute 'buildInputs' of the derivation 'xrootd-5.5.1'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating call site

       at «none»:0: (source not available)

       … while calling 'getOutput'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/attrsets.nix:832:23:

          831|   */
          832|   getOutput = output: pkg:
             |                       ^
          833|     if ! pkg ? outputSpecified || ! pkg.outputSpecified

       … while evaluating call site

       at «none»:0: (source not available)

       … while calling anonymous lambda

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:233:13:

          232|       (map (drv: drv.__spliced.hostHost or drv) (checkDependencyList "depsHostHost" depsHostHost))
          233|       (map (drv: drv.__spliced.hostTarget or drv) (checkDependencyList "buildInputs" buildInputs'))
             |             ^
          234|     ]

       … while evaluating call site

       at «none»:0: (source not available)

       … while calling anonymous lambda

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/lists.nix:117:29:

          116|   */
          117|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                             ^
          118|

       … while evaluating call site

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/lists.nix:117:32:

          116|   */
          117|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                                ^
          118|

       … while calling anonymous lambda

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:200:81:

          199|   checkDependencyList = checkDependencyList' [];
          200|   checkDependencyList' = positions: name: deps: lib.flip lib.imap1 deps (index: dep:
             |                                                                                 ^
          201|     if lib.isDerivation dep || isNull dep || builtins.typeOf dep == "string" || builtins.typeOf dep == "path" then dep

       … while evaluating call site

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:201:8:

          200|   checkDependencyList' = positions: name: deps: lib.flip lib.imap1 deps (index: dep:
          201|     if lib.isDerivation dep || isNull dep || builtins.typeOf dep == "string" || builtins.typeOf dep == "path" then dep
             |        ^
          202|     else if lib.isList dep then checkDependencyList' ([index] ++ positions) name dep

       … while calling 'isDerivation'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/attrsets.nix:576:5:

          575|     # Value to check.
          576|     value: value.type or null == "derivation";
             |     ^
          577|

       error: undefined variable 'hostPlatform'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/top-level/all-packages.nix:1606:10:

         1605|     fuse =
         1606|       if hostPlatform.isDarwin then osxfuse
             |          ^
         1607|       else if hostPlatform.isLinux then fuse

and here:

nixpkgs-review pr 206969
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/206969/head:refs/nixpkgs-review/1
Von https://github.com/NixOS/nixpkgs
 + 885bf6b3a46...4f424907226 refs/pull/206969/head -> refs/nixpkgs-review/1  (Aktualisierung erzwungen)
$ git worktree add /home/wogan/.cache/nixpkgs-review/pr-206969/nixpkgs 89fae351e4a69b429182e6ffdd348202dd28522e
Bereite Arbeitsverzeichnis vor (losgelöster HEAD 89fae351e4a)
Aktualisiere Dateien: 100% (33563/33563), fertig.
HEAD ist jetzt bei 89fae351e4a Merge pull request #210875 from oluceps/update-maple
$ git merge --no-commit --no-ff 4f424907226c783b9203c5cddcd0a5276980dc79
Bereits aktuell.
error: undefined variable 'hostPlatform'

       at /home/wogan/.cache/nixpkgs-review/pr-206969/nixpkgs/pkgs/top-level/all-packages.nix:1606:10:

         1605|     fuse =
         1606|       if hostPlatform.isDarwin then osxfuse
             |          ^
         1607|       else if hostPlatform.isLinux then fuse
(use '--show-trace' to show detailed location information)
nix --experimental-features nix-command --system x86_64-linux eval --json --impure --no-allow-import-from-derivation --expr (import /nix/store/gn0vlbi471779yizjkvlz87xk71haq0h-nixpkgs-review-2.8.0/lib/python3.10/site-packages/nixpkgs_review/nix/evalAttrs.nix { allowAliases = false; attr-json = /tmp/tmpvjqi88bh; }) failed to run, /tmp/tmpvjqi88bh was stored inspection
https://github.com/NixOS/nixpkgs/pull/206969 failed to build

wegank · 2023-02-03T07:28:16Z

This seems to break nixpkgs-review (from an unrelated PR):

$ nix-env --option system x86_64-linux -f /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs -qaP --xml --out-path --show-trace --no-allow-import-from-derivation
error:
       … while querying the derivation named 'hepmc3-3.2.5'

       … while evaluating the attribute 'out.outPath'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/customisation.nix:215:13:

          214|             drvPath = assert condition; drv.${outputName}.drvPath;
          215|             outPath = assert condition; drv.${outputName}.outPath;
             |             ^
          216|           };

       … while evaluating the attribute 'buildInputs' of the derivation 'hepmc3-3.2.5'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating the attribute 'buildInputs' of the derivation 'root-6.26.10'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating the attribute 'buildInputs' of the derivation 'xrootd-5.5.1'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:286:7:

          285|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          286|       name =
             |       ^
          287|         let

       … while evaluating call site

       at «none»:0: (source not available)

       … while calling 'getOutput'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/attrsets.nix:832:23:

          831|   */
          832|   getOutput = output: pkg:
             |                       ^
          833|     if ! pkg ? outputSpecified || ! pkg.outputSpecified

       … while evaluating call site

       at «none»:0: (source not available)

       … while calling anonymous lambda

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:233:13:

          232|       (map (drv: drv.__spliced.hostHost or drv) (checkDependencyList "depsHostHost" depsHostHost))
          233|       (map (drv: drv.__spliced.hostTarget or drv) (checkDependencyList "buildInputs" buildInputs'))
             |             ^
          234|     ]

       … while evaluating call site

       at «none»:0: (source not available)

       … while calling anonymous lambda

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/lists.nix:117:29:

          116|   */
          117|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                             ^
          118|

       … while evaluating call site

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/lists.nix:117:32:

          116|   */
          117|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                                ^
          118|

       … while calling anonymous lambda

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:200:81:

          199|   checkDependencyList = checkDependencyList' [];
          200|   checkDependencyList' = positions: name: deps: lib.flip lib.imap1 deps (index: dep:
             |                                                                                 ^
          201|     if lib.isDerivation dep || isNull dep || builtins.typeOf dep == "string" || builtins.typeOf dep == "path" then dep

       … while evaluating call site

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:201:8:

          200|   checkDependencyList' = positions: name: deps: lib.flip lib.imap1 deps (index: dep:
          201|     if lib.isDerivation dep || isNull dep || builtins.typeOf dep == "string" || builtins.typeOf dep == "path" then dep
             |        ^
          202|     else if lib.isList dep then checkDependencyList' ([index] ++ positions) name dep

       … while calling 'isDerivation'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/lib/attrsets.nix:576:5:

          575|     # Value to check.
          576|     value: value.type or null == "derivation";
             |     ^
          577|

       error: undefined variable 'hostPlatform'

       at /home/wogan/.cache/nixpkgs-review/pr-214311/nixpkgs/pkgs/top-level/all-packages.nix:1606:10:

         1605|     fuse =
         1606|       if hostPlatform.isDarwin then osxfuse
             |          ^
         1607|       else if hostPlatform.isLinux then fuse

and here:

nixpkgs-review pr 206969
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/206969/head:refs/nixpkgs-review/1
Von https://github.com/NixOS/nixpkgs
 + 885bf6b3a46...4f424907226 refs/pull/206969/head -> refs/nixpkgs-review/1  (Aktualisierung erzwungen)
$ git worktree add /home/wogan/.cache/nixpkgs-review/pr-206969/nixpkgs 89fae351e4a69b429182e6ffdd348202dd28522e
Bereite Arbeitsverzeichnis vor (losgelöster HEAD 89fae351e4a)
Aktualisiere Dateien: 100% (33563/33563), fertig.
HEAD ist jetzt bei 89fae351e4a Merge pull request #210875 from oluceps/update-maple
$ git merge --no-commit --no-ff 4f424907226c783b9203c5cddcd0a5276980dc79
Bereits aktuell.
error: undefined variable 'hostPlatform'

       at /home/wogan/.cache/nixpkgs-review/pr-206969/nixpkgs/pkgs/top-level/all-packages.nix:1606:10:

         1605|     fuse =
         1606|       if hostPlatform.isDarwin then osxfuse
             |          ^
         1607|       else if hostPlatform.isLinux then fuse
(use '--show-trace' to show detailed location information)
nix --experimental-features nix-command --system x86_64-linux eval --json --impure --no-allow-import-from-derivation --expr (import /nix/store/gn0vlbi471779yizjkvlz87xk71haq0h-nixpkgs-review-2.8.0/lib/python3.10/site-packages/nixpkgs_review/nix/evalAttrs.nix { allowAliases = false; attr-json = /tmp/tmpvjqi88bh; }) failed to run, /tmp/tmpvjqi88bh was stored inspection
https://github.com/NixOS/nixpkgs/pull/206969 failed to build

Oh, sorry! I've reverted this, and I'll redo the PR afterwards.

ShamrockLee · 2023-02-03T08:44:48Z

@gador Sorry for the trouble.

I guess it would be better to test it on a real Darwing setup (either bare-metal or virtual) instead of relying on OfBorg.

ShamrockLee · 2023-02-03T15:02:50Z

@wegank Just curious, how does revert and redo solves the problem? The code seems the same.

wegank · 2023-02-03T19:45:30Z

@wegank Just curious, how does revert and redo solves the problem? The code seems the same.

I think hostPlatform should be replaced by stdenv.hostPlatform.

My PR was just a minimal one, with zero rebuilds on Linux. Feel free to add the tests again.

ShamrockLee · 2023-02-10T00:02:03Z

My PR was just a minimal one, with zero rebuilds on Linux. Feel free to add the tests again.

That's because the position of fuse in the buildInputs aren't changed.

I try to place them in alphabetical order, which changes its position when moving it up and causes the rebuild.

xrootd: provide fuse support (xrootdfs) for Darwin

a5b4e5b

ofborg bot added 6.topic: darwin Running or building packages on Darwin 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Dec 20, 2022

ShamrockLee force-pushed the xrootd-fuse branch 2 times, most recently from 7fd221b to 0388532 Compare December 20, 2022 09:39

ShamrockLee added 2 commits December 20, 2022 18:45

xrootd.passthru.tests: only add test-runner for Linux

16516fa

xrootd.passthru.test-xrdcp: init

4f42490

ShamrockLee force-pushed the xrootd-fuse branch from 0388532 to 4f42490 Compare December 20, 2022 10:48

ShamrockLee changed the title ~~xrootd: provide fuse support (xrootdfs) for Darwin~~ xrootd: provide fuse support (xrootdfs) for Darwin and fix the tests Dec 20, 2022

veprbl approved these changes Dec 22, 2022

View reviewed changes

SuperSandro2000 added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Jan 23, 2023

wegank merged commit d34529b into NixOS:master Feb 2, 2023

gador mentioned this pull request Feb 3, 2023

pgadmin: pin flask-babel to fix build failure #214311

Merged

13 tasks

wegank mentioned this pull request Feb 3, 2023

Revert "xrootd: provide fuse support (xrootdfs) for Darwin and fix the tests" #214315

Merged

wegank mentioned this pull request Feb 3, 2023

xrootd: provide fuse support on darwin #214316

Merged

13 tasks

ShamrockLee deleted the xrootd-fuse branch February 3, 2023 08:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xrootd: provide fuse support (xrootdfs) for Darwin and fix the tests #206969

xrootd: provide fuse support (xrootdfs) for Darwin and fix the tests #206969

ShamrockLee commented Dec 20, 2022 •

edited

Loading

ShamrockLee commented Dec 20, 2022

wegank commented Feb 2, 2023 •

edited

Loading

gador commented Feb 3, 2023 •

edited

Loading

wegank commented Feb 3, 2023

ShamrockLee commented Feb 3, 2023

ShamrockLee commented Feb 3, 2023 •

edited

Loading

wegank commented Feb 3, 2023

ShamrockLee commented Feb 10, 2023

xrootd: provide fuse support (xrootdfs) for Darwin and fix the tests #206969

xrootd: provide fuse support (xrootdfs) for Darwin and fix the tests #206969

Conversation

ShamrockLee commented Dec 20, 2022 • edited Loading

Description of changes

Things done

ShamrockLee commented Dec 20, 2022

wegank commented Feb 2, 2023 • edited Loading

gador commented Feb 3, 2023 • edited Loading

wegank commented Feb 3, 2023

ShamrockLee commented Feb 3, 2023

ShamrockLee commented Feb 3, 2023 • edited Loading

wegank commented Feb 3, 2023

ShamrockLee commented Feb 10, 2023

ShamrockLee commented Dec 20, 2022 •

edited

Loading

wegank commented Feb 2, 2023 •

edited

Loading

gador commented Feb 3, 2023 •

edited

Loading

ShamrockLee commented Feb 3, 2023 •

edited

Loading