mkYarnPackage: EACCES: permission denied, unlink '/nix/store/...

[Silver-Golden]

Describe the bug

Using mkYarnPackage and it fails.
After searching for the error I came across this: nix-community/yarn2nix#53 , they seemed to have fixed it, but I dont know enough nix to understand how they fixed it.

On nixpkgs these two also seem to have been effected, but before nix-community/yarn2nix was merged in (#108138)

• heroku: 3.43.16 -> 7.5.7 #42869
• Package request: Oni (react based IDE) #38231

Steps To Reproduce

You can find my source code and flake.nix at https://gitlab.com/Silvers_Gw2/Stats_Frontend/-/tree/cc5d783abd54e95363410592c390ca6925462262

• Clone repo above at that commit
• nix build

Expected behavior

A normal build with no issues.
In shell it (manually) installs with no issues

Screenshots

Output of my nix build

[nix-shell:~/DataWars2_Frontend]# nix build
error: builder for '/nix/store/f926i9kkpmd9givl1hzpis6x6cr62d6d-gw2_stats-modules-2.23.0.drv' failed with exit code 1;
       last 10 log lines:
       > verbose 0.216445485 Checking for configuration file "/nix/store/i9x1ajwr756r2z9jnb2nhip50a3xms51-nodejs-16.15.0/etc/yarnrc".
       > verbose 0.216755828 Checking for configuration file "/build/.yarnrc".
       > verbose 0.22065349 current time: 2022-05-23T12:51:27.411Z
       > [1/4] Resolving packages...
       > [2/4] Fetching packages...
       > [] 0/1535verbose 1.383621076 Error: EACCES: permission denied, unlink '/nix/store/y57xf6nfhhdp80482izrzzsvnydz5yzy-offline/gw2e_tradingpost_fees___gw2e_tradingpost_fees_1.1.1.tgz'
       > error An unexpected error occurred: "EACCES: permission denied, unlink '/nix/store/y57xf6nfhhdp80482izrzzsvnydz5yzy-offline/gw2e_tradingpost_fees___gw2e_tradingpost_fees_1.1.1.tgz'".
       > info If you think this is a bug, please open a bug report with the information provided in "/build/yarn-error.log".
       > info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
       >
       For full logs, run 'nix log /nix/store/f926i9kkpmd9givl1hzpis6x6cr62d6d-gw2_stats-modules-2.23.0.drv'.
       note: build failure may have been caused by lack of free disk space
error: 1 dependencies of derivation '/nix/store/4mn6b0b3y32br37sxagj5b9fwradn0hp-datawars2-frontend.drv' failed to build
error: 1 dependencies of derivation '/nix/store/6smdwm3b9270jd1i5x4zvhk89f66z315-datawars2-frontend.drv' failed to build

[nix-shell:~/DataWars2_Frontend]# nix log /nix/store/f926i9kkpmd9givl1hzpis6x6cr62d6d-gw2_stats-modules-2.23.0.drv
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
@nix { "action": "setPhase", "phase": "buildPhase" }
building
yarn config v1.22.18
success Set "yarn-offline-mirror" to "/nix/store/y57xf6nfhhdp80482izrzzsvnydz5yzy-offline".
Done in 0.04s.
yarn install v1.22.18
verbose 0.213228892 Checking for configuration file "/build/.npmrc".
verbose 0.21361644 Checking for configuration file "/build/yarn_home/.npmrc".
verbose 0.213916534 Checking for configuration file "/nix/store/i9x1ajwr756r2z9jnb2nhip50a3xms51-nodejs-16.15.0/etc/npmrc".
verbose 0.21415968 Checking for configuration file "/build/.npmrc".
verbose 0.215209132 Checking for configuration file "/build/.yarnrc".
verbose 0.215645342 Checking for configuration file "/build/yarn_home/.yarnrc".
verbose 0.216052106 Found configuration file "/build/yarn_home/.yarnrc".
verbose 0.216445485 Checking for configuration file "/nix/store/i9x1ajwr756r2z9jnb2nhip50a3xms51-nodejs-16.15.0/etc/yarnrc".
verbose 0.216755828 Checking for configuration file "/build/.yarnrc".
verbose 0.22065349 current time: 2022-05-23T12:51:27.411Z
[1/4] Resolving packages...
[2/4] Fetching packages...
[] 0/1535verbose 1.383621076 Error: EACCES: permission denied, unlink '/nix/store/y57xf6nfhhdp80482izrzzsvnydz5yzy-offline/gw2e_tradingpost_fees___gw2e_tradingpost_fees_1.1.1.tgz'
error An unexpected error occurred: "EACCES: permission denied, unlink '/nix/store/y57xf6nfhhdp80482izrzzsvnydz5yzy-offline/gw2e_tradingpost_fees___gw2e_tradingpost_fees_1.1.1.tgz'".
info If you think this is a bug, please open a bug report with the information provided in "/build/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Additional context

Add any other context about the problem here.

Notify maintainers

I dont see any meta.maintainers, @roberth you are teh last person to commit to the default.nix so would you know who to pull in?

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[nix-shell:~/DataWars2_Frontend]# nix-shell -p nix-info --run "nix-info -m"
this path will be fetched (0.06 MiB download, 0.30 MiB unpacked):
  /nix/store/hk2mydhxsxbw7xalwa4s0dyk0c6gmgkf-bash-interactive-5.1-p16-dev
copying path '/nix/store/hk2mydhxsxbw7xalwa4s0dyk0c6gmgkf-bash-interactive-5.1-p16-dev' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.91, NixOS, 22.05 (Quokka), 22.05.20220514.43ff6cb`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.0`
 - channels(root): `"nixos-22.05pre377261.d89d7af1ba2"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[roberth]

My changes related to the use of functions and variables that aren't specific to yarn2nix at all. I don't maintainer yarn2nix and I'm not familiar with its implementation.

[Silver-Golden]

@roberth ye I know, apologies for teh ping and pulling you into this. the form to create teh issue said:

Please @ people who are in the meta.maintainers list of the offending package or module.
If in doubt, check git blame for whoever last touched something.

Since as far as I can tell there are no mantainers listed, well git blame (and you are a member of Nix/Nixpkgs/NixOS).

Would you have any idea who to ping?

[roberth]

Maybe someone who's been involved with a real fix or feature and hasn't just merged it. Perhaps @tomberek or @jtojnar?

[jtojnar]

Can you paste the contents of yarn-error.log (run the nix-build command with --keep-failed flag).

[jtojnar]

I tried running nix build -L gitlab:Silvers_Gw2/Stats_Frontend/cc5d783abd54e95363410592c390ca6925462262 --keep-failed but it offers no new information. Replied back upstream:

yarnpkg/yarn#6058 (comment)

[Silver-Golden]

Apologies for the delay, took a little bit to figure out where teh outputs were kept.

yarn-error.log

It hit the max chars so I have attached it instead: yarn-error.log

[jtojnar]

Yes, that offers no hints, see the upstream issue.

[Silver-Golden]

Ah, didn't notice ye had replied again, hadn't refreshed the page.

I hope more info comes to light that fixes this properly.
For teh moment though I am sidestepping it by building the react site in a docker container (not ideal but it works)

Thank you for your time (both of you)

[aakropotkin]

This happens because Yarn is designed with the assumption that it can freely modify installed modules, which isn't going to work on store paths. Additionally Yarn tends to hard code store paths into its cache, which isn't traceable by Nix, so it can be difficult to find or predict which paths you would need to "copy" into a RW area. Even if you could, changing locator tags to point to the new location will cause yarn to rebuild every tarball in the cache.

Honestly it's a mess. I've spent about 3 months trying to get a 6000+ package Yarn tool chain nixified and I've finally accepted that Yarn and Nix are incompatible "in the general case".

You can definitely build a lot of stuff without issues, but Yarn's requirements for caching are fundamentally at odds with Nix's. This means you really have to build a full yarn cache from scratch in your derivation, and you can't recycle any parts of it in another - which really really sucks. The spots where you get in trouble are attempts to run yarn build on anything that isn't explicitly in your build area - this means any package with an install script that you try to pass in using a store path, so fetchGit and fetchTree style fetchers are out of the question unless you explicitly copy them.

NPM is significantly easier to work with and I strongly recommend it.

[kolpav]

I've spent about 3 months trying to get a 6000+ package Yarn tool chain nixified

Suddenly my project not building doesn't seem so bad..