Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hedgedoc does not build on master #176127

Closed
yu-re-ka opened this issue Jun 3, 2022 · 9 comments · Fixed by #182431
Closed

hedgedoc does not build on master #176127

yu-re-ka opened this issue Jun 3, 2022 · 9 comments · Fixed by #182431
Labels
0.kind: bug Something is broken

Comments

@yu-re-ka
Copy link
Contributor

yu-re-ka commented Jun 3, 2022

as mentioned in #175972

(in node_modules/sqlite3)

npm run install --build-from-source --nodedir=/nix/store/nbayk94b9w60mgdq9640241jr24hrdw4-nodejs-16.15.1/include/node

exits with code 243

responsible commit is e6188b6

@yu-re-ka yu-re-ka added the 0.kind: bug Something is broken label Jun 3, 2022
@fatho
Copy link
Contributor

fatho commented Jun 4, 2022

I encountered the same symptom while building vscode-extensions.vadimcn.vscode-lldb. It too failed with exit code 243 on an npm run webpack ... line.

Thanks to you tracking it down to the nodejs update, I found that the biggest change was an upgrade of the included npm version from 8.5-something to 8.10 or so.

And for npm 8.6, there seems to be this matching issue: npm/cli#4769

Apparently, it now fails when the home-directory does not exist. And inside sandboxed nix builds, there is no home directory.

I got the vscode-lldb build to work with a small hack, but that's obviously not a general solution:

      codelldb = pkgs.vscode-extensions.vadimcn.vscode-lldb.overrideAttrs (old: {
        preConfigure = ''
          mkdir /tmp/nodejs-home
          export HOME=/tmp/nodejs-home
        '';
      });

@phaer
Copy link
Member

phaer commented Jun 5, 2022

prepareAndInvokeNPM which is used by buildNodeDependencies just sets export HOME=$TMPDIR which seems to be /build in practice.

Adding export HOME=$TMPDIR to the beginning of buildPhase fixes both hedgedoc as well as jellyfin-web (mentioned in related issue above) for me.

As this issue seems to concern pretty much all node packages which call npm manually during their buildPhase, it might make sense to set HOME for all of them. Not sure yet whats the best way to do so.

@marius851000
Copy link
Contributor

This issue also affect the peertube service. It set it's home directory to the peertube derivation, and so, it doesn't start. @mohe2015 (or another peertube maintainer), you may be interested in this issue (not sure it's a good idea to ping the whole peertube maintainers maintainer team...)

happysalada pushed a commit that referenced this issue Jun 5, 2022
npm now fails if $HOME is not set, see #176127
@BBBSnowball
Copy link
Contributor

This works for me as a temporary fix:

  nixpkgs.overlays = [
    (self: super: {
       hedgedoc = super.hedgedoc.overrideAttrs (old: {
         # see https://github.com/NixOS/nixpkgs/issues/176127#issuecomment-1146782555
         preBuild = ''
           export HOME=$TMPDIR
         '';
       });
    })
  ];

Should we commit a fix for hedgedoc while we are thinking about more general solutions?

The responsible commit is a patch update of nodejs but this seems to do six minor updates for npm (see this and this).

I think it might be interesting to know why the newer npm insist on having a home. strace shows several statx and one openat on paths in $HOME but none of this is fatal. Then, it tries mkdir and bails out. The stacktrace has libuv and libc so I assume that the call is from Javascript and I don't know an easy way to find out more from the strace.

If it can access its home, it will create a .npm directory with _update-notifier-last-checked, _cacache and _logs. This first one should be fine. The location of the others can be configured by cache and logs-dir. The important one seems to be cache (the error code is not 243 but I think this is because the mkdir will see EROFS rather than ENOENT here):

$ export HOME=/nix/store/homeless
$ npm help ; echo $?
254
$ npm help --logs-dir=y ; echo $?
254
$ npm help --cache=x ; echo $?
npm <command>
...
$ find x
x
x/_logs
x/_logs/2022-06-06T21_08_26_555Z-debug-0.log
$ npm --cache=x2 run install --build-from-source --nodedir=/nix/store/nbayk94b9w60mgdq9640241jr24hrdw4-nodejs-16.15.1/include/node
... (lots of output, does not abort immediately)

As this also breaks the npm help command, we should maybe report this upstream rather than work around it? I think this might be on purpose but I would expect a useful error message in that case. What do you think?

Regarding a fix in nixpkgs: We can set $npm_config_cache instead of $HOME (e.g. export npm_config_cache=$TMPDIR/.npm). This should do the same thing as passing --cache but we can apply it in a hook script. As this should only be used by npm (in contrast to $HOME), I think we could apply it to all builds that are using npm. Here is the documentation on using env vars for config and it seems to work as expected:

$ npm_config_cache=x3 npm help ; echo $?
npm <command>
...
0

@mohe2015
Copy link
Contributor

mohe2015 commented Jun 6, 2022

This issue also affect the peertube service. It set it's home directory to the peertube derivation, and so, it doesn't start. @mohe2015 (or another peertube maintainer), you may be interested in this issue (not sure it's a good idea to ping the whole peertube maintainers maintainer team...)

Currently not using NixOS so pinging @Izorkin

@pinpox
Copy link
Member

pinpox commented Jun 10, 2022

This works for me as a temporary fix:

Tried this fix on latest unstable, but it still fails to build. This is what I got from nixos-rebuild

building '/nix/store/p2q1h6bs2gapsqcfbjqrrby75kz5j3jk-unit-systemd-udev-settle.service.drv'...
building '/nix/store/8iw002nmpqv682pv135ik6pdcirsifqh-unit-systemd-udevd.service.drv'...
building '/nix/store/3nkj3m79vlhsll64dq870lsgfwnkkhxs-unit-systemd-update-utmp.service.drv'...
93% after chunk asset optimization SourceMapDevToolPlugin index.cc145078bb69406a9fc3.js attach SourceMap/nix/store/ypszvxqskanxpxplcda0pfa6kmj2yjy0-stdenv-linux/setup: line 1387:   354 Killed                  npm run builds/ui/toolbar.cssmm2m2K2KmKs=emojify!emojify.js flowchart.js js-sequence-diagrams expose-loader?exposes=Reveal!reveal.js expose-loader?exposes=RevealMarkdown!reveal-markdown /build/source/deps/HedgeDoc/public/js/slide.jssimplescrollbars.css /build/source/deps/HedgeDoc/node_modules/codemirror/addon/search/matchesonscrollbar.css /build/source/deps/HedgeDoc/node_modules/codemirror/theme/monokai.css /build/source/deps/HedgeDoc/node_modules/codemirror/theme/one-dark.css /build/source/deps/HedgeDoc/node_modules/codemirror/mode/tiddlywiki/tiddlywiki.css /build/source/deps/HedgeDoc/node_modules/codemirror/mode/mediawiki/mediawiki.css /build/source/deps/HedgeDoc/node_modules/spin.js/spin.css /build/source/deps/HedgeDoc/public/css/github-extract.css /build/source/deps/HedgeDoc/public/vendor/showup/showup.css /build/source/deps/HedgeDoc/public/css/mermaid.css /build/source/deps/HedgeDoc/public/css/markdown.css /build/source/deps/HedgeDoc/public/css/slide-preview.css
building '/nix/store/yjpvcx8lrqzdr9mcpgd206b3kiigr82k-unit-systemd-user-sessions.service.drv'...
error: builder for '/nix/store/fas34457n3m77vk46c334plqf8ipmbs3-HedgeDoc-1.9.0.drv' failed with exit code 137
error: 1 dependencies of derivation '/nix/store/ipn37hrbvis2qainsl9ckcywc4jvbzgs-unit-hedgedoc.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xhd877x733msg5fjhvmjvvbh2c8zhy1l-system-units.drv' failed to build
building '/nix/store/rj4zda80washrnnqfzpv6adzka1nbymv-user-units.drv'...
building '/nix/store/3fxxydg5fq1xjp7qnihbq9q730nr6xrd-useradd.drv'...
error: 1 dependencies of derivation '/nix/store/qcix9mr1yji5nms6p222bphd6w0171hn-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/klz8jgki9pms30m52ixrxgvvckv58y3i-nixos-system-kfbox-22.11.20220608.e0169d7.drv' failed to build

@NickHu
Copy link
Contributor

NickHu commented Jun 10, 2022

@pinpox To me, it looks like your machine ran out of memory while building so it got killed

gador pushed a commit to gador/nixpkgs that referenced this issue Jun 10, 2022
Node.js 16.15.1 updated its vendored npm, which breaks sandboxed builds

See: NixOS#176127

This reverts commit e6188b6.
@Izorkin
Copy link
Contributor

Izorkin commented Jun 15, 2022

Currently not using NixOS so pinging @Izorkin

Fix work PeerTube with nodejs 16.15.1

diff --git a/nixos/modules/services/web-apps/peertube.nix b/nixos/modules/services/web-apps/peertube.nix
index e6b6aa273e7..40838e5807a 100644
--- a/nixos/modules/services/web-apps/peertube.nix
+++ b/nixos/modules/services/web-apps/peertube.nix
@@ -409,6 +409,7 @@ in {
           password: '$(cat ${cfg.smtp.passwordFile})'
         ''}
         EOF
+        mkdir /tmp/.npm
         ln -sf ${cfg.package}/config/default.yaml /var/lib/peertube/config/default.yaml
         ln -sf ${configFile} /var/lib/peertube/config/production.json
         npm start
@@ -433,7 +434,7 @@ in {
         RestrictAddressFamilies = [ "AF_UNIX" "AF_INET" "AF_INET6" "AF_NETLINK" ];
         MemoryDenyWriteExecute = false;
         # System Call Filtering
-        SystemCallFilter = [ ("~" + lib.concatStringsSep " " systemCallsList) "pipe" "pipe2" ];
+        SystemCallFilter = [ ("~" + lib.concatStringsSep " " systemCallsList) "@chown" "pipe" "pipe2" ];
       } // cfgService;
     };
diff --git a/pkgs/servers/peertube/default.nix b/pkgs/servers/peertube/default.nix
index 558c21c6cd1..8c107047af2 100644
--- a/pkgs/servers/peertube/default.nix
+++ b/pkgs/servers/peertube/default.nix
@@ -97,6 +97,7 @@ in stdenv.mkDerivation rec {
     mkdir $out/client
     mv ~/client/{dist,node_modules,package.json,yarn.lock} $out/client
     mv ~/{config,scripts,support,CREDITS.md,FAQ.md,LICENSE,README.md,package.json,tsconfig.json,yarn.lock} $out
+    ln -s /tmp/.npm $out/.npm
   '';

   passthru.tests.peertube = nixosTests.peertube;

@Izorkin
Copy link
Contributor

Izorkin commented Jul 11, 2022

After that PR npm started to give an error of creating temporary files - npm/cli#4594
Issue - npm/cli#4996

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants