Add to CI builds for mac, linux, windows

[guylouis]

Could the infra team setup CI for this project so that it is built for mac, linux, windows?
The command to build is npm dist.
Releases builds are enough, no need for PR builds

cc @bitgamma

[jakubgs]

I'm a bit confused. Why do you want this to be build for 3 different platforms if this is essentially a Node.js package?

[bitgamma]

This is an Electron app (like VSCode) so the build actually generates native binaries

[jakubgs]

Well, first of all I'm getting installation failures on linux when running npm install:

npm ERR! npm ERR! In file included from ../src/addon.cpp:1:
npm ERR! npm ERR! ../src/pcsclite.h:9:10: fatal error: winscard.h: No such file or directory
npm ERR! npm ERR!     9 | #include <winscard.h>
npm ERR! npm ERR!       |          ^~~~~~~~~~~~
npm ERR! npm ERR! compilation terminated.

Apparently you can get that on Ubuntu from libpcsclite-dev:

• #winscard.h missing  Proxmark/proxmark3#961

But I'd rather use a Nix shell.

[bitgamma]

yeah libpcsclite-dev is needed on Linux, same as Status Desktop. How was it solved there?

[jakubgs]

I tried using pcsclite from nixpkgs but it doesn't have the header:

 > find -L /nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcsclite.so.1.0.0
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcsclite.so
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcscspy.so
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcscspy.la
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcsclite.la
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcscspy.so.0.0.0
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcscspy.so.0
/nix/store/0fpm05grjpjrbgbpca0r20jwjsa5i60p-pcsclite-1.9.5/lib/libpcsclite.so.1

It's just the libraries.

[jakubgs]

Ah, but the dev package does have it:

 > find /nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev | grep include
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include/PCSC
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include/PCSC/reader.h
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include/PCSC/debuglog.h
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include/PCSC/wintypes.h
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include/PCSC/winscard.h
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include/PCSC/pcsclite.h
/nix/store/1frjsjkf7zhdkn77g87frxhdhlaiqg7c-pcsclite-1.9.5-dev/include/PCSC/ifdhandler.h

[jakubgs]

Managed to fix that, but now I'm getting:

npm ERR! src/global-platform-commandset.ts(2,30): error TS2307: Cannot find module './SCP02-channel' or its corresponding type declarations.
npm ERR! src/global-platform.ts(4,30): error TS2307: Cannot find module './SCP02-channel' or its corresponding type declarations.

[jakubgs]

What Node.JS version is recommended for building this? I see Electron 21 was using NodeJS 16:
https://www.electronjs.org/blog/electron-21-0

[jakubgs]

I tried using moduleResolution: node as recommended here:

• "error TS2307: Cannot find module" when using @types/ws with typescript 2.0 microsoft/TypeScript#11491

But it doesn't seem to work.

[jakubgs]

BTW, why is this keycard-sdk repository that's a direct dependency for this owned by @choppu ?
https://github.com/status-im/keycard-certify/blob/67a32a8294397b6b958864a23e1d019441466478/package.json#L32
Also the appId references this user:
https://github.com/status-im/keycard-certify/blob/67a32a8294397b6b958864a23e1d019441466478/package.json#L14
Which is probably wrong.

[bitgamma]

@choppu is the developer of the SDK and this app (this repo is a fork)

[choppu]

Hi, I will check and let you know

[jakubgs]

I can see there is src/scp02-channel.ts:
https://github.com/choppu/keycard-sdk/blob/master/src/scp02-channel.ts
But the error clearly uses an upper case filename:

error TS2307: Cannot find module './SCP02-channel'

[jakubgs]

But this file references it as upper case:

import { SCP02Channel } from "./SCP02-channel";
import { SCP02Keys } from "./scp02-keys";

https://github.com/choppu/keycard-sdk/blob/384eedab6f8de5f8d3796c1444b96bde132ba46b/src/global-platform-commandset.ts#L2-L3

Which seems like a mistake to me.

[choppu]

I found the issue, I'm fixing it now

[choppu]

Fixed. By the way, I use v18.16.1 of nodejs

[jakubgs]

Thanks for the fix @choppu, now npm install actually works.

[choppu]

great!

[jakubgs]

Now I can run npm run dist, but it fails with:

  ⨯ cannot execute  cause=fork/exec /home/jakubgs/work/keycard-certify/node_modules/7zip-bin/linux/x64/7za: no such file or directory
                    command=/home/jakubgs/work/keycard-certify/node_modules/7zip-bin/linux/x64/7za x -bd /home/jakubgs/.cache/electron-builder/appimage/158061214.7z -o/home/jakubgs/.cache/electron-builder/appimage/158061214
                    workingDir=/home/jakubgs/.cache/electron-builder/appimage

Which is obviously because I'm on NixOS, but why the hell does this intall 7zip inside of node_modules? Absolutely retarded.

[jakubgs]

How the fuck is this a thing? https://www.npmjs.com/package/7zip-bin

[jakubgs]

This appears to be coming from electron-builder:
https://github.com/electron-userland/electron-builder/blob/4517d97f48d822f446c48937df4b542a638fcab6/packages/app-builder-lib/src/targets/nsis/NsisTarget.ts#L253

And it appears to be a source of other issues:

• app-builder-lib reliance on 7zip-bin breaks builds running on arm64 windows (e.g. parallels on M1) electron-userland/electron-builder#6677

Why would you ever install binary tools via NPM...

[choppu]

latest electron-builder seems to not use 7zip on linux. I will try to update to the latest version and will let you know soon.

[jakubgs]

That would be cool, but if it's too much of a pain in the ass please tell me and I'll work around it.

[choppu]

Absolutely no problem. I have updated electron-builder to the latest stable version. I don't have linux, so can't check if this removes all dependencies from 7zip

[jakubgs]

Thank you, I will test this after I finish my meetings.

[jakubgs]

Initial build on Jenkins CI:
https://ci.infra.status.im/job/status-keycard/job/keycard-certify/job/platforms/job/linux/job/x86_64/4/console
Looks like we are failing at npm install with:

40 info run @pokusew/[email protected] install { code: 1, signal: null }
41 timing command:rebuild Completed in 4662ms
42 verbose stack Error: command failed
42 verbose stack     at ChildProcess.<anonymous> (/nix/store/hdwy8j1jvj2d68p449vvpf5j4mrgzldi-nodejs-18.16.0/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/lib/index.js:53:27)
42 verbose stack     at ChildProcess.emit (node:events:513:28)
42 verbose stack     at maybeClose (node:internal/child_process:1091:16)
42 verbose stack     at ChildProcess._handle.onexit (node:internal/child_process:302:5)
43 verbose pkgid @pokusew/[email protected]
44 verbose cwd /home/jenkins/workspace/status-keycard/keycard-certify/platforms/linux/x86_64
45 verbose Linux 5.15.0-75-generic
46 verbose node v18.16.0
47 verbose npm  v9.5.1
48 error code 1
49 error path /home/jenkins/workspace/status-keycard/keycard-certify/platforms/linux/x86_64/node_modules/@pokusew/pcsclite
50 error command failed
51 error command sh -c node-gyp rebuild
52 error make: Entering directory '/home/jenkins/workspace/status-keycard/keycard-certify/platforms/linux/x86_64/node_modules/@pokusew/pcsclite/build'
52 error   CXX(target) Release/obj.target/pcsclite/src/addon.o
52 error make: Leaving directory '/home/jenkins/workspace/status-keycard/keycard-certify/platforms/linux/x86_64/node_modules/@pokusew/pcsclite/build'

[jakubgs]

Here are some actual build errors:

53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:106:45: error: 'is_lvalue_reference_v' is not a member of 'std'; did you mean 'is_lvalue_reference'?
53 error   106 |   template <class U, std::enable_if_t<!std::is_lvalue_reference_v<U>>*>
53 error       |                                             ^~~~~~~~~~~~~~~~~~~~~
53 error       |                                             is_lvalue_reference
53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:106:66: warning: logical not is only applied to the left hand side of comparison [-Wlogical-not-parentheses]
53 error   106 |   template <class U, std::enable_if_t<!std::is_lvalue_reference_v<U>>*>
53 error       |                                                                  ^
53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:106:67: error: template argument 1 is invalid
53 error   106 |   template <class U, std::enable_if_t<!std::is_lvalue_reference_v<U>>*>
53 error       |                                                                   ^
53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:106:71: error: expected unqualified-id before '>' token
53 error   106 |   template <class U, std::enable_if_t<!std::is_lvalue_reference_v<U>>*>
53 error       |                                                                       ^
53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:123:43: error: 'is_lvalue_reference_v' is not a member of 'std'; did you mean 'is_lvalue_reference'?
53 error   123 | template <class T, std::enable_if_t<!std::is_lvalue_reference_v<T>>* = nullptr>
53 error       |                                           ^~~~~~~~~~~~~~~~~~~~~
53 error       |                                           is_lvalue_reference
53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:123:64: warning: logical not is only applied to the left hand side of comparison [-Wlogical-not-parentheses]
53 error   123 | template <class T, std::enable_if_t<!std::is_lvalue_reference_v<T>>* = nullptr>
53 error       |                                                                ^
53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:123:65: error: template argument 1 is invalid
53 error   123 | template <class T, std::enable_if_t<!std::is_lvalue_reference_v<T>>* = nullptr>
53 error       |                                                                 ^
53 error /home/jenkins/.electron-gyp/21.2.3/include/node/v8-maybe.h:123:70: error: expected unqualified-id before '=' token
53 error   123 | template <class T, std::enable_if_t<!std::is_lvalue_reference_v<T>>* = nullptr>

I'm not getting this locally.

[jakubgs]

This appears to be a known issue:

• [Bug]: Issues building native modules with electron-20 electron/electron#35244
• [Bug]: building native module fails in electron 21.1.1 electron/electron#36045
• https://stackoverflow.com/questions/64286346/error-no-template-named-is-same-v-in-namespace-std-did-you-mean-is-same

[jakubgs]

And now it fails at signing:

12:46:25    • signing         file=dist/mac/Keycard Certify.app identityName=Apple Distribution: Status Research & Development GmbH (8B5X2M6H2Y) identityHash=723EB78FC0FB68F46E4C9EA8251D36C9A1423979 provisioningProfile=none
12:46:25    ⨯ Command failed: codesign --sign 723EB78FC0FB68F46E4C9EA8251D36C9A1423979 --force --timestamp --options runtime --entitlements /Users/jenkins/workspace/status-keycard/keycard-certify/platforms/macos/x86_64/node_modules/app-builder-lib/templates/entitlements.mac.plist /Users/jenkins/workspace/status-keycard/keycard-certify/platforms/macos/x86_64/dist/mac/Keycard Certify.app/Contents/Resources/app.asar.unpacked/node_modules/@pokusew/pcsclite/build/Release/pcsclite.node
12:46:25  /Users/jenkins/workspace/status-keycard/keycard-certify/platforms/macos/x86_64/dist/mac/Keycard Certify.app/Contents/Resources/app.asar.unpacked/node_modules/@pokusew/pcsclite/build/Release/pcsclite.node: errSecInternalComponent
12:46:25    failedTask=build stackTrace=Error: Command failed: codesign --sign 723EB78FC0FB68F46E4C9EA8251D36C9A1423979 --force --timestamp --options runtime --entitlements /Users/jenkins/workspace/status-keycard/keycard-certify/platforms/macos/x86_64/node_modules/app-builder-lib/templates/entitlements.mac.plist /Users/jenkins/workspace/status-keycard/keycard-certify/platforms/macos/x86_64/dist/mac/Keycard Certify.app/Contents/Resources/app.asar.unpacked/node_modules/@pokusew/pcsclite/build/Release/pcsclite.node
12:46:25  /Users/jenkins/workspace/status-keycard/keycard-certify/platforms/macos/x86_64/dist/mac/Keycard Certify.app/Contents/Resources/app.asar.unpacked/node_modules/@pokusew/pcsclite/build/Release/pcsclite.node: errSecInternalComponent

Do we need to sign this?

[jakubgs]

Apparently you can disable signing:

To disable Code Signing when building for macOS leave all the above vars unset except for CSC_IDENTITY_AUTO_DISCOVERY which needs to be set to false. This can be done by running export CSC_IDENTITY_AUTO_DISCOVERY=false.

Another way — set mac.identity to null. You can pass aditional configuration using CLI as well: -c.mac.identity=null.

https://www.electron.build/code-signing.html#how-to-disable-code-signing-during-the-build-process-on-macos

[jakubgs]

Okay, now what is this now:

12:50:14    • packaging       platform=darwin arch=x64 electron=25.4.0 appOutDir=dist/mac
12:50:18    • skipped macOS application code signing  reason=, see https://electron.build/code-signing CSC_IDENTITY_AUTO_DISCOVERY=false
12:50:18    • building        target=macOS zip arch=x64 file=dist/Keycard Certify-1.0.0-mac.zip
12:51:16    • building block map  blockMapFile=dist/Keycard Certify-1.0.0-mac.zip.blockmap
12:51:16    ⨯ GitHub Personal Access Token is not set, neither programmatically, nor using env "GH_TOKEN"

Why does it want a GitHub token?

[jakubgs]

It seems like it's trying to publish the app to GitHub:

Up until yesterday I was able to publish my app to Github but after upgrading to Electron 4.1.3 I get the following error:

GitHub Personal Access Token is not set, neither programmatically, nor using env "GH_TOKEN"

• GitHub Personal Access Token is not set, neither programmatically, nor using env "GH_TOKEN" electron-userland/electron-builder#3792

But I don't want it to do that.

[jakubgs]

Apparently you can set publish: "never":

• How to disable publishing using programmatic API? electron-userland/electron-builder#1805

But where?

[choppu]

I'm checking why it is trying to publish.

[jakubgs]

Apparently you set it in build section in package.json:
https://www.electron.build/configuration/publish

[jakubgs]

And there we have it:

https://ci.infra.status.im/job/keycard-certify/job/platforms/job/macos/job/x86_64/18/

Now, the Windows part will be tricky.

[choppu]

Nice to hear that it works now. In any case, I have added the publish never option.

[jakubgs]

I found out that publish: "never" fails but publish: false works correctly.
I have added mine in the build section. I don't think using flags is the kosher way.

[choppu]

I added it as a command line flag because in theory the "publish" section should be an array of publishing providers whereas the flag says when you want to publish (never, always, on tag, etc). Of course setting publish: false has the side effect of disabling publishing. I think the flag is semantically more correct but either works for me

[jakubgs]

The "side effect" of disabling publishing is exactly what we want.

[choppu]

Ok, updated as requested.

[jakubgs]

In a perfect world I'd use Windows with WSL2 and then we could also use Nix on there, but I don't have that currently working.

@guylouis how important is the Windows platform? Because that's going to be the one that requires the most work.

[choppu]

by the way, since I am on Windows I can provide a windows binary if/when needed if it is not important to have automatic builds for it

[jakubgs]

Another major question is: how important is signing for this? Is this for internal use only or will this be published to external users?

Because signing setup is usually more tricky than just getting the build to work.

[guylouis]

Ok to not have automatic build for windows!

[guylouis]

Signing is nice to have but not mandatory since the app is for internal use

[jakubgs]

I will be working on Nix-based builds for Windows desktop app soon, and while doing that I will look into using WSL2 + Nix.

If that turns out to work well then we will also get Windows builds for this for free.

[jakubgs]

Looking good:

https://ci.infra.status.im/job/keycard-certify/job/manual/5/

But we're missing the snap file because it has different filename:

16:08:53    • building        target=snap arch=x64 file=dist/keycard-certify_1.0.0_amd64.snap
16:08:53    • building        target=AppImage arch=x64 file=dist/Keycard Certify-1.0.0.AppImage

I wonder if we can fix this.

[jakubgs]

Apparently it's conttrolled by the artifactName variable:

artifactName String | “undefined” - The artifact file name template. Defaults to ${productName}-${version}.${ext} (some target can have other defaults, see corresponding options).

Which in turn uses productName:

productName String | “undefined” - As name, but allows you to specify a product name for your executable which contains spaces and other special characters not allowed in the name property. If not specified inside of the build configuration, productName property defined at the top level of package.json is used. If not specified at the top level of package.json, name property is used.

Which we already have set:
https://github.com/status-im/keycard-certify/blob/608aacd9994c917235caf79e7e6cd67b6f209b2c/package.json#L15
So why is SNAP file output ignoring it?

[jakubgs]

Well, I just changed productName to be the same as is used for the SNAP file, so keycard-certify, same as repo name.

I'm assuming SNAP is using name field from package.json.

[jakubgs]

Okay, this looks pretty good:

https://ci.infra.status.im/job/keycard-certify/job/manual/10/

Next we test the GitHub release.

[guylouis]

I tried the mac build above and got the following error message. Seems related to the build with some dependancies on nix. cc @jakubgs

A javascript error happened in the main process

Uncaught Exception:
Error: dlopen(/private/var/folders/0f/3xbfmhr54f18h3r5gl006bn40000gn/T/AppTranslocation/F1511A16-9E5D-4907-BE2D-9379675AC04D/d/keycard-certify.app/Contents/Resources/app.asar.unpacked/node_modules/@pokusew/pcsclite/build/Release/pcsclite.node, 0x0001): Library not loaded: /nix/store/sxwkpd5cl8nkfkr98q51b5mh4rwqqlk6-libcxxabi-11.1.0/lib/libc++abi.1.dylib
Referenced from: <no uuid> /private/var/folders/0f/3xbfmhr54f18h3r5gl006bn40000gn/T/AppTranslocation/F1511A16-9E5D-4907-BE2D-9379675AC04D/d/keycard-certify.app/Contents/Resources/app.asar.unpacked/node_modules/@pokusew/pcsclite/build/Release/pcsclite.node
Reason: tried: '/nix/store/sxwkpd5cl8nkfkr98q51b5mh4rwqqlk6-libcxxabi-11.1.0/lib/libc++abi.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/nix/store/sxwkpd5cl8nkfkr98q51b5mh4rwqqlk6-libcxxabi-11.1.0/lib/libc++abi.1.dylib' (no such file), '/nix/store/sxwkpd5cl8nkfkr98q51b5mh4rwqqlk6-libcxxabi-11.1.0/lib/libc++abi.1.dylib' (no such file), '/usr/local/lib/libc++abi.1.dylib' (no such file), '/usr/lib/libc++abi.1.dylib' (no such file, not in dyld cache)
at process.func [as dlopen] (node:electron/js2c/asar_bundle:2:1822)
at Module._extensions..node (node:internal/modules/cjs/loader:1354:18)
at Object.func [as .node] (node:electron/js2c/asar_bundle:2:2049)
at Module.load (node:internal/modules/cjs/loader:1124:32)
at Module._load (node:internal/modules/cjs/loader:965:12)
at f._load (node:electron/js2c/asar_bundle:2:13330)
at Module.require (node:internal/modules/cjs/loader:1148:19)
at require (node:internal/modules/cjs/helpers:110:18)
at bindings (/private/var/folders/0f/3xbfmhr54f18h3r5gl006bn40000gn/T/AppTranslocation/F1511A16-9E5D-4907-BE2D-9379675AC04D/d/keycard-certify.app/Contents/Resources/app.asar/node_modules/bindings/bindings.js:112:48)
at Object.<anonymous> (/private/var/folders/0f/3xbfmhr54f18h3r5gl006bn40000gn/T/AppTranslocation/F1511A16-9E5D-4907-BE2D-9379675AC04D/d/keycard-certify.app/Contents/Resources/app.asar/node_modules/@pokusew/pcsclite/lib/pcsclite.js:4:37)

[jakubgs]

Please use triple backticks for pasting logs in markdown.

And it's not a dependency on Nix, it's a dependency on /usr/lib/libc++abi.1.dylib. I have no idea what that library is but seems like something standard on Darwin systems, no idea, I don't use Macs.

[choppu]

seems similar to https://github.com/orgs/Homebrew/discussions/3038#discussion-3911704

it might be that nix links to the runtime of the LLVM compiler it is managing and not the system one. The difference being, apparently, that the system lib is called /usr/lib/libc++.1.dylib (without abi in the name)

[jakubgs]

Yeah, I was reading that. Why do Macs always have to be different.

[jakubgs]

Actually, I've tested the build and it appears to work on Windows too:

https://ci.infra.status.im/job/keycard-certify/job/platforms/job/windows/job/x86_64/2/

Nice.

[jakubgs]

Here's the PR:

• ci: add Jenkinsfile and shell.nix for builds #2

If need be we can add signing later.