proton-packager

[mischnic]

I have built a cli tool to package proton apps (currently only macOS bundles are supported).
Maybe there is someone who knows how the linux (or windows) packaging should work and would like to contribute (platform-independent parts can be reused).

https://github.com/mischnic/proton-packager

This could be added the Mac section of the "Packaging" docs.

#39 @gordygordy This will create a proper .app bundle.

---

The main thing on macOS, that is keeping bundles from being indistinguishable from native apps, is #13.

[kusti8]

I've been experimenting with nexe and pkg and I've got them working. I would prefer creating a single binary this way, rather than creating a .app with what you've done. I would love the keep the cli tool, but replace the packaging with nexe. I just need to finish up testing and then I'll make a write-up

[kusti8]

Ok, so pkg works. I'm going to add this to create-proton-app. One problem which you can still solve is the fact that nbind.node and the libui library still have to be present outside the executable.

[mischnic]

Pkg and nexe both create a single binary file regardless of the platform (please correct me if I'm wrong). Double-clicking such a file on macOS would first open a Terminal and the app itself would start. It's also not possible to give an icon to a single binary file which is then also visible in the Dock.

My solution:

How it would look like on mac without an .app bundle:

[kusti8]

Ok. I'm still going to create a single binary using pkg and then you can create an .app from that.

[albe-rosado]

I see, this week me too will be working on packaging issues with nexe/pkg. I like what @mischnic has done, but something more cross-platform would be ideal. Although we could detect the user's platform an expose the right script for them in the package.json file: ie.
In Mac:

{
  "name": "my-proton-app",
  "scripts": {
    "package": "proton-packager mac MyApp -f"
  },
  ...
}

And in Linux/Windows:

{
  "name": "my-proton-app",
  "scripts": {
    "package": "nexe --build"//or just `node build.js` and add nexe as dev dep.
  },
  ...
}

Other thing, @kusti8 having library files next to the binary file is not ideal but isn't too bad either, we can design a installer kind of thing that will expose a shortcut to the user and copy our files to other folder, in Linux, something like having an my-app.desktop file will get the user the app in their launcher in debian based distros (at least), and in Windows, I guess doing just a shortcut and sending our files to C:\\program\ files\. What do you think?

[mischnic]

I want thinking of integrating the nexe/pkg approach into proton-packager, so the user could just pass win instead of mac. (Maybe even the windows installer creation if needed)

[albe-rosado]

@mischnic yeah, that would be ideal.

[kusti8]

Yeah that sounds like a good idea. Libui-node may go statically linked which would make it easier to package.

…

On Mon, Mar 5, 2018, 7:54 PM Alberto Rosado ***@***.***> wrote: @mischnic <https://github.com/mischnic> yeah, that would be ideal. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#43 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJW73PWZyL5iGAy2lTFRoMlv6i96ncoxks5tbd5dgaJpZM4SbT9e> .

[albe-rosado]

I have been working in issuing appImages files for Linux, so with @mischnic piece and mine(in progress ), we will only need something similar for windows?

[albe-rosado]

Added the appImage packaging for Linux @kusti8 , @mischnic . Take a look at gitter. Update create-proton-app to 1.1.0.

[develar]

I suggest you to join forces. I can add proton support to electron-builder. Despite the name, electron can be easily decoupled (actually, modules is already decoupled and some part of code even available as a separate internal tool app-builder https://github.com/develar/app-builder).

So, you can just implement specific for proton pars, and you get generic features that electron-builder provides for free and right now:

• publishing
• a lot of targets (including cross-platform and bundled tools for any OS without manual installation by user)
• proven and tested implementation, since electron-builder is a well established and popular tool (e.g. Snapcraft team and AppImage team helps us directly).
• code signing

[albe-rosado]

I think that would be a good idea, ideally, and this is what I'm working to achieve on create-proton-app(always open to feedback) is raw performance, as little dependencies as possible, and usability. That is why I have chosen to write bash scripts for appImages and MacOS app bundles. I don't know what you mean for "proton pars" @develar , at the end it depends on @kusti8 decision, if this is what we decide I will be happy to include that logic. In the other hand, the library you reference is written in Go if any bug is reported we would be relying on the maintainers of that library to get it fixed, It will affect the stability of this as well. I will be trying it out to see how it goes. I'm happy to see we may finally have a "one size fits all" kind of thing.

[mischnic]

The problem I see with create-proton-app is that distributing new script versions is quite hard because it is installed globally (create-react-app had this as well). Also, putting the script into the created project means that the user has to update it himself. This is why separate packagers are better suited (and they can be used in projects created without create-...-app).

That is why I have chosen to write bash scripts for appImages

What if you wanted to package for linux on windows (after we got precompiled binaries for libui-node)? This is the reason why most scripts in the npm ecosystem are node scripts themselves.

In the other hand, the library you reference is written in Go if any bug is reported we would be relying on the maintainers of that library to get it fixed, It will affect the stability of this as well

So what is better, creating an untested script from scratch or using an already tested tool?

[develar]

write bash scripts for appImages and MacOS app bundles

electron-builder is able to produce AppImage on macOS/Linux and no need to install AppImage tools manually. The same for snap. And for other targets (yes, we don't try to reinvent the wheel — FPM is used under the hood for deb, rpm, apk and other Linux targets). "usability"

what you mean for "proton pars"

1. Packaging. Transform user project into packable format, suitable for distributable formats (dmg, appimage, snap and so on)
2. Apply user project metadata (e.g. modify icon and file properties (windows executable))
3. Sign binaries
4. Package into distributable format.
5. Sign artifacts (required for NSI, DMG and MSI).
6. Publish binaries.

Here framework specific only first step (yes, because of performance, other steps can be also depends, for example, electron-builder tries to do all using in-memory and avoid file copying). "proton pars" means "how to transform user project into packable format" (step 1).

the library you reference

Because of usability, NodeJS cannot be used for all raw and native things. Native node module it is hell (must be no native modules to ensure that users will not suffer). JS implementation of some things in most cases leads to bloated package ("as little dependencies as possible"). So, image tasks (electron-builder is able to produce icns or ico from one png file), downloading (multi-part downloader cannot be effectively implemented using NodeJS) and other low-level tasks implemented in Golang.

we would be relying on the maintainers of that library to get it fixed

Golang is much more better suitable language for build tools. But in any case right now electron-builder implemented in TS. Golang used only for low-level tasks.

[albe-rosado]

mmm, I see, totally forgot the case when someone on windows wants to produce other platform binaries, LOL. I like Golang, and I like app-builder. @mischnic right now you can use create-proton-app with npx so no need to install stuff globally. but I see your point on using shell scripts and you are right. I'm thinking on wrapping app-builder with a node script and make it super generic, so anyone could package their binary to their favorite format from their home platform. That's definetely awesome. @develar

[develar]

Ok. If no one has some objections, I will incorporate https://github.com/mischnic/proton-packager into a new module proton-builder, that will use electron-builder-lib under the hood and will decouple electron related code.

For example, issues like mischnic/proton-packager#3 will be in the past, since electron-builder is able correctly copy only production deps (including support for Yarn/Lerna workspaces). (actually, proton-packager can also use electron-builder-lib for such tasks, but I think it will be better to reuse common electron-builder framework — I mean framework will delegate specific tasks to, instead "some other tool will provide own front-end and use generic lib functions).

[albe-rosado]

Awesome!!! 👍 I just add that to the scripts and we will be done with packaging.

[mischnic]

👍

[kusti8]

Sounds good.

[develar]

I have done first part (download node and prepare app template, code is taken from https://github.com/mischnic/proton-packager/blob/master/src/mac.js) of required changes to electron-builder to support proton (btw, electron-builder also supports muon, so, electron was already a little bit decoupled).

• node downloading and extracting using Go, so, no dependency on decompress, no new nodejs dependencies. Change compared to proton-packager — to reduce download time, tar.xz is used instead of tar.gz.
• generated Info.plist now contains correct version.

electron-builder is not yet able to build Proton, I will continue (develar/electron-builder@f20d7cc#diff-d0d77e592f1e2eac7e50d28153a8d0c0). Next part — copying project files.

These changes is not yet in the master branch of electron-builder, it is my working branch.

[mischnic]

@develar Two important things that (I think) are different to electron (just as a reminder):

• the build folder in node_modules/libui-node has to be deleted, because it is ~80mb big and not needed anymore. The file node_modules/libui-node/nbind.node has to stay (that was my problem in Remove devDependencies after copying into bundle mischnic/proton-packager#3).
• the source files need to be transpiled with babel, because we are using JSX.

[develar]

@mischnic Thanks. Also, as far I see, libui-node/docs, libui-node/src should be also ignored.

[mischnic]

Technically yes, but in total that's at most only 500kb.

[develar]

the source files need to be transpiled with babel

It is a little bit strange for me, since it should be handled somehow outside of builder tool to be fully controlled by user... but I totally agree that this functionality should be fully integrated since most users will use runtime compilation during development.

For example, electron-webpack is not supported by electron-builder, instead, electron-webpack provides special files configuration (https://github.com/electron-userland/electron-webpack/blob/master/packages/electron-webpack/electron-builder.yml). Yes — another advantage of using electron-builder to package Proton is that you will get all functionality that was implemented and requested for Electron apps for last 3 years :) https://www.electron.build/configuration/contents

So, it will be easy for user to package whatever complex project structure. But by default solution like electron-compile support (build on the fly without intermediate files) and how it is currently implemented in the proton-packager.

BTW — for Windows it is not good to have a lot of small files. Solution or https://github.com/pmq20/node-packer (on a first glance, development env will be very complex to setup), or WebPack bundle (easy solution, but requires changes to the app and can lead to runtime errors due to difference to development version). In any case for now I want to support Linux/macOS at first.

[mischnic]

Yes, a bundler would work as well, might even be faster running the app then.

I don't fully understand how electron-webpack and electron-bundler work together, but we can use that if they do.

[mischnic]

The icon is in #102 (as a svg).

[develar]

I have converted provided SVG file to ICNS, electron-builder 20.14.3 uses it as a default icon for macOS.

[zlfnhmd]

Hey guys what is the status on windows support? @kusti8 @develar @mischnic is there anyway i can help?

[mischnic]

In the meantime, I've set up an example for how to use launchui-packager with proton-native: https://github.com/mischnic/proton-launchui.

[zlfnhmd]

@mischnic The main question to be answered with using launchui-packager is how to update the app after publishing. Is there any workaround for that?

[develar]

https://github.com/electron-userland/electron-builder/releases/tag/v20.36.1

1. New option framework is added to support pure libui. One of electron, proton-native, libui. Defaults to electron. proton vs libui — for now, only babel is not used for libui. And libui is not tested at all.

2. nodeVersion is added and option protonNodeVersion is deprecated. So, to build for proton, you now need to set framework to proton. And set nodeVersion if you want specific node version (see below why for now it doesn't taken in account for Linux/Windows).

3. electron-builder now supports building proton applications for Windows. Because libui currently doesn't provide prebuilt binaries, I decided to use launchui. It means that now you can build for Linux and Windows on any OS.

   How does it work? app-builder instructed to exclude libui-node dependency during production dependency graph building. So, packed application doesn't have libui-node dependency (and all sub dependencies) at all. App deps located in app/node_modules. Launchui skeleton provides libui in root node_modules.

   So, you can build for Windows or Linux on any OS. Binaries available on https://github.com/develar/launchui/releases For now only v0.1.4-10.13.0 is available (Node 10.13.0 LTS). It is easy to build for other version because appveyor.yml is provided (you can send PR and appveyour will build it and then you can grab artifacts and do not wait when it will be merged and released ;))

4. For Linux only x64 is supported (for bloody Windows I have set up AppVeyour, for Linux unexpectedly I also cannot build ia32, so, need to add Travis (it is required anyway for security reasons). If you don't want to use launchui for Linux and can build Proton app on Linux, set "launchUiVersion": false (obviously, this false is respected only for Linux).

5. Default icon for Linux is fixed.

6. rcedit doesn't support LaunchUI on Wine (crashes). So, app installer will have correct meta info and icon (because of amazing NSIS), but installed app icon will be not. I found solution, but it will be implemented not in electron-builder 20.36.0

I think this issue can be closed. Further issue please file to electron-builder issue tracker (prefix with "proton: " to ensure that it will be fixed faster than electron framework issues ;)).

[develar]

What do you think about launchui? Right now, when no prebuilds for libui, it is the only solution to build proton app on any OS for any OS. In case of Windows no variants — bat file or another script is not good. But for Linux and macOS... I am worry about fact that in case of LaunchUI user cannot easily use own node version. But... probably it is ok as I have prepared appveyor and so, it is easy to release new LaunchUI builds for new versions of libui/node.

[mischnic]

libui binaries are already available and libui-node prebuilds should be in the near future (not sure where I said that a few days ago - there are too many issues regarding the packaging 😄).
Is there really no other way than recompiling node to not always have a black command prompt window on Windows? (Only advantage of launchui on Linux/macOS: reporting fatal errors in a dialog instead of printing them to the console.)

[develar]

Is there really no other way than recompiling node to not always have a black command prompt window on Windows?

Actually we can simply use link args as solution (for both desktop and start-up shortcuts). Another solution is VBScript. For me problem is that on bloody Windows I didn't managed to make libui working and instead of investigating WHY bloody libui-node cannot find compiled libui node native module I decided to just use launchui (I forked it and build using latest LTS node). Maybe on this weekend I will do another round of debugging libui-node and bloody windows.

[develar]

BTW — I have security concerns of using node.exe directly with arg. Not sure are all antiviruses will be happy. It is why I gave up and started to use LaunchUI for Windows. Windows is so bad that it is better to avoid any unclear solution for packaging apps, that can be used in various conditions.

[develar]

Ok, I have managed to rebuild *** native module and now Windows is happy to run app and load native module directly without LaunchUI.

Problem is that Windows doesn't have such concept as application. On Linux or macOS you can define your application as unit, but on Windows it is just a bunch of files.

See screenshot.

Nice? Nice. So, user can by mistake run notepad.exe instead of notepad and then will be disappointed, because notepad.exe here it is just a plain node.exe that expects some args to run app. Of course, I am not Windows user, but I am sure that there are number of another such gotchas, that as result will ruin and discredit the whole idea to use libui apps.

Yes — we can name helper executable as -helper to make clear that it is an internal, sub executable.

Should we? I think we should not. The only price of LaunchUI — send PR once in several months to build a new release for recent libui/nodejs version. And as AppVeyour file to do it is provided, even if main maintainers doesn't provide required versions for you, it takes only 30-40 minutes (per arch) to build your own if need (fork repo, add project to AppVeyour, change download-deps.json, wait 30 minutes and grab artifacts from build — no need even touch Windows and deal with Microsoft tools).

"avoid base binaries" vs "app will differ from native and consists of two executable (that will also slighly slowdown start because it is Windows)"

And don't tell me about AppX — thanks to Microsoft, this technology supported only on Windows 10 and so, we can use it only in 2030 or later.

[mimecorg]

Just to clarify, my intent is to maintain LaunchUI and keep releasing new versions, and the AppVeyor integration will definitely help with that, but first there must be some consensus whether it's the "right" solution or not.

[mischnic]

@develar In case you haven't seen it, here is the initial "proposal" for launchui.

@mimecorg Could launchui just use the precompiled libui and libui-node (-napi) binaries once they're available?

[mimecorg]

@mimecorg Could launchui just use the precompiled libui and libui-node (-napi) binaries once they're available?

It could, but I think that libui-node still has to be recompiled to link against node.dll instead of node.exe, even if it's using N-API.

[mischnic]

libui-node still has to be recompiled to link against node.dll instead of node.exe, even if it's using N-API.

I don't think so (at least on macOS). libui-napi isn't linked to anything regarding node. I'm guessing that everything of that is done when node loads the module at runtime.

ui.node:
	@rpath/libui.A.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)

[develar]

but first there must be some consensus whether it's the "right" solution or not.

NodeJS doesn't going to offer official solution — nodejs/node#556 So, it means that for now even if you can build simple launcher like

package main

import (
  "os"
  "os/exec"
  "path/filepath"
)

func main() {
  ex, err := os.Executable()
  if err != nil {
    panic(err)
  }

  appDir := filepath.Dir(ex)

  command := exec.Command(filepath.Join(appDir, "node.exe"), filepath.Join(appDir, "app", "main.js"))
  command.Dir = appDir
  //command.Stdout = os.Stderr
  //command.Stderr = os.Stderr
  err = command.Run()
  if err != nil {
    panic(err)
  }
}

GOOS=windows GOARCH=amd64 go build -ldflags='-s -w -H=windowsgui' -o dist/app-runner.exe

you still have to rebuild node.exe to avoid console window. So, I see that LaunchUI is the only solution.

[antillgrp]

One question guys, will these executable (i.e. .exe) be nodejs independent, no node platform installed needed in the target machine, or am I still gonna need node installed to run it?

Which is, as of right now, the more straight forward solution? This #43 (comment) ?

[mischnic]

One question guys, will these executable (i.e. .exe) be nodejs independent, no node platform installed needed in the target machine, or am I still gonna need node installed to run it?

They don't require node to be installed. And by now, electron-builder supports Windows with proton-native (so all platforms).

[antillgrp]

There was something I did not understand well, do I run that script over the folder (no matter how many js files are conforming the app) or do I have to consolidate everything in a single js file as was done for notepad 1.0.1 example?

[kusti8]

If you have installed proton native with create-proton-app, then you can just run npm run build && npm run dist to automatically create a packaged version. It will scan for all files.

[antillgrp]

Is there any way to create a single .exe file instead of an application folder full of dlls?

I found it

https://www.electron.build/configuration/nsis

Portable¶

To build portable app, set target to portable (or pass --win portable).

For portable app, following environment variables are available:

PORTABLE_EXECUTABLE_DIR - dir where portable executable located.
PORTABLE_EXECUTABLE_APP_FILENAME - sanitized app name to use in file paths.

[yrik]

And by now, electron-builder supports Windows with proton-native (so all platforms).

For some reason on the website it says that

Currently, only macOS and Linux supported.

See https://www.electron.build/configuration/configuration#proton-native

Is that an outdated doc or still not officially fixed?

[yrik]

I'm having some strange issues with proton native and native packages (keytar),
electron-userland/electron-builder#3736
Even super simple app built with create-proton-app and only keytar as a dependency fails when packaged with electron-builder. However, it works fine with npm run start

Maybe anyone has a quick idea on how to approach it?

I have tried to build it on Win 10 with Node 8 and Node 11