Skip to content

Commit

Permalink
feat: rebuild native dependencies for any project structure
Browse files Browse the repository at this point in the history
BREAKING CHANGE: rebuild native dependencies for any project structure
develar committed Nov 10, 2016
1 parent 94cc7be commit b1ea8cd
Showing 10 changed files with 96 additions and 88 deletions.
22 changes: 2 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@
A complete solution to package and build a ready for distribution Electron app for MacOS, Windows and Linux with “auto update” support out of the box.

* NPM packages management:
* [Native application dependencies](http://electron.atom.io/docs/latest/tutorial/using-native-node-modules/) compilation (only if the [two-package.json project structure](#two-packagejson-structure) is used).
* [Native application dependencies](http://electron.atom.io/docs/latest/tutorial/using-native-node-modules/) compilation.
* Development dependencies are never included. You don't need to ignore them explicitly.
* [Code Signing](https://github.com/electron-userland/electron-builder/wiki/Code-Signing) on a CI server or development machine.
* [Auto Update](#auto-update) ready application packaging.
@@ -12,6 +12,7 @@ A complete solution to package and build a ready for distribution Electron app f
* [MacOS](https://github.com/electron-userland/electron-builder/wiki/Options#MacOptions-target): `dmg`, `pkg`, `mas`.
* [Linux](https://github.com/electron-userland/electron-builder/wiki/Options#LinuxBuildOptions-target): `AppImage`, `deb`, `rpm`, `freebsd`, `pacman`, `p5p`, `apk`.
* [Windows](https://github.com/electron-userland/electron-builder/wiki/Options#WinBuildOptions-target): NSIS, Squirrel.Windows.
* [Two package.json Structure](https://github.com/electron-userland/electron-builder/wiki/Two-package.json-Structure) is supported, but you are not forced to use it even if you have native production dependencies.
* [Publishing artifacts](https://github.com/electron-userland/electron-builder/wiki/Publishing-Artifacts) to GitHub Releases and Bintray.

_Note: Platform specific `7zip-bin-*` packages are `optionalDependencies`, which may require manual install if you have npm configured to [not install optional deps by default](https://docs.npmjs.com/misc/config#optional)._
@@ -20,25 +21,6 @@ Real project example — [onshape-desktop-shell](https://github.com/develar/onsh

Consider to use `nsis` target for Windows ([auto-update](https://github.com/electron-userland/electron-builder/issues/529) will be implemented this month) for new projects.

# Two package.json structure

We recommend to use two package.json files (it is not required, you can build your project with any structure).

1. For development (`./package.json`)

The `package.json` resides in the root of your project. Here you declare the dependencies for your development environment and build scripts (`devDependencies`).

2. For your application (`./app/package.json`)

The `package.json` resides in the `app` directory. Declare your application dependencies (`depencencies`) here. *Only this directory is distributed with the final, packaged application.*

Why?

1. Native npm modules (those written in C, not JavaScript) need to be compiled and here we have two different compilation targets for them. Those used within the application need to be compiled against the electron runtime and all `devDependencies` need to be compiled against your local node.js environment. Thanks to the two `package.json` structure, this is trivial (see [#39](https://github.com/electron-userland/electron-builder/issues/39)).
2. No need to specify which [files](https://github.com/electron-userland/electron-builder/wiki/Options#BuildMetadata-files) to include in the app (because development files reside outside the `app` directory).

Please see [Loading App Dependencies Manually](https://github.com/electron-userland/electron-builder/wiki/Loading-App-Dependencies-Manually) and [#379](https://github.com/electron-userland/electron-builder/issues/379#issuecomment-218503881).

# Configuration

See [options](https://github.com/electron-userland/electron-builder/wiki/Options) for a full reference but consider following the simple guide outlined below first.
2 changes: 1 addition & 1 deletion docker/7/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM electronuserland/electron-builder:base

ENV NODE_VERSION 7.0.0
ENV NODE_VERSION 7.1.0

# https://github.com/npm/npm/issues/4531
RUN curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz" \
16 changes: 16 additions & 0 deletions docs/Two package.json Structure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Since version 8 electron-builder rebuilds only production dependencies, so, you are not forced to use two package.json structure.

1. For development (`./package.json`)

The `package.json` resides in the root of your project. Here you declare the dependencies for your development environment and build scripts (`devDependencies`).

2. For your application (`./app/package.json`)

The `package.json` resides in the `app` directory. Declare your application dependencies (`depencencies`) here. *Only this directory is distributed with the final, packaged application.*

Why?

1. Native npm modules (those written in C, not JavaScript) need to be compiled and here we have two different compilation targets for them. Those used within the application need to be compiled against the electron runtime and all `devDependencies` need to be compiled against your local node.js environment. Thanks to the two `package.json` structure, this is trivial (see [#39](https://github.com/electron-userland/electron-builder/issues/39)).
2. No need to specify which [files](https://github.com/electron-userland/electron-builder/wiki/Options#BuildMetadata-files) to include in the app (because development files reside outside the `app` directory).

Please see [Loading App Dependencies Manually](https://github.com/electron-userland/electron-builder/wiki/Loading-App-Dependencies-Manually) and [#379](https://github.com/electron-userland/electron-builder/issues/379#issuecomment-218503881).
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@
},
"scripts": {
"compile": "ts-babel . nsis-auto-updater test",
"lint": "tslint 'src/**/*.ts' 'test/src/**/*.ts' 'nsis-auto-updater/src/**/*.ts'",
"lint": "node ./test/lint.js",
"pretest": "yarn run compile && yarn run lint",
"check-deps": "node ./test/out/helpers/checkDeps.js",
"test": "node ./test/out/helpers/runTests.js",
4 changes: 2 additions & 2 deletions src/install-app-deps.ts
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ import BluebirdPromise from "bluebird-lst-c"
import { DevMetadata } from "./metadata"
import yargs from "yargs"
import { readPackageJson } from "./util/readPackageJson"
import { installDependencies } from "./yarn"
import { installDependencies, computeExtraArgs } from "./yarn"

const args: any = yargs
.option("arch", {
@@ -28,7 +28,7 @@ async function main() {
throw new Error("install-app-deps is only useful for two package.json structure")
}

await installDependencies(results[0], results[1], args.arch, devMetadata.build.npmSkipBuildFromSource !== true)
await installDependencies(results[0], results[1], args.arch, computeExtraArgs(devMetadata.build))
}

main()
44 changes: 16 additions & 28 deletions src/packager.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
import * as path from "path"
import {
computeDefaultAppDirectory, getElectronVersion, use, exec, isEmptyOrSpaces, exists,
asArray
} from "./util/util"
import { computeDefaultAppDirectory, getElectronVersion, use, exec, isEmptyOrSpaces, exists } from "./util/util"
import { all, executeFinally } from "./util/promise"
import { EventEmitter } from "events"
import BluebirdPromise from "bluebird-lst-c"
@@ -20,7 +17,7 @@ import { createTargets } from "./targets/targetFactory"
import { readPackageJson } from "./util/readPackageJson"
import { TmpDir } from "./util/tmp"
import { BuildOptions } from "./builder"
import { getGypEnv, installDependencies, rebuild } from "./yarn"
import { getGypEnv, installDependencies, rebuild, computeExtraArgs } from "./yarn"

function addHandler(emitter: EventEmitter, event: string, handler: Function) {
emitter.on(event, handler)
@@ -236,31 +233,22 @@ export class Packager implements BuildInfo {
})
}

if (this.isTwoPackageJsonProjectLayoutUsed) {
if (this.devMetadata.build.npmRebuild === false) {
log("Skip app dependencies rebuild because npmRebuild is set to false")
}
else {
const forceBuildFromSource = this.devMetadata.build.npmSkipBuildFromSource !== true
if (forceBuildFromSource && platform.nodeName !== process.platform) {
log("Skip app dependencies rebuild because platform is different")
}
else {
if (await exists(path.join(this.appDir, "node_modules"))) {
const args = asArray(this.devMetadata.build.npmArgs)
if (forceBuildFromSource) {
args.push("--build-from-source")
}
await rebuild(this.appDir, this.electronVersion, Arch[arch], args)
}
else {
await installDependencies(this.appDir, this.electronVersion, Arch[arch], forceBuildFromSource, this.devMetadata.build.npmArgs)
}
}
}
if (this.devMetadata.build.npmRebuild === false) {
log("Skip app dependencies rebuild because npmRebuild is set to false")
return
}

if (this.devMetadata.build.npmSkipBuildFromSource !== true && platform.nodeName !== process.platform) {
log("Skip app dependencies rebuild because platform is different")
}
else {
log("Skip app dependencies rebuild because dev and app dependencies are not separated")
const args = computeExtraArgs(this.devMetadata.build)
if (await exists(path.join(this.appDir, "node_modules"))) {
await rebuild(this.appDir, this.electronVersion, Arch[arch], args)
}
else if (this.isTwoPackageJsonProjectLayoutUsed) {
await installDependencies(this.appDir, this.electronVersion, Arch[arch], args)
}
}
}
}
4 changes: 2 additions & 2 deletions src/targets/dmg.ts
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ import { log, warn } from "../util/log"
import { PlatformPackager, TargetEx } from "../platformPackager"
import { MacOptions, DmgOptions, DmgContent } from "../options/macOptions"
import BluebirdPromise from "bluebird-lst-c"
import { debug, use, exec, statOrNull, isEmptyOrSpaces, spawn } from "../util/util"
import { debug, use, exec, statOrNull, isEmptyOrSpaces, spawn, exists } from "../util/util"
import { copy, unlink, outputFile, remove } from "fs-extra-p"
import { executeFinally } from "../util/promise"
import sanitizeFileName from "sanitize-filename"
@@ -54,7 +54,7 @@ export class DmgTarget extends TargetEx {
]).concat(tempDmg))

const volumePath = path.join("/Volumes", volumeName)
if (await statOrNull(volumePath) != null) {
if (await exists(volumePath)) {
debug("Unmounting previous disk image")
await detach(volumePath)
}
63 changes: 33 additions & 30 deletions src/yarn.ts
Original file line number Diff line number Diff line change
@@ -2,10 +2,11 @@ import BluebirdPromise from "bluebird-lst-c"
import * as path from "path"
import { task, log } from "./util/log"
import { homedir } from "os"
import { spawn, exists } from "./util/util"
import { spawn, exists, asArray } from "./util/util"
import { BuildMetadata } from "./metadata"

export function installDependencies(appDir: string, electronVersion: string, arch: string = process.arch, forceBuildFromSource: boolean, additionalArgs?: any): Promise<any> {
return task(`Installing app dependencies for arch ${arch} to ${appDir}`, spawnNpmProduction(appDir, forceBuildFromSource, getGypEnv(electronVersion, arch), additionalArgs))
export function installDependencies(appDir: string, electronVersion: string, arch: string = process.arch, additionalArgs: Array<string>): Promise<any> {
return task(`Installing app dependencies for arch ${arch} to ${appDir}`, spawnNpmProduction(appDir, getGypEnv(electronVersion, arch), additionalArgs))
}

export function getGypEnv(electronVersion: string, arch: string): any {
@@ -20,41 +21,44 @@ export function getGypEnv(electronVersion: string, arch: string): any {
})
}

function spawnNpmProduction(appDir: string, forceBuildFromSource: boolean, env?: any, additionalArgs?: any): Promise<any> {
export function computeExtraArgs(options: BuildMetadata) {
const args = asArray(options.npmArgs)
if (options.npmSkipBuildFromSource !== true) {
args.push("--build-from-source")
}
return args
}

function spawnNpmProduction(appDir: string, env: any, additionalArgs: Array<string>): Promise<any> {
let npmExecPath = process.env.npm_execpath || process.env.NPM_CLI_JS
const npmExecArgs = ["install", "--production"]

const isNotYarn = npmExecPath == null || !npmExecPath.includes("yarn")
if (isNotYarn) {
const isYarn = npmExecPath != null || npmExecPath.includes("yarn")
if (!isYarn) {
if (process.env.NPM_NO_BIN_LINKS === "true") {
npmExecArgs.push("--no-bin-links")
}
npmExecArgs.push("--cache-min", "999999999")
}

if (npmExecPath == null) {
npmExecPath = process.platform === "win32" ? "npm.cmd" : "npm"
npmExecPath = getPackageToolPath()
}
else {
npmExecArgs.unshift(npmExecPath)
npmExecPath = process.env.npm_node_execpath || process.env.NODE_EXE || "node"
}
if (isNotYarn && forceBuildFromSource) {
npmExecArgs.push("--build-from-source")
}

if (additionalArgs) {
if (Array.isArray(additionalArgs)) {
npmExecArgs.push(...additionalArgs)
}
else {
npmExecArgs.push(additionalArgs)
for (let a of additionalArgs) {
if (!isYarn || a !== "--build-from-source") {
npmExecArgs.push(a)
}
}

console.log("AAA " + npmExecPath + " " + npmExecArgs.join(" "))
return spawn(npmExecPath, npmExecArgs, {
cwd: appDir,
env: env || process.env
env: env
})
}

@@ -89,6 +93,15 @@ function flatDependencies(data: any, result: Set<string>, seen: Set<string>, ext
}
}

function getPackageToolPath() {
if (process.env.FORCE_YARN === "true") {
return process.platform === "win32" ? "yarn.cmd" : "yarn"
}
else {
return process.platform === "win32" ? "npm.cmd" : "npm"
}
}

export async function rebuild(appDir: string, electronVersion: string, arch: string = process.arch, additionalArgs: Array<string>) {
const deps = new Set<string>()
await dependencies(appDir, false, deps)
@@ -98,18 +111,13 @@ export async function rebuild(appDir: string, electronVersion: string, arch: str
return
}

log(`Rebuilding native app dependencies for arch ${arch} to ${appDir}`)
log(`Rebuilding native production dependencies for arch ${arch}`)

let execPath = process.env.npm_execpath || process.env.NPM_CLI_JS
const execArgs = ["run", "install", "--"]

if (execPath == null) {
if (process.env.FORCE_YARN === "true") {
execPath = process.platform === "win32" ? "yarn.cmd" : "yarn"
}
else {
execPath = process.platform === "win32" ? "npm.cmd" : "npm"
}
execPath = getPackageToolPath()
}
else {
execArgs.unshift(execPath)
@@ -128,10 +136,5 @@ export async function rebuild(appDir: string, electronVersion: string, arch: str
execArgs.push(`--arch=${arch}`)
execArgs.push(...additionalArgs)

for (let dir of nativeDeps) {
await spawn(execPath, execArgs, {
cwd: dir,
env: env
})
}
await BluebirdPromise.each(nativeDeps, it => spawn(execPath, execArgs, {cwd: it, env: env}))
}
23 changes: 21 additions & 2 deletions tslint.json → test/lint.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
{
const Linter = require("tslint")
const path = require("path")

const configuration = {
"extends": "tslint:recommended",
"rules": {
"member-ordering": [
@@ -51,6 +54,22 @@
"check-type"
],
"no-bitwise": false,
"jsdoc-format": false
"jsdoc-format": false,
"no-for-in-array": true
}
}
const options = {
configuration: configuration,
}

for (let projectDir of [path.join(__dirname, ".."), path.join(__dirname, "..", "nsis-auto-updater"), __dirname]) {
const program = Linter.createProgram("tsconfig.json", projectDir)
for (let file of Linter.getFileNames(program)) {
const fileContents = program.getSourceFile(file).getFullText()
const linter = new Linter(file, fileContents, options, program)
const out = linter.lint().output
if (out.length > 0) {
process.stdout.write(out)
}
}
}
4 changes: 2 additions & 2 deletions yarn.lock
Original file line number Diff line number Diff line change
@@ -1536,8 +1536,8 @@ dateformat@^1.0.11, dateformat@^1.0.12:
meow "^3.3.0"

debug@^2.1.1, debug@^2.1.3, debug@^2.2.0, debug@2:
version "2.3.0"
resolved "https://registry.yarnpkg.com/debug/-/debug-2.3.0.tgz#3912dc55d7167fc3af17d2b85c13f93deaedaa43"
version "2.3.2"
resolved "https://registry.yarnpkg.com/debug/-/debug-2.3.2.tgz#94cb466ef7d6d2c7e5245cdd6e4104f2d0d70d30"
dependencies:
ms "0.7.2"

0 comments on commit b1ea8cd

Please sign in to comment.