Skip to content

Commit

Permalink
feat: Development dependencies are never copied in any case
Browse files Browse the repository at this point in the history
You don't need to ignore it explicitly anymore
  • Loading branch information
develar committed Jun 8, 2016
1 parent fc1587f commit 6d4ab11
Show file tree
Hide file tree
Showing 9 changed files with 160 additions and 113 deletions.
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
osx_image: xcode7
osx_image: xcode7.3

matrix:
include:
Expand All @@ -13,7 +13,6 @@ language: c
cache:
directories:
- node_modules
- test/testApp/node_modules
- $HOME/.electron
- $HOME/.cache/fpm

Expand All @@ -23,6 +22,7 @@ before_install:

install:
- nvm install $NODE_VERSION
- nvm use --delete-prefix $NODE_VERSION
- if [[ "$TRAVIS_OS_NAME" == "osx" && "$NODE_VERSION" == "4" ]]; then npm install npm -g ; fi
- npm install
- npm prune
Expand Down
4 changes: 1 addition & 3 deletions docs/Options.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Here documented only `electron-builder` specific options:
| app-category-type | <a name="BuildMetadata-app-category-type"></a><p>*OS X-only.* The application category type, as shown in the Finder via *View -&gt; Arrange by Application Category* when viewing the Applications directory.</p> <p>For example, <code>app-category-type=public.app-category.developer-tools</code> will set the application category to *Developer Tools*.</p> <p>Valid values are listed in [Apple’s documentation](https://developer.apple.com/library/ios/documentation/General/Reference/InfoPlistKeyReference/Articles/LaunchServicesKeys.html#//apple_ref/doc/uid/TP40009250-SW8).</p>
| asar | <a name="BuildMetadata-asar"></a><p>Whether to package the application’s source code into an archive, using [Electron’s archive format](https://github.com/electron/asar). Defaults to <code>true</code>. Reasons why you may want to disable this feature are described in [an application packaging tutorial in Electron’s documentation](http://electron.atom.io/docs/latest/tutorial/application-packaging/#limitations-on-node-api/).</p> <p>Or you can pass object of any asar options.</p>
| productName | <a name="BuildMetadata-productName"></a>See [AppMetadata.productName](#AppMetadata-productName).
| files | <a name="BuildMetadata-files"></a><p>A [glob patterns](https://www.npmjs.com/package/glob#glob-primer) relative to the [app directory](#MetadataDirectories-app), which specifies which files to include when copying files to create the package. Defaults to <code>\*\*\/\*</code> (i.e. [hidden files are ignored by default](https://www.npmjs.com/package/glob#dots)).</p> <p>[Multiple patterns](#multiple-glob-patterns) are supported. You can use <code>${os}</code> (expanded to osx, linux or win according to current platform) and <code>${arch}</code> in the pattern.</p> <p>If directory matched, all contents are copied. So, you can just specify <code>foo</code> to copy <code>foo</code> directory.</p> <p>Remember that default pattern <code>\*\*\/\*</code> is not added to your custom, so, you have to add it explicitly — e.g. <code>[&quot;\*\*\/\*&quot;, &quot;!ignoreMe${/\*}&quot;]</code>.</p> <p>May be specified in the platform options (e.g. in the <code>build.osx</code>).</p>
| files | <a name="BuildMetadata-files"></a><p>A [glob patterns](https://www.npmjs.com/package/glob#glob-primer) relative to the [app directory](#MetadataDirectories-app), which specifies which files to include when copying files to create the package. Defaults to <code>\*\*\/\*</code> (i.e. [hidden files are ignored by default](https://www.npmjs.com/package/glob#dots)).</p> <p>Development dependencies are never copied in any case. You don’t need to ignore it explicitly.</p> <p>[Multiple patterns](#multiple-glob-patterns) are supported. You can use <code>${os}</code> (expanded to osx, linux or win according to current platform) and <code>${arch}</code> in the pattern. If directory matched, all contents are copied. So, you can just specify <code>foo</code> to copy <code>foo</code> directory.</p> <p>Remember that default pattern <code>\*\*\/\*</code> is not added to your custom, so, you have to add it explicitly — e.g. <code>[&quot;\*\*\/\*&quot;, &quot;!ignoreMe${/\*}&quot;]</code>.</p> <p>May be specified in the platform options (e.g. in the <code>build.osx</code>).</p>
| extraResources | <a name="BuildMetadata-extraResources"></a><p>A [glob patterns](https://www.npmjs.com/package/glob#glob-primer) relative to the project directory, when specified, copy the file or directory with matching names directly into the app’s resources directory (<code>Contents/Resources</code> for OS X, <code>resources</code> for Linux/Windows).</p> <p>Glob rules the same as for [files](#BuildMetadata-files).</p>
| extraFiles | <a name="BuildMetadata-extraFiles"></a>The same as [extraResources](#BuildMetadata-extraResources) but copy into the app's content directory (`Contents` for OS X, root directory for Linux/Windows).
| osx | <a name="BuildMetadata-osx"></a>See [.build.osx](#OsXBuildOptions).
Expand All @@ -63,8 +63,6 @@ Here documented only `electron-builder` specific options:
| linux | <a name="BuildMetadata-linux"></a>See [.build.linux](#LinuxBuildOptions).
| compression | <a name="BuildMetadata-compression"></a>The compression level, one of `store`, `normal`, `maximum` (default: `normal`). If you want to rapidly test build, `store` can reduce build time significantly.
| afterPack | <a name="BuildMetadata-afterPack"></a>*programmatic API only* The function to be run after pack (but before pack into distributable format and sign). Promise must be returned.
| npmPrune | <a name="BuildMetadata-npmPrune"></a><p>Whether to [prune](https://docs.npmjs.com/cli/prune) native dependencies (<code>npm prune --production</code>) before starting to package the app. Defaults to <code>true</code> if [two package.json structure](https://github.com/electron-userland/electron-builder#two-packagejson-structure) is not used.</p>
| npmRebuild | <a name="BuildMetadata-npmRebuild"></a>Whether to [rebuild](https://docs.npmjs.com/cli/rebuild) native dependencies (`npm rebuild`) before starting to package the app. Defaults to `true`.

<a name="OsXBuildOptions"></a>
### `.build.osx`
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@
"pre-git": "^3.8.4",
"semantic-release": "^6.3.0",
"should": "^9.0.0",
"ts-babel": "^1.0.0",
"ts-babel": "^1.0.2",
"tsconfig-glob": "^0.4.3",
"tslint": "3.10.0-dev.2",
"typescript": "1.9.0-dev.20160607-1.0",
Expand Down
2 changes: 1 addition & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ export { Packager } from "./packager"
export { PackagerOptions, ArtifactCreated, DIR_TARGET, BuildInfo } from "./platformPackager"
export { BuildOptions, build, createPublisher, CliOptions, createTargets } from "./builder"
export { PublishOptions, Publisher } from "./gitHubPublisher"
export { AppMetadata, DevMetadata, Platform, Arch, archFromString, getProductName, BuildMetadata, OsXBuildOptions, WinBuildOptions, LinuxBuildOptions } from "./metadata"
export { AppMetadata, DevMetadata, Platform, Arch, archFromString, getProductName, BuildMetadata, OsXBuildOptions, WinBuildOptions, LinuxBuildOptions, CompressionLevel } from "./metadata"
6 changes: 4 additions & 2 deletions src/metadata.ts
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ export interface AuthorMetadata {
readonly email: string
}

export type CompressionLevel = "store" | "normal" | "maximum"

/*
## `.build`
*/
Expand Down Expand Up @@ -156,7 +158,7 @@ export interface BuildMetadata {
/*
The compression level, one of `store`, `normal`, `maximum` (default: `normal`). If you want to rapidly test build, `store` can reduce build time significantly.
*/
readonly compression?: "store" | "normal" | "maximum" | null
readonly compression?: CompressionLevel | null

readonly "build-version"?: string | null

Expand All @@ -171,7 +173,7 @@ export interface BuildMetadata {
// */
// readonly npmPrune?: boolean
// deprecated
readonly prune?: boolean
// readonly prune?: boolean

/*
Whether to [rebuild](https://docs.npmjs.com/cli/rebuild) native dependencies (`npm rebuild`) before starting to package the app. Defaults to `true`.
Expand Down
164 changes: 61 additions & 103 deletions src/platformPackager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,18 @@ import EventEmitter = NodeJS.EventEmitter
import { Promise as BluebirdPromise } from "bluebird"
import * as path from "path"
import { pack, ElectronPackagerOptions, userIgnoreFilter } from "electron-packager-tf"
import { readdir, copy, unlink, lstat, remove } from "fs-extra-p"
import { statOrNull, use, spawn, debug7zArgs, debug, warn, log, spawnNpmProduction } from "./util"
import { readdir, copy, unlink, lstat, remove, realpath } from "fs-extra-p"
import { statOrNull, use, warn, log, exec } from "./util"
import { Packager } from "./packager"
import { listPackage, statFile, AsarFileMetadata, createPackageFromFiles, AsarOptions } from "asar"
import { path7za } from "7zip-bin"
import { archiveApp } from "./targets/archive"
import { Glob } from "glob"
import { Minimatch } from "minimatch"
import deepAssign = require("deep-assign")

//noinspection JSUnusedLocalSymbols
const __awaiter = require("./awaiter")

class CompressionDescriptor {
constructor(public flag: string, public env: string, public minLevel: string, public maxLevel: string = "-9") {
}
}

const extToCompressionDescriptor: { [key: string]: CompressionDescriptor; } = {
"tar.xz": new CompressionDescriptor("--xz", "XZ_OPT", "-0", "-9e"),
"tar.lz": new CompressionDescriptor("--lzip", "LZOP", "-0"),
"tar.gz": new CompressionDescriptor("--gz", "GZIP", "-1"),
"tar.bz2": new CompressionDescriptor("--bzip2", "BZIP2", "-1"),
}

export const commonTargets = ["dir", "zip", "7z", "tar.xz", "tar.lz", "tar.gz", "tar.bz2"]

export const DIR_TARGET = "dir"
Expand Down Expand Up @@ -168,18 +156,32 @@ export abstract class PlatformPackager<DC extends PlatformSpecificBuildOptions>
promise = copy(this.info.appDir, appPath, {filter: userIgnoreFilter(opts), dereference: true})
}
else {
const ignoreFiles = new Set([path.relative(this.info.appDir, opts.out!), path.relative(this.info.appDir, this.buildResourcesDir)])
if (!this.info.isTwoPackageJsonProjectLayoutUsed) {
const result = await BluebirdPromise.all([listDependencies(this.info.appDir, false), listDependencies(this.info.appDir, true)])
const productionDepsSet = new Set(result[1])

// npm returns real path, so, we should use relative path to avoid any mismatch
const realAppDirPath = await realpath(this.info.appDir)

for (let it of result[0]) {
if (!productionDepsSet.has(it)) {
if (it.startsWith(realAppDirPath)) {
it = it.substring(realAppDirPath.length + 1)
}
else if (it.startsWith(this.info.appDir)) {
it = it.substring(this.info.appDir.length + 1)
}
ignoreFiles.add(it)
}
}
}

let patterns = this.getFilePatterns("files", customBuildOptions)
if (patterns == null || patterns.length === 0) {
patterns = ["**/*"]
}

const parsedPatterns = this.getParsedPatterns(patterns, arch)
if (!this.info.isTwoPackageJsonProjectLayoutUsed) {
const dotOptions = {dot: true}
parsedPatterns.push(new Minimatch("!node_modules/@(appdmg|electron-download|electron-builder|electron-prebuilt|electron-packager-tf|electron-winstaller-fixed|electron-osx-sign-tf|electron-osx-sign){,/**/*}", dotOptions))
parsedPatterns.push(new Minimatch(`!@(${path.relative(this.info.appDir, this.buildResourcesDir)}|${path.relative(this.info.appDir, opts.out!)}){,/**/*}`, dotOptions))
}
promise = copyFiltered(this.info.appDir, appPath, parsedPatterns, true)
promise = copyFiltered(this.info.appDir, appPath, this.getParsedPatterns(patterns, arch), true, ignoreFiles)
}

const promises = [promise]
Expand All @@ -193,24 +195,8 @@ export abstract class PlatformPackager<DC extends PlatformSpecificBuildOptions>

await BluebirdPromise.all(promises)

let npmPrune = this.devMetadata.build.npmPrune
if (npmPrune == null) {
npmPrune = this.devMetadata.build.prune
if (npmPrune != null) {
warn("prune is deprecated and renamed to npmPrune, please specify as npmPrune")
}
}

if (npmPrune == null) {
npmPrune = !this.info.isTwoPackageJsonProjectLayoutUsed
}
else if (typeof npmPrune !== "boolean") {
throw new Error(`npmPrune expected to be boolean value, but string '"${npmPrune}"' was specified`)
}

if (npmPrune) {
log("Pruning app dependencies")
await spawnNpmProduction("prune", appPath)
if (opts.prune != null) {
warn("prune is deprecated — development dependencies are never copied in any case")
}

if (asarOptions != null) {
Expand Down Expand Up @@ -469,66 +455,7 @@ export abstract class PlatformPackager<DC extends PlatformSpecificBuildOptions>
}

protected async archiveApp(format: string, appOutDir: string, outFile: string): Promise<any> {
const compression = this.devMetadata.build.compression
const storeOnly = compression === "store"

const dirToArchive = this.platform === Platform.OSX ? path.join(appOutDir, `${this.appName}.app`) : appOutDir
if (format.startsWith("tar.")) {
// we don't use 7z here - develar: I spent a lot of time making pipe working - but it works on OS X and often hangs on Linux (even if use pipe-io lib)
// and in any case it is better to use system tools (in the light of docker - it is not problem for user because we provide complete docker image).
const info = extToCompressionDescriptor[format]
let tarEnv = process.env
if (compression != null && compression !== "normal") {
tarEnv = Object.assign({}, process.env)
tarEnv[info.env] = storeOnly ? info.minLevel : info.maxLevel
}

await spawn(process.platform === "darwin" || process.platform === "freebsd" ? "gtar" : "tar", [info.flag, "--transform", `s,^\.,${path.basename(outFile, "." + format)},`, "-cf", outFile, "."], {
cwd: dirToArchive,
stdio: ["ignore", debug.enabled ? "inherit" : "ignore", "inherit"],
env: tarEnv
})
return
}

const args = debug7zArgs("a")
if (compression === "maximum") {
if (format === "7z" || format.endsWith(".7z")) {
args.push("-mx=9", "-mfb=64", "-md=32m", "-ms=on")
}
else if (format === "zip") {
// http://superuser.com/a/742034
//noinspection SpellCheckingInspection
args.push("-mfb=258", "-mpass=15")
}
else {
args.push("-mx=9")
}
}
else if (storeOnly) {
if (format !== "zip") {
args.push("-mx=1")
}
}

// remove file before - 7z doesn't overwrite file, but update
try {
await unlink(outFile)
}
catch (e) {
// ignore
}

if (format === "zip" || storeOnly) {
args.push("-mm=" + (storeOnly ? "Copy" : "Deflate"))
}

args.push(outFile, dirToArchive)

await spawn(path7za, args, {
cwd: path.dirname(dirToArchive),
stdio: ["ignore", debug.enabled ? "inherit" : "ignore", "inherit"],
})
return archiveApp(this.devMetadata.build.compression, format, outFile, this.platform === Platform.OSX ? path.join(appOutDir, `${this.appName}.app`) : appOutDir)
}
}

Expand Down Expand Up @@ -592,15 +519,21 @@ function minimatchAll(path: string, patterns: Array<Minimatch>): boolean {
return match
}

function copyFiltered(src: string, destination: string, patterns: Array<Minimatch>, dereference: boolean = false): Promise<any> {
// we use relative path to avoid canonical path issue - e.g. /tmp vs /private/tmp
function copyFiltered(src: string, destination: string, patterns: Array<Minimatch>, dereference: boolean = false, ignoreFiles?: Set<string>): Promise<any> {
return copy(src, destination, {
dereference: dereference,
filter: it => {
if (src === it) {
return true
}

let relative = it.substring(src.length + 1)

// yes, check before path sep normalization
if (ignoreFiles != null && ignoreFiles.has(relative)) {
return false
}

if (path.sep === "\\") {
relative = relative.replace(/\\/g, "/")
}
Expand All @@ -612,4 +545,29 @@ function copyFiltered(src: string, destination: string, patterns: Array<Minimatc
export function computeEffectiveTargets(rawList: Array<string>, targetsFromMetadata: Array<string> | n): Array<string> {
let targets = normalizeTargets(rawList.length === 0 ? targetsFromMetadata : rawList)
return targets == null ? ["default"] : targets
}

async function listDependencies(appDir: string, production: boolean): Promise<Array<string>> {
let npmExecPath = process.env.npm_execpath || process.env.NPM_CLI_JS
const npmExecArgs = ["ls", production ? "--production" : "--dev", "--parseable"]
if (npmExecPath == null) {
npmExecPath = process.platform === "win32" ? "npm.cmd" : "npm"
}
else {
npmExecArgs.unshift(npmExecPath)
npmExecPath = process.env.npm_node_execpath || process.env.NODE_EXE || "node"
}

const result = (await exec(npmExecPath, npmExecArgs, {
cwd: appDir,
stdio: "inherit",
maxBuffer: 1024 * 1024,
})).trim().split("\n")
if (result.length > 0 && !result[0].includes("/node_modules/")) {
// first line is a project dir
const lastIndex = result.length - 1
result[0] = result[lastIndex]
result.length = result.length - 1
}
return result
}
81 changes: 81 additions & 0 deletions src/targets/archive.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import { spawn, debug, debug7zArgs } from "../util"
import { CompressionLevel } from "../metadata"
import * as path from "path"
import { unlink } from "fs-extra-p"
import { path7za } from "7zip-bin"

//noinspection JSUnusedLocalSymbols
const __awaiter = require("../awaiter")

class CompressionDescriptor {
constructor(public flag: string, public env: string, public minLevel: string, public maxLevel: string = "-9") {
}
}

const extToCompressionDescriptor: { [key: string]: CompressionDescriptor; } = {
"tar.xz": new CompressionDescriptor("--xz", "XZ_OPT", "-0", "-9e"),
"tar.lz": new CompressionDescriptor("--lzip", "LZOP", "-0"),
"tar.gz": new CompressionDescriptor("--gz", "GZIP", "-1"),
"tar.bz2": new CompressionDescriptor("--bzip2", "BZIP2", "-1"),
}

export async function archiveApp(compression: CompressionLevel | n, format: string, outFile: string, dirToArchive: string): Promise<any> {
const storeOnly = compression === "store"

if (format.startsWith("tar.")) {
// we don't use 7z here - develar: I spent a lot of time making pipe working - but it works on OS X and often hangs on Linux (even if use pipe-io lib)
// and in any case it is better to use system tools (in the light of docker - it is not problem for user because we provide complete docker image).
const info = extToCompressionDescriptor[format]
let tarEnv = process.env
if (compression != null && compression !== "normal") {
tarEnv = Object.assign({}, process.env)
tarEnv[info.env] = storeOnly ? info.minLevel : info.maxLevel
}

await spawn(process.platform === "darwin" || process.platform === "freebsd" ? "gtar" : "tar", [info.flag, "--transform", `s,^\.,${path.basename(outFile, "." + format)},`, "-cf", outFile, "."], {
cwd: dirToArchive,
stdio: ["ignore", debug.enabled ? "inherit" : "ignore", "inherit"],
env: tarEnv
})
return
}

const args = debug7zArgs("a")
if (compression === "maximum") {
if (format === "7z" || format.endsWith(".7z")) {
args.push("-mx=9", "-mfb=64", "-md=32m", "-ms=on")
}
else if (format === "zip") {
// http://superuser.com/a/742034
//noinspection SpellCheckingInspection
args.push("-mfb=258", "-mpass=15")
}
else {
args.push("-mx=9")
}
}
else if (storeOnly) {
if (format !== "zip") {
args.push("-mx=1")
}
}

// remove file before - 7z doesn't overwrite file, but update
try {
await unlink(outFile)
}
catch (e) {
// ignore
}

if (format === "zip" || storeOnly) {
args.push("-mm=" + (storeOnly ? "Copy" : "Deflate"))
}

args.push(outFile, dirToArchive)

await spawn(path7za, args, {
cwd: path.dirname(dirToArchive),
stdio: ["ignore", debug.enabled ? "inherit" : "ignore", "inherit"],
})
}
Loading

0 comments on commit 6d4ab11

Please sign in to comment.