fix: efficient implementation of copy extra files/resources

docs/Options.md

@@ -54,7 +54,7 @@ Here documented only `electron-builder` specific options:
 | app-category-type | <a name="BuildMetadata-app-category-type"></a><p>*OS X-only.* The application category type, as shown in the Finder via *View -&gt; Arrange by Application Category* when viewing the Applications directory.</p> <p>For example, <code>app-category-type=public.app-category.developer-tools</code> will set the application category to *Developer Tools*.</p> <p>Valid values are listed in [Apple’s documentation](https://developer.apple.com/library/ios/documentation/General/Reference/InfoPlistKeyReference/Articles/LaunchServicesKeys.html#//apple_ref/doc/uid/TP40009250-SW8).</p>
 | asar | <a name="BuildMetadata-asar"></a><p>Whether to package the application’s source code into an archive, using [Electron’s archive format](https://github.com/electron/asar). Defaults to <code>true</code>. Reasons why you may want to disable this feature are described in [an application packaging tutorial in Electron’s documentation](http://electron.atom.io/docs/latest/tutorial/application-packaging/#limitations-on-node-api/).</p> <p>Or you can pass object of any asar options.</p>
 | productName | <a name="BuildMetadata-productName"></a>See [AppMetadata.productName](#AppMetadata-productName).
-| extraResources | <a name="BuildMetadata-extraResources"></a><p>A [glob expression](https://www.npmjs.com/package/glob#glob-primer), when specified, copy the file or directory with matching names directly into the app’s resources directory (<code>Contents/Resources</code> for OS X, <code>resources</code> for Linux/Windows).</p> <p>You can use <code>${os}</code> (expanded to osx, linux or win according to current platform) and <code>${arch}</code> in the pattern.</p> <p>If directory matched, all contents are copied. So, you can just specify <code>foo</code> to copy <code>&lt;project_dir&gt;/foo</code> directory.</p> <p>May be specified in the platform options (i.e. in the <code>build.osx</code>).</p>
+| extraResources | <a name="BuildMetadata-extraResources"></a><p>A [glob expression](https://www.npmjs.com/package/glob#glob-primer), when specified, copy the file or directory with matching names directly into the app’s resources directory (<code>Contents/Resources</code> for OS X, <code>resources</code> for Linux/Windows). [Multiple patterns](#multiple-glob-patterns) are supported.</p> <p>You can use <code>${os}</code> (expanded to osx, linux or win according to current platform) and <code>${arch}</code> in the pattern.</p> <p>If directory matched, all contents are copied. So, you can just specify <code>foo</code> to copy <code>&lt;project_dir&gt;/foo</code> directory.</p> <p>May be specified in the platform options (e.g. in the <code>build.osx</code>).</p>
 | extraFiles | <a name="BuildMetadata-extraFiles"></a>The same as [extraResources](#BuildMetadata-extraResources) but copy into the app's content directory (`Contents` for OS X, `` for Linux/Windows).
 | osx | <a name="BuildMetadata-osx"></a>See [.build.osx](#OsXBuildOptions).
 | mas | <a name="BuildMetadata-mas"></a>See [.build.mas](#MasBuildOptions).
@@ -107,16 +107,31 @@ MAS (Mac Application Store) specific options (in addition to `build.osx`).
 | synopsis | <a name="LinuxBuildOptions-synopsis"></a>*deb-only.* The [short description](https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Description).
 | maintainer | <a name="LinuxBuildOptions-maintainer"></a>The maintainer. Defaults to [author](#AppMetadata-author).
 | vendor | <a name="LinuxBuildOptions-vendor"></a>The vendor. Defaults to [author](#AppMetadata-author).
-| compression | <a name="LinuxBuildOptions-compression"></a>*deb-only.* The compression type, one of `gz`, `bzip2`, `xz` (default: `xz`).
+| compression | <a name="LinuxBuildOptions-compression"></a>*deb-only.* The compression type, one of `gz`, `bzip2`, `xz`. Defaults to `xz`.
 | depends | <a name="LinuxBuildOptions-depends"></a>Package dependencies. Defaults to `["libappindicator1", "libnotify-bin"]`.
 | target | <a name="LinuxBuildOptions-target"></a><p>Target package type: list of <code>default</code>, <code>deb</code>, <code>rpm</code>, <code>freebsd</code>, <code>pacman</code>, <code>p5p</code>, <code>apk</code>, <code>7z</code>, <code>zip</code>, <code>tar.xz</code>, <code>tar.lz</code>, <code>tar.gz</code>, <code>tar.bz2</code>. Defaults to <code>default</code> (<code>deb</code>).</p> <p>Only <code>deb</code> is tested. Feel free to file issues for <code>rpm</code> and other package formats.</p>
 
 <a name="MetadataDirectories"></a>
 ## `.directories`
 | Name | Description
 | --- | ---
-| buildResources | <a name="MetadataDirectories-buildResources"></a>The path to build resources, default `build`.
-| output | <a name="MetadataDirectories-output"></a>The output directory, default `dist`.
-| app | <a name="MetadataDirectories-app"></a>The application directory (containing the application package.json), default `app`, `www` or working directory.
+| buildResources | <a name="MetadataDirectories-buildResources"></a>The path to build resources, defaults to `build`.
+| output | <a name="MetadataDirectories-output"></a>The output directory, defaults to `dist`.
+| app | <a name="MetadataDirectories-app"></a>The application directory (containing the application package.json), defaults to `app`, `www` or working directory.
 
 <!-- end of generated block -->
+
+
+# Multiple Glob Patterns
+ ```js
+ [
+   // match all files
+   "**/*",
+
+   // except for js files in the foo/ directory
+   "!foo/*.js",
+
+   // unless it's foo/bar.js
+   "foo/bar.js",
+ ]
+ ```

package.json

@@ -61,7 +61,7 @@
     "asar": "^0.11.0",
     "bluebird": "^3.4.0",
     "chalk": "^1.1.3",
-    "compare-versions": "^2.0.1",
+    "compare-versions": "^2.0.2",
     "debug": "^2.2.0",
     "deep-assign": "^2.0.0",
     "electron-osx-sign-tf": "0.6.0",
@@ -73,6 +73,7 @@
     "image-size": "^0.5.0",
     "lodash.template": "^4.2.5",
     "mime": "^1.3.4",
+    "minimatch": "^3.0.0",
     "progress": "^1.1.8",
     "progress-stream": "^1.2.0",
     "read-package-json": "^2.0.4",
@@ -106,10 +107,10 @@
     "pre-git": "^3.8.4",
     "semantic-release": "^6.3.0",
     "should": "^9.0.0",
-    "ts-babel": "^0.8.6",
+    "ts-babel": "^1.0.0",
     "tsconfig-glob": "^0.4.3",
     "tslint": "3.10.0-dev.2",
-    "typescript": "1.9.0-dev.20160520-1.0",
+    "typescript": "1.9.0-dev.20160607-1.0",
     "whitespace": "^2.0.0"
   },
   "babel": {

src/linuxPackager.ts

@@ -66,7 +66,7 @@ export class LinuxPackager extends PlatformPackager<LinuxBuildOptions> {
 
     promises.push(this.computeDesktop(tempDir))
 
-    return [].concat(...await BluebirdPromise.all(promises))
+    return Array.prototype.concat.apply([], await BluebirdPromise.all(promises))
   }
 
   async pack(outDir: string, arch: Arch, targets: Array<string>, postAsyncTasks: Array<Promise<any>>): Promise<any> {

src/metadata.ts

@@ -109,12 +109,13 @@ export interface BuildMetadata {
 
   /**
    A [glob expression](https://www.npmjs.com/package/glob#glob-primer), when specified, copy the file or directory with matching names directly into the app's resources directory (`Contents/Resources` for OS X, `resources` for Linux/Windows).
+   [Multiple patterns](#multiple-glob-patterns) are supported.
 
    You can use `${os}` (expanded to osx, linux or win according to current platform) and `${arch}` in the pattern.
 
    If directory matched, all contents are copied. So, you can just specify `foo` to copy `<project_dir>/foo` directory.
 
-   May be specified in the platform options (i.e. in the `build.osx`).
+   May be specified in the platform options (e.g. in the `build.osx`).
    */
   readonly extraResources?: Array<string> | null
 
@@ -307,7 +308,7 @@ export interface LinuxBuildOptions extends PlatformSpecificBuildOptions {
   afterRemove?: string | null
 
   /*
-  *deb-only.* The compression type, one of `gz`, `bzip2`, `xz` (default: `xz`).
+  *deb-only.* The compression type, one of `gz`, `bzip2`, `xz`. Defaults to `xz`.
    */
   readonly compression?: string | null
 
@@ -329,17 +330,17 @@ export interface LinuxBuildOptions extends PlatformSpecificBuildOptions {
  */
 export interface MetadataDirectories {
   /*
-   The path to build resources, default `build`.
+   The path to build resources, defaults to `build`.
    */
   readonly buildResources?: string | null
 
   /*
-   The output directory, default `dist`.
+   The output directory, defaults to `dist`.
    */
   readonly output?: string | null
 
   /*
-   The application directory (containing the application package.json), default `app`, `www` or working directory.
+   The application directory (containing the application package.json), defaults to `app`, `www` or working directory.
    */
   readonly app?: string | null
 }

src/platformPackager.ts

@@ -4,14 +4,14 @@ import EventEmitter = NodeJS.EventEmitter
 import { Promise as BluebirdPromise } from "bluebird"
 import * as path from "path"
 import { pack, ElectronPackagerOptions } from "electron-packager-tf"
-import { globby } from "./globby"
 import { readdir, copy, unlink, lstat, remove } from "fs-extra-p"
 import { statOrNull, use, spawn, debug7zArgs, debug, warn, log } from "./util"
 import { Packager } from "./packager"
 import { listPackage, statFile, AsarFileMetadata, createPackageFromFiles, AsarOptions } from "asar"
 import { path7za } from "7zip-bin"
 import deepAssign = require("deep-assign")
 import { Glob } from "glob"
+import { Minimatch } from "minimatch"
 
 //noinspection JSUnusedLocalSymbols
 const __awaiter = require("./awaiter")
@@ -234,15 +234,6 @@ export abstract class PlatformPackager<DC extends PlatformSpecificBuildOptions>
     return options
   }
 
-  private getExtraResources(isResources: boolean, arch: Arch, customBuildOptions: DC): Promise<Set<string>> {
-    let patterns: Array<string> | n = (<any>this.devMetadata.build)[isResources ? "extraResources" : "extraFiles"]
-    const platformSpecificPatterns = isResources ? customBuildOptions.extraResources : customBuildOptions.extraFiles
-    if (platformSpecificPatterns != null) {
-      patterns = patterns == null ? platformSpecificPatterns : patterns.concat(platformSpecificPatterns)
-    }
-    return patterns == null ? BluebirdPromise.resolve(new Set()) : globby(this.expandPatterns(patterns, arch), {cwd: this.projectDir});
-  }
-
   private computeAsarOptions(customBuildOptions: DC): AsarOptions | null {
     let result = this.devMetadata.build.asar
     let platformSpecific = customBuildOptions.asar
@@ -313,20 +304,50 @@ export abstract class PlatformPackager<DC extends PlatformSpecificBuildOptions>
     await remove(src)
   }
 
-  private expandPatterns(list: Array<string>, arch: Arch): Array<string> {
-    return list.map(it => it
+  private expandPattern(pattern: string, arch: Arch): string {
+    return pattern
       .replace(/\$\{arch}/g, Arch[arch])
-      .replace(/\$\{os}/g, this.platform.buildConfigurationKey))
+      .replace(/\$\{os}/g, this.platform.buildConfigurationKey)
   }
 
   protected async copyExtraFiles(appOutDir: string, arch: Arch, customBuildOptions: DC): Promise<any> {
     await this.doCopyExtraFiles(true, appOutDir, arch, customBuildOptions)
     await this.doCopyExtraFiles(false, appOutDir, arch, customBuildOptions)
   }
 
-  private async doCopyExtraFiles(isResources: boolean, appOutDir: string, arch: Arch, customBuildOptions: DC): Promise<Array<string>> {
+  private async doCopyExtraFiles(isResources: boolean, appOutDir: string, arch: Arch, customBuildOptions: DC): Promise<any> {
     const base = isResources ? this.getResourcesDir(appOutDir) : this.platform === Platform.OSX ? path.join(appOutDir, `${this.appName}.app`, "Contents") : appOutDir
-    return await BluebirdPromise.map(await this.getExtraResources(isResources, arch, customBuildOptions), it => copy(path.join(this.projectDir, it), path.join(base, it)))
+
+    let patterns: Array<string> | n = (<any>this.devMetadata.build)[isResources ? "extraResources" : "extraFiles"]
+    const platformSpecificPatterns = isResources ? customBuildOptions.extraResources : customBuildOptions.extraFiles
+    if (platformSpecificPatterns != null) {
+      patterns = patterns == null ? platformSpecificPatterns : patterns.concat(platformSpecificPatterns)
+    }
+
+    if (patterns == null) {
+      return
+    }
+
+    const minimatchOptions = {}
+    const parsedPatterns: Array<Minimatch> = []
+    for (let i = 0; i < patterns.length; i++) {
+      parsedPatterns[i] = new Minimatch(this.expandPattern(patterns[i], arch), minimatchOptions)
+    }
+
+    const src = this.projectDir
+    return copy(src, base, {
+      filter: (it) => {
+        if (src === it) {
+          return true
+        }
+
+        let relative = it.substring(src.length + 1)
+        if (path.sep === "\\") {
+          relative = relative.replace(/\\/g, "/")
+        }
+        return minimatchAll(relative, parsedPatterns)
+      }
+    })
   }
 
   protected async computePackageUrl(): Promise<string | null> {
@@ -501,3 +522,28 @@ export function smarten(s: string): string {
   s = s.replace(/"/g, "\u201d")
   return s
 }
+
+// https://github.com/joshwnj/minimatch-all/blob/master/index.js
+function minimatchAll(path: string, patterns: Array<Minimatch>): boolean {
+  let match = false
+  for (let pattern of patterns) {
+    // If we've got a match, only re-test for exclusions.
+    // if we don't have a match, only re-test for inclusions.
+    if (match !== pattern.negate) {
+      continue
+    }
+
+    // partial match — pattern: foo/bar.txt path: foo — we must allow foo
+    match = pattern.match(path, true)
+    if (!match && !pattern.negate) {
+      const rawPattern = pattern.pattern
+      // 1 - slash
+      const patternLengthPlusSlash = rawPattern.length + 1
+      if (path.length > patternLengthPlusSlash) {
+        // foo: include all directory content
+        match = path[rawPattern.length] === "/" && path.startsWith(rawPattern)
+      }
+    }
+  }
+  return match
+}

test/src/globTest.ts

@@ -29,7 +29,7 @@ test.ifDevOrLinuxCi("ignore build resources", () => {
   })
 })
 
-test("copy extra content", async () => {
+test("extraResources", async () => {
   for (let platform of getPossiblePlatforms().keys()) {
     const osName = platform.buildConfigurationKey
 

test/tsconfig.json

@@ -42,6 +42,7 @@
     "../typings/main/definitions/debug/index.d.ts",
     "../typings/main/definitions/source-map-support/source-map-support.d.ts",
     "../typings/modules/glob/index.d.ts",
+    "../typings/modules/minimatch/index.d.ts",
     "../typings/node.d.ts",
     "../typings/progress-stream.d.ts",
     "../typings/read-package-json.d.ts",

tsconfig.json

@@ -46,6 +46,7 @@
     "typings/main/definitions/debug/index.d.ts",
     "typings/main/definitions/source-map-support/source-map-support.d.ts",
     "typings/modules/glob/index.d.ts",
+    "typings/modules/minimatch/index.d.ts",
     "typings/node.d.ts",
     "typings/progress-stream.d.ts",
     "typings/read-package-json.d.ts",
@@ -65,7 +66,6 @@
     "src/fpmDownload.ts",
     "src/gitHubPublisher.ts",
     "src/gitHubRequest.ts",
-    "src/globby.ts",
     "src/httpRequest.ts",
     "src/index.ts",
     "src/install-app-deps.ts",

typings.json

@@ -5,6 +5,7 @@
   },
   "dependencies": {
     "glob": "registry:npm/glob#6.0.0+20160211003958",
+    "minimatch": "registry:npm/minimatch#3.0.0+20160211003958",
     "source-map-support": "github:typed-typings/npm-source-map-support#900ed4180a22285bce4bbabc0760427e71a59eca"
   }
 }