Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gulp is sometimes slow to start #632

Closed
kud opened this issue Aug 21, 2014 · 16 comments
Closed

gulp is sometimes slow to start #632

kud opened this issue Aug 21, 2014 · 16 comments

Comments

@kud
Copy link

kud commented Aug 21, 2014

On my machine with lots of softwares (sketch, firefox as frontend dev, VMs, etc), I don't have enough ram for gulp and its start is in this case really slow.

Once it was started, all is ok, the compilation is really fast and the next time I start gulp, it'll be fast too.

Note : This issue is for the moment a memo, I'll be back with more information. ;)

An indication of a compilation without troubles:

Thu 21 Aug 2014 12:08:33 CEST
[12:08:34] Using gulpfile ~/xxx/xxx/xxx/gulpfile.js
[12:08:34] Starting 'clean:dist'...
[12:08:34] Finished 'clean:dist' after 16 ms
[12:08:34] Starting 'clean:build'...
[12:08:34] Finished 'clean:build' after 252 μs
[12:08:34] Starting 'assets'...
[12:08:34] Starting 'images'...
[12:08:34] Starting 'svg'...
[12:08:34] Starting 'templates'...
[12:08:34] Finished 'assets' after 170 ms
[12:08:34] Finished 'templates' after 177 ms
[12:08:34] Starting 'scripts'...
[12:08:34] Finished 'svg' after 384 ms
[12:08:34] Starting 'styles:all'...
[12:08:38] Finished 'styles:all' after 3.92 s
[12:08:38] Finished 'scripts' after 4.15 s
[12:08:38] Finished 'images' after 4.43 s
[12:08:38] Starting 'compile'...
[12:08:38] Finished 'compile' after 9.18 μs
Thu 21 Aug 2014 12:08:38 CEST

Packages:

{
    "gulp": "3.8.6",
    "gulp-concat": "2.2.0",
    "gulp-consolidate": "0.1.2",
    "gulp-declare": "0.2.0",
    "gulp-define-module": "0.1.1",
    "gulp-filter": "0.5.0",
    "gulp-handlebars": "2.1.0",
    "gulp-iconfont": "0.1.0",
    "gulp-imagemin": "0.6.1",
    "gulp-jshint": "1.6.3",
    "gulp-livereload": "2.1.0",
    "gulp-minify-css": "0.3.5",
    "gulp-myth": "0.3.1",
    "gulp-plumber": "0.6.3",
    "gulp-rename": "1.2.0",
    "gulp-replace": "0.3.0",
    "gulp-rev": "0.4.2",
    "gulp-svg-symbols": "0.1.2",
    "gulp-svgmin": "0.4.6",
    "gulp-uglify": "0.3.1",
    "gulp-util": "2.2.17"
  }

gulpfile.js:

/**
 * Imports
 */
var gulp       = require('gulp')
var livereload = require('gulp-livereload')

/**
 * Helpers
 */
var path = __dirname + '/gulp/'

/**
 * Private tasks
 */
gulp.task('clean:dist', require(path + 'clean').dist)
gulp.task('clean:build', require(path + 'clean').build)
gulp.task('assets', require(path + 'assets'))
gulp.task('images', require(path + 'images'))
gulp.task('glyphicons', require(path + 'glyphicons'))
gulp.task('svg', require(path + 'svg'))
gulp.task('styles', require(path + 'styles'))
gulp.task('styles:all', ['svg'], require(path + 'styles'))
gulp.task('jshint', require(path + 'jshint'))
gulp.task('scripts', ['templates'], require(path + 'scripts'))
gulp.task('templates', require(path + 'templates'))


/**
 * Public tasks
 */
gulp.task('compile', ['clean:dist', 'clean:build', 'assets', 'images', 'styles:all', 'scripts'])
gulp.task('watch', ['compile'], function() {

  livereload.listen()

  gulp.watch('src/frontend/assets/**/*}', ['assets'])
  gulp.watch('src/frontend/images/**/*}', ['images'])
  gulp.watch('src/frontend/**/{*.js, *.hbs}', ['scripts'])
  gulp.watch('src/frontend/styles/**/*.css', ['styles'])
  gulp.watch(['web/assets/**/*', 'resources/views/**/*.tpl']).on('change', livereload.changed)

})

node : v0.10.31
macosx : 10.9.4


An example of task:

var gulp     = require('gulp')

var es       = require('event-stream')

var gutil    = require('gulp-util')
var imagemin = require('gulp-imagemin')
var svgmin   = require('gulp-svgmin')

module.exports = function() {

  var bitmap = gulp.src('src/frontend/images/**/{*.png,*.gif,*.jpg,*.jpeg}')
      .pipe( gutil.env.dist ? imagemin( { optimizationLevel: 5 } ) : gutil.noop()  )

  var vector = gulp.src('src/frontend/images/**/*.svg')
      .pipe( gutil.env.dist ? svgmin() : gutil.noop()  )

  return es.merge( bitmap, vector).pipe( gulp.dest('web/assets/images') )

}
@kud kud changed the title gulp is slow to start gulp is sometimes slow to start Aug 21, 2014
@yocontra
Copy link
Member

gulpfile? node version? OS?

@yocontra
Copy link
Member

First guess: too many dependencies being loaded on start is taxing the fs?

Solution: lazily load plugins and deps in each task instead of at the top of the file

@kud
Copy link
Author

kud commented Aug 21, 2014

Updated ;) What do you think so?

@kud
Copy link
Author

kud commented Aug 21, 2014

It's certainly the loading of tasks/dependencies which is slow, but how to do it better? my code isn't correct there? it doesn't lazyload 'em?

@yocontra
Copy link
Member

To lazyload, this

var gulp     = require('gulp')

var es       = require('event-stream')

var gutil    = require('gulp-util')
var imagemin = require('gulp-imagemin')
var svgmin   = require('gulp-svgmin')

module.exports = function() {

  var bitmap = gulp.src('src/frontend/images/**/{*.png,*.gif,*.jpg,*.jpeg}')
      .pipe( gutil.env.dist ? imagemin( { optimizationLevel: 5 } ) : gutil.noop()  )

  var vector = gulp.src('src/frontend/images/**/*.svg')
      .pipe( gutil.env.dist ? svgmin() : gutil.noop()  )

  return es.merge( bitmap, vector).pipe( gulp.dest('web/assets/images') )

}

would become this

module.exports = function() {

  var gulp     = require('gulp')

  var es       = require('event-stream')

  var gutil    = require('gulp-util')
  var imagemin = require('gulp-imagemin')
  var svgmin   = require('gulp-svgmin')

  var bitmap = gulp.src('src/frontend/images/**/{*.png,*.gif,*.jpg,*.jpeg}')
      .pipe( gutil.env.dist ? imagemin( { optimizationLevel: 5 } ) : gutil.noop()  )

  var vector = gulp.src('src/frontend/images/**/*.svg')
      .pipe( gutil.env.dist ? svgmin() : gutil.noop()  )

  return es.merge( bitmap, vector).pipe( gulp.dest('web/assets/images') )

}

@yocontra
Copy link
Member

Also I would recommend using gulp-if instead of the gutil.env.dist ? work() : gutil.noop() thing you are doing

@sindresorhus
Copy link
Contributor

You could also lazy require the gulp plugins so they're only required on first use.

@kud
Copy link
Author

kud commented Aug 22, 2014

Thank you, I'll try all your advice. ;)

@kud
Copy link
Author

kud commented Aug 22, 2014

After some tests, I must admit on a whole compilation, it changes not really (but it was premeditated). Instead of taking lots of time to start, gulp takes some times on each task.

@sindresorhus
Copy link
Contributor

@kud you can use time-require to figure out which require is the slowest. But the require mechanism is expensive, especially on non-SSD harddrives.

@kud
Copy link
Author

kud commented Aug 22, 2014

grazie mille ;)

@kud
Copy link
Author

kud commented Aug 22, 2014

Interesting:

Start time: (2014-08-22 13:34:47 UTC) [treshold=1%]
 #  module                                             time  %
 1  gulp-util (node_module...s/gulp-util/index.js)     1.8s  ▇▇ 3%
 2  ./lib/src (node_module...-fs/lib/src/index.js)     1.8s  ▇▇ 3%
 3  vinyl-fs (node_modules...es/vinyl-fs/index.js)     2.5s  ▇▇ 4%
 4  gulp (node_modules/gulp/index.js)                  4.7s  ▇▇▇▇ 7%
 5  gulp-util (node_module...s/gulp-util/index.js)     1.4s  ▇ 2%
 6  lodash._createwrapper...eatewrapper/index.js)     843ms  ▇ 1%
 7  lodash.bind (node_modu...lodash.bind/index.js)    889ms  ▇ 1%
 8  lodash._basecreatecall...atecallback/index.js)    977ms  ▇ 1%
 9  lodash.merge (node_mod...odash.merge/index.js)     1.9s  ▇▇ 3%
10  gulp-livereload (node_...d/gulp-livereload.js)     3.9s  ▇▇▇ 6%
11  gulp-util (node_modules/gulp-util/index.js)        1.4s  ▇ 2%
12  gulp-util (node_module...s/gulp-util/index.js)     1.2s  ▇ 2%
13  bin-wrapper (node_modu...bin-wrapper/index.js)    728ms  ▇ 1%
14  gifsicle (node_modules...es/gifsicle/index.js)     1.3s  ▇ 2%
15  imagemin-gifsicle (nod...in-gifsicle/index.js)     1.8s  ▇▇ 3%
16  imagemin-jpegtran (nod...in-jpegtran/index.js)    751ms  ▇ 1%
17  imagemin-optipng (node...min-optipng/index.js)    736ms  ▇ 1%
18  imagemin-pngquant (nod...in-pngquant/index.js)    727ms  ▇ 1%
19  ./schema/default_safe...hema/default_safe.js)      1.6s  ▇▇ 2%
20  ./js-yaml/loader (node...ib/js-yaml/loader.js)       2s  ▇▇ 3%
21  ./lib/js-yaml.js (node...-yaml/lib/js-yaml.js)     2.2s  ▇▇ 3%
22  js-yaml (node_modules/...les/js-yaml/index.js)     2.3s  ▇▇ 3%
23  ./svgo/config (node_mo...o/lib/svgo/config.js)     2.6s  ▇▇ 4%
24  svgo (node_modules/gul...les/svgo/lib/svgo.js)     3.4s  ▇▇▇ 5%
25  imagemin-svgo (node_mo...agemin-svgo/index.js)     3.5s  ▇▇▇ 5%
26  imagemin (node_modules...es/imagemin/index.js)     8.1s  ▇▇▇▇▇▇ 12%
27  gulp-imagemin (node_mo...lp-imagemin/index.js)     9.4s  ▇▇▇▇▇▇▇ 14%
28  ./lib/_stream_readable.../_stream_readable.js)    866ms  ▇ 1%
29  readable-stream (node_...e-stream/readable.js)     1.3s  ▇ 2%
30  bufferstreams (node_mo...streams/src/index.js)     1.4s  ▇ 2%
31  ./json (node_modules/g...-yaml/schema/json.js)    722ms  ▇ 1%
32  ./core (node_modules/g...-yaml/schema/core.js)    747ms  ▇ 1%
33  ./schema/default_safe...hema/default_safe.js)      1.4s  ▇ 2%
34  ./schema/default_full...hema/default_full.js)     708ms  ▇ 1%
35  ./js-yaml/loader (node...ib/js-yaml/loader.js)     2.3s  ▇▇ 3%
36  ./lib/js-yaml.js (node...-yaml/lib/js-yaml.js)     2.7s  ▇▇ 4%
37  js-yaml (node_modules/...les/js-yaml/index.js)     3.4s  ▇▇▇ 5%
38  ./svgo/config (node_mo...o/lib/svgo/config.js)     3.5s  ▇▇▇ 5%
39  svgo (node_modules/gul...les/svgo/lib/svgo.js)       4s  ▇▇▇ 6%
40  gulp-util (node_module...s/gulp-util/index.js)     1.6s  ▇▇ 2%
41  gulp-svgmin (node_modu...gulp-svgmin/index.js)     7.2s  ▇▇▇▇▇ 10%
42  /Users/kud/Projects/_p...ages (gulp/images.js)    18.3s  ▇▇▇▇▇▇▇▇▇▇▇▇▇ 27%
43  gulp-util (node_module...s/gulp-util/index.js)     1.6s  ▇▇ 2%
44  svgicons2svgfont (node...svgfont/src/index.js)    894ms  ▇ 1%
45  lodash.template (node_...sh.template/index.js)    690ms  ▇ 1%
46  ./lib/template (node_m...util/lib/template.js)    695ms  ▇ 1%
47  gulp-util (node_module...s/gulp-util/index.js)     2.6s  ▇▇ 4%
48  gulp-svgicons2svgfont...svgfont/src/index.js)      3.8s  ▇▇▇ 6%
49  gulp-util (node_module...s/gulp-util/index.js)     1.1s  ▇ 2%
50  gulp-svg2ttf (node_mod...svg2ttf/src/index.js)     2.1s  ▇▇ 3%
51  gulp-util (node_module...s/gulp-util/index.js)     1.3s  ▇ 2%
52  gulp-ttf2eot (node_mod...ttf2eot/src/index.js)     1.7s  ▇▇ 2%
53  gulp-util (node_module...s/gulp-util/index.js)     1.7s  ▇▇ 2%
54  gulp-ttf2woff (node_mo...tf2woff/src/index.js)     2.4s  ▇▇ 3%
55  gulp-iconfont (node_mo...confont/src/index.js)    11.9s  ▇▇▇▇▇▇▇▇ 17%
56  /Users/kud/Projects/_p...(gulp/glyphicons.js)     12.1s  ▇▇▇▇▇▇▇▇▇ 18%
57  gulp-util (node_module...s/gulp-util/index.js)     1.8s  ▇▇ 3%
58  gulp-plumber (node_mod...ulp-plumber/index.js)     2.1s  ▇▇ 3%
59  lodash.template (node_...sh.template/index.js)    719ms  ▇ 1%
60  ./lib/template (node_m...util/lib/template.js)    722ms  ▇ 1%
61  gulp-util (node_module...s/gulp-util/index.js)     1.6s  ▇▇ 2%
62  ./lib/math.js (node_mo...s/mathjs/lib/math.js)     2.3s  ▇▇ 3%
63  mathjs (node_modules/g...ules/mathjs/index.js)     2.3s  ▇▇ 3%
64  ./lib/format-svg-data...b/format-svg-data.js)        3s  ▇▇ 4%
65  gulp-svg-symbols (node...svg-symbols/index.js)     5.4s  ▇▇▇▇ 8%
66  gulp-util (node_module...s/gulp-util/index.js)       1s  ▇ 2%
67  gulp-filter (node_modu...gulp-filter/index.js)     1.5s  ▇ 2%
68  /Users/kud/Projects/_p...ulp/svg (gulp/svg.js)       9s  ▇▇▇▇▇▇ 13%
69  ./features (node_modul...myth/lib/features.js)     1.3s  ▇ 2%
70  myth (node_modules/myth/lib/index.js)              1.4s  ▇ 2%
71  gulp-myth (node_modules/gulp-myth/index.js)        2.1s  ▇▇ 3%
72  gulp-util (node_module...s/gulp-util/index.js)     1.2s  ▇ 2%
73  gulp-minify-css (node_...-minify-css/index.js)     1.7s  ▇▇ 3%
74  gulp-util (node_module...s/gulp-util/index.js)    695ms  ▇ 1%
75  gulp-rev (node_modules/gulp-rev/index.js)         731ms  ▇ 1%
76  /Users/kud/Projects/_p...yles (gulp/styles.js)     4.6s  ▇▇▇▇ 7%
77  gulp-util (node_module...s/gulp-util/index.js)     1.8s  ▇▇ 3%
78  jshint/src/cli (node_m...es/jshint/src/cli.js)    711ms  ▇ 1%
79  ./extract (node_module...shint/src/extract.js)    877ms  ▇ 1%
80  gulp-jshint (node_modu...-jshint/src/index.js)     2.9s  ▇▇ 4%
81  /Users/kud/Projects/_p...hint (gulp/jshint.js)     2.9s  ▇▇ 4%
82  gulp-util (node_module...s/gulp-util/index.js)     1.4s  ▇ 2%
83  gulp-concat (node_modu...gulp-concat/index.js)     1.9s  ▇▇ 3%
84  gulp-uglify (node_modu...gulp-uglify/index.js)    915ms  ▇ 1%
85  /Users/kud/Projects/_p...pts (gulp/scripts.js)     2.8s  ▇▇ 4%
86  gulp-util (node_module...s/gulp-util/index.js)       1s  ▇ 2%
87  gulp-handlebars (node_...-handlebars/index.js)     1.2s  ▇ 2%
88  gulp-util (node_module...s/gulp-util/index.js)    983ms  ▇ 1%
89  gulp-define-module (no...fine-module/index.js)     1.1s  ▇ 2%
90  /Users/kud/Projects/_p...s (gulp/templates.js)       3s  ▇▇▇ 4%
91  /Users/kud/Projects/_p...file.js (gulpfile.js)  1m 1.5s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 89%
Total require(): 6755
Total time: 1m 8.9s

@sindresorhus
Copy link
Contributor

Lol, there's something really wrong with your computer. It seems like everything takes 60x the time it should.

gulp (node_modules/gulp/index.js) 4.7s

On my computer:

gulp (node_modules/gulp/index.js) 84ms

@kud
Copy link
Author

kud commented Aug 22, 2014

My computer needs more ram and a SSD. :)

screen shot 2014-08-22 at 15 51 38

@yocontra
Copy link
Member

yocontra commented Sep 1, 2014

Outside the scope of gulp, your computer is loading modules really slow. Closing.

@yocontra yocontra closed this as completed Sep 1, 2014
@kud
Copy link
Author

kud commented Sep 1, 2014

I think I've found the problem.

As loading gulp modules ask lots of require, it does some I/O, and as I'm swapping at the same time, the hard disk hurts a lot. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants