Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement dev live optimizer #4611

Merged
merged 57 commits into from
Aug 11, 2015
Merged

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Aug 7, 2015

Fixes: #4569
Fixes: #4554
Fixes: #4556
Fixes: #4555
Fixes: #4549

This pr offers the the following changes:

  • in dev mode, use version of the compiler that:
    • watches files for changes
    • only rebuilds when needed by delaying rebuilds until files are requested by the browser
  • only use babel to compile .babel.js files, dramatically shortening webpack build time
  • switch to prebuilt moment.js without locale support
  • out of dev mode, bundles are correctly reused
  • restored the loading screen from before Apps/App Switcher #4335

There are also a number of new config options available, each with the goal of giving users more control over how things are built.

  • optimize.bundleDir : the directory that bundles and related assets should be written.
    • default: '{kibanaRoot}/optimize/bundles'
  • optimize.bundleFilter : a glob that selects which bundles to build. Other bundles are ignored and might as well not exist
    • default: * unless optimize.tests is true
  • optimize.sourceMaps : Defines what type of source maps should be created.
    • default: true in dev, false otherwise
    • options:
      • true: produce high quality maps with line and column precision
      • false: disable source maps completely
      • "fast": make quick and dirty maps (currently alias for cheap-module-eval-source-map)
      • Anything that webpack's devTool option accepts
  • optimize.unsafeCache : Enables a caching mode that may cause the optimizer to not notice added or removed files. Using this option greatly improves build performance, but is opt-in so that users know to look out for invalid cache hits.
    • default: '/[\/\\](node_modules|bower_components)[\/\\]/'
    • options:
      • true
      • false
      • a string wrapped in / used to create a RegExp
  • optimize.lazyPrebuild : When starting up the dev server, run the optimizer once before initializing the server.
    • default: true
  • optimize.profile : Profile the build and upon successful completion write a stats.json file into the optimize.bundleDir that can be used with the webpack analyze tool to analyze build performance and contents
  • optimize.tests : Create the test bundle and nothing else, combine with optimize.bundleFilter to build the tests along with other bundles
    • default: false

The primary motivator for this change was the speed of the feedback loop. My testing offers the following stats:

Optimization times:

./bin/kibana

no cache~18s

./bin/kibana --optimize.bundleFilter=appSwitcher

no cache~8s

./bin/kibana --dev

initial~28s
refresh~7s

./bin/kibana --dev --optimize.unsafeCache=true

initial~25s
refresh~3s

./bin/kibana --dev --optimize.unsafeCache=true --optimize.sourceMaps=false

initial~18s
refresh~1s

./bin/kibana --dev --optimize.unsafeCache=true --optimize.sourceMaps=false --optimize.bundleFilter=appSwitcher

initial~8s
refresh~0.25s

Memory usage:

dev
coordinator~105 MB
server~150 MB
optimizer~550 MB
production with single optimization run
server~600 MB
production without optimization run
server~150 MB
production post #4597 without optimization [1]
server~80 MB

CPU Usage still climbs to 100% during optimization, but afterward the processing again idle at (or near) 0
image

[1] All threads are currently compiling the server's source code with babel at require-time which results in a slight performance penalty and memory overhead, but with #4597 we should be able to optimize this away.

spalger added 30 commits August 5, 2015 08:44
<form name="queryInput"


<from name="queryInput"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do with less newlines here...

@w33ble w33ble assigned spalger and unassigned w33ble Aug 11, 2015
spalger added a commit that referenced this pull request Aug 11, 2015
@spalger spalger merged commit 153e195 into elastic:master Aug 11, 2015
@spalger spalger deleted the implement/devLiveOptimizer branch February 25, 2016 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants