Snapshot startup is too slow

[nex3]

In my tests on pub, it takes on the order of 100ms for the main() to begin executing when running from a snapshot. This is a very long time for just startup, especially since it's a snapshot and as such there's no additional parsing that needs to occur. As we're trying to get the runtime of "pub run" down, this will become more and more of an issue.

[iposva-google]

The pub snapshot is about 3MB and contains the sources for at least pub, barback, and dart2js. Siva can you take a look where we spend the time here?

[munificent]

The pub snapshot is about 3MB and contains the sources for at least pub, barback, and dart2js.

Barback is relatively tiny, but dart2js (and the analyzer) are not. To get some more data, I built and ran a few variations of pub with some things stripped out. This is all on my Mac laptop with a ReleaseIA32 build.

For timing, I hacked pub to immediately return from main(), then ran "time pub" and took the best of a few runs.

The normal pub snapshot is 3,107,592 bytes and runs in 0.101s.
If I take dart2js out, the snapshot is 1,689,526 bytes and runs in 0.087s.
If I take the analyzer out, the snapshot is 766,628 bytes and runs in 0.075s.
If I take everything out, and just make pub a hello world, the snapshot is 1,904 bytes (:D) and runs in 0.057s.

For comparison, a hello world C program runs in 0.003s on my machine.

While 0.05s seems pretty fast, it leaves little room for a Dart program to provide some initial output to let the user know it's running and doing something. Usability literature[1] says we have about 0.1s to pull that off before the user starts to feel like they aren't in direct control of the system.

Of course, we have a lot of work to do on our end to get pub more responsive too!

[iposva-google]

Set owner to @a-siva.
Added Accepted label.

[a-siva]

Could you post the command line used to measure the 100ms startup time for pub.

[munificent]

1. Change main() in sdk/lib/_internal/pub/bin/pub.dart to return immediately.
2. Build the SDK.
3. Run: time dart <build dir>/dart-sdk/bin/snapshots/pub.dart.snapshot

[a-siva]

Additional step needed now :
1.5 run the "pub" shell script in sdk/bin

[zanderso]

It looks like the pub snapshot has grown to ~4.8MB which likely increases loading time.

[rmacnak-google]

Stale. App-jit and app-aot snapshots are recommended where startup time is a concern.

https://github.com/dart-lang/sdk/wiki/Snapshots

[nex3]

@rmacnak-google Do app-aot snapshots have competitive runtime performance with code snapshots, or do they lose out because they don't get JITed? If we have to choose between slow startup and slow runtime, our problem isn't really solved.

[zoechi]

According to https://github.com/dart-lang/sdk/wiki/Snapshots snapshots get a training run with --snapshot-kind=app-jit

[nex3]

That's true for app-jit snapshots, but I'm not sure how you'd construct an effective training run for an application like pub that has many different commands that all execute different code paths. And a training run just isn't possible in general for some of our snapshot use-cases, such as generating snapshots of user executables for use by pub run.

[zanderso]

Even very simple training runs manage to pick up a lot of start up code, which allows for fast startup when running from the app-jit snapshot.

When running from an app-jit snapshot, JIT compilation and optimization are still possible, which means we still get good peak performance.

[nex3]

But what about situations (like pub run) where the executables are total black boxes, and we don't know any way to safely invoke them to do a training run?

[zanderso]

I think I might not be completely understanding your concern. However, you may be underestimating the amount of code that we have to compile and run just to get to main() and do e.g. command line argument parsing and printing some strings. Getting all of that code compiled in the app-jit snapshot has proved to be very beneficial.

Either way, it's certainly possible to come up with application-appropriate training runs by modifying e.g.: https://github.com/dart-lang/sdk/blob/master/utils/pub/BUILD.gn#L7 but you have to keep in mind that there is a trade-off with size. As you add more code to the snapshot, there will be diminishing returns to startup performance.