-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run all microbenchmarks #154374
Run all microbenchmarks #154374
Conversation
Mac+iOS
Mac+Android
|
Flakes in flutter#153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it. Potential solution flutter#153828
1a353e6
to
7e958be
Compare
Does this README need an update? https://github.com/flutter/flutter/blob/master/dev/benchmarks/microbenchmarks/README.md It'll be interesting to see whether there are shifts in the metrics in the benchmarks that come later in the sequence. That would indicate that we still have some post-install or first-run warmup happening somewhere. |
I'll update it. I ran the benchmarks against a mokey and an iphone se and they were comperable to skia perf numbers. |
Done. |
Reverts: flutter#154374 Initiated by: jtmcdole Reason for reverting: A different benchmark was using the microbenchmark parser and timing out. Original PR Author: jtmcdole Reviewed By: {zanderso} This change reverts the following previous change: Two things: **Re-land**: Uninstall microbenchmarks before running them. Flakes in flutter#153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys. Potential solution flutter#153828 **Make things go fast** Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
**Three things** Re-lands flutter#154374 New: fix `platform_channels_benchmarks` to print the "done" key. Updated notes for the microbenchmark parser. There are no other users of `microbenchmarks.readJsonResults`. Re-Re-land: Uninstall microbenchmarks before running them. Flakes in flutter#153828 stem from adb saying the app isn't installed, but then failing to install wtih -r. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys. Potential solution flutter#153828 Re-land Make things go fast Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
Two things:
Re-land: Uninstall microbenchmarks before running them.
Flakes in #153828 stem from adb saying the app isn't installed, but then failing to install wtih
-r
. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys.Potential solution #153828
Make things go fast
Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.