Code

ci: benchmarking updates#1926

Code

Merged

armcknight merged 37 commits intomastergetsentry/sentry-cocoa:masterfrom armcknight/benchmark-raw-exportgetsentry/sentry-cocoa:armcknight/benchmark-raw-exportCopy head branch name to clipboardJul 13, 2022

+709-106

Commits

Commits on Jun 30, 2022

• ref: split benchmarking into separate target/action/sauce-config

  armcknight

  committed
  8af6508

  Copy full SHA for 8af6508

• fix: artifact key

  armcknight

  committed
  480bb49

  Copy full SHA for 480bb49

Commits on Jul 1, 2022

• fix: quote suite names with spaces

  armcknight

  committed
  bb32159

  Copy full SHA for bb32159

• fix: use custom config

  armcknight

  committed
  6cb60fa

  Copy full SHA for 6cb60fa

• fix: name of test runner app

  armcknight

  committed
  4d403af

  Copy full SHA for 4d403af

• WIP: run 20 trials

  armcknight

  committed
  f0cf67d

  Copy full SHA for f0cf67d

• TEST: figure out where to write the artifact in sauce labs

  armcknight

  committed
  03dd64d

  Copy full SHA for 03dd64d

• fixup! TEST: figure out where to write the artifact in sauce labs

  armcknight

  committed
  c0cdc13

  Copy full SHA for c0cdc13

Commits on Jul 6, 2022

• fix: report 20 trials in logs

  armcknight

  committed
  52c46f4

  Copy full SHA for 52c46f4

• fix: fixit for format

  armcknight

  committed
  5ead763

  Copy full SHA for 5ead763

• fix: increase whole-job timeout

  armcknight

  committed
  d83ab35

  Copy full SHA for d83ab35

• fix: increase timeout more, print raw values from benchmark, assert p90 upon completion of separate test cases

  armcknight

  committed
  0d1218f

  Copy full SHA for 0d1218f

• fix: shuffle raw results to test runner app for logging/analysis

  armcknight

  committed
  8cc7549

  Copy full SHA for 8cc7549

Commits on Jul 7, 2022

• ci: put back to 20 trials

  armcknight

  committed
  3db365f

  Copy full SHA for 3db365f

• fix: remove unconditional enablement of user interaction tracing that breaks benchmarks

  armcknight

  committed
  e8b2eb0

  Copy full SHA for e8b2eb0

• ref: simplify data marshalling, add script to compile report

  armcknight

  committed
  6794cf9

  Copy full SHA for 6794cf9

• ci: make each test case shorter; update docs

  armcknight

  committed
  7408e19

  Copy full SHA for 7408e19

• Format code

  getsentry-bot

  committed
  184bfff

  Copy full SHA for 184bfff

• ci: add tags to the saucectl invocations fro benchmarking

  armcknight

  committed
  a02a2cd

  Copy full SHA for a02a2cd

• fix: increase whole-job timeout to 200m

  armcknight

  committed
  518343f

  Copy full SHA for 518343f

• ref: use saucectl --retries arg instead of bash for loop

  armcknight

  committed
  88b021e

  Copy full SHA for 88b021e

• fix: some string parsing

  armcknight

  committed
  f265e03

  Copy full SHA for f265e03

• TRY: just run 1 test case with 5 iterations

  armcknight

  committed
  bb74bc2

  Copy full SHA for bb74bc2

Commits on Jul 8, 2022

• fix: upgrade to higher iOS versions for the selected devices

  armcknight

  committed
  98bf297

  Copy full SHA for 98bf297

• ref: remove unused xcode key from matrix entries

  armcknight

  committed
  bd273f2

  Copy full SHA for bd273f2

• fix: try 20 cases of 1 iteration each

  armcknight

  committed
  be3a3eb

  Copy full SHA for be3a3eb

• fix: remove device classes that never finish, put back to original configuration of 5 test cases running 4 iterations each

  armcknight

  committed
  59b3ae9

  Copy full SHA for 59b3ae9

Commits on Jul 11, 2022

• Update Samples/iOS-Swift/PerformanceBenchmarks/SentryProcessInfo.m

  Show description for ce67d78
  

  armcknight

  and

  philipphofmann

  authored
  ce67d78

  Copy full SHA for ce67d78

• Update .github/workflows/benchmarking.yml

  Show description for 68df84e
  

  armcknight

  and

  philipphofmann

  authored
  68df84e

  Copy full SHA for 68df84e

• ref: remove unneeded duplicate scheme

  armcknight

  committed
  4439031

  Copy full SHA for 4439031

• ci: put back to always run on PRs (and release branches)

  armcknight

  committed
  127b688

  Copy full SHA for 127b688

• ci: bring down timeout to 30 minutes (observing 24-28 minutes)

  armcknight

  committed
  9a2554b

  Copy full SHA for 9a2554b

• fix: link to qa article on proc flags; add proc flags init to sentry crash impl

  armcknight

  committed
  2285b8d

  Copy full SHA for 2285b8d

• Format code

  getsentry-bot

  committed
  62a245c

  Copy full SHA for 62a245c

Commits on Jul 12, 2022

• fix: disable ANR tracking during benchmarks

  armcknight

  committed
  d5f3c62

  Copy full SHA for d5f3c62

• docs: more explainers and cross references for test setup

  armcknight

  committed
  ee199e7

  Copy full SHA for ee199e7

• Format code

  getsentry-bot

  committed
  146d9c3

  Copy full SHA for 146d9c3