⚗ performance impact summary tool

[BenoitZugmeyer]

Motivation

Evaluate the Browser SDK performance impact on a given scenario

Changes

Implement a tool to run various profilings with puppeteer. You can try it with:

Testing

git checkout benoit/performance-doc
yarn
cd performances
yarn start

Example output:

# Wikipedia

Illustrates a mostly static site scenario.

* Navigate on three Wikipedia articles
* Do a search (with dynamic autocompletion) and go the first result

Memory (median): 294 kB (4.83%)
CPU: 258 ms (4.24%)
Bandwidth:
  upload: 116 kB
  download: 20.5 kB


# Twitter

Illustrates a SPA scenario.

* Navigate to the top trending topics
* Click on the first trending topic
* Click on Top, Latest, People, Photos and Videos tabs
* Navigate to the Settings page
* Click on a few checkboxes

Memory (median): 225 kB (0.51%)
CPU: 391 ms (1.01%)
Bandwidth:
  upload: 186 kB
  download: 20.5 kB

From my observations, results are quite stable in general.

---

I have gone over the contributing documentation.

[ThibautGeriz]

Regarding the stat I would add more than the median like 95 and 99 percentile.
The CPU is not super clear:

CPU: 146,665 microseconds (6.455%)

Is it microsecond per something? seconds? session?

It would be super cool to add this to graph that to see the evolution in time.

[BenoitZugmeyer]

Regarding the stat I would add more than the median like 95 and 99 percentile.

Maybe p75 would make sense, but 95 or 99 would just show the max value, because we don't take that many measurements.

The CPU is not super clear:

CPU: 146,665 microseconds (6.455%)

Is it microsecond per something? seconds? session?

It is plain microseconds. How much CPU time Browser SDK-located code has taken during the scenario.

It would be super cool to add this to graph that to see the evolution in time.

Would be cool indeed :)

[webNeat]

I am not sure if it would be statistically better to run takeMeasurements periodically (like every 100 ms or so) instead of triggering it manually during the scenario. WDYT?

[BenoitZugmeyer]

I don't think it would matter too much. I like controling when to take measurements, when the page is idle after doing something. If we didn't:

• measurements could be taken while the page is loading, when the SDK is not yet loaded, yielding not applicable results for our use-case
• measurements could be taken while some JS is running, and may impact the CPU profiling.

[ThibautGeriz]

It is plain microseconds. How much CPU time Browser SDK-located code has taken during the scenario.

Cool then I would add the time it took to run the scenario as 146,665 microseconds or upload: 162,482 bytes is not the same if the scenario lasted 1h ou 2.5 seconds.

[BenoitZugmeyer]

It is plain microseconds. How much CPU time Browser SDK-located code has taken during the scenario.

Cool then I would add the time it took to run the scenario as 146,665 microseconds or upload: 162,482 bytes is not the same if the scenario lasted 1h ou 2.5 seconds.

Well, it could be the same if the scenario lasts 1h or 2.5 seconds. The impact is not proportional to the scenario duration, but more on which actions are being done during it (ex: number of new views, resources loaded, etc.) The scenario duration would not be indicative of real world SDK usage, because no human will read 5 Wikipedia articles in 2 seconds :)

[bcaudan]

Nice 👍

[webNeat]

Good job!