Provide network and CPU throttling

[hmdhk]

Currently synthetics tests results are not representative of the majority of users that visit a specific web application. To better represent a real world situation we should run the tests with CPU and network throttling enabled. We can run the tests multiple times with different CPU/network throttling profiles.

@paulb-elastic @drewpost, IMHO it's preferable to make a decision on throttling while still in the experimental phase since the generated report results will be affect by this.

[paulb-elastic]

I completely agree that this is so important.

From a CPU perspective, we need to ensure in the first instance that there is a consistent allocation of CPU resources available for the test to run. We have had some discussions internally about this, and it’s listed as a requirement for GA (#93), being able to provide a consistent CPU boundary for both on prem and within cloud.

The most important thing in the first instance is to ensure consistency of the resources available, which dovetails from protecting against noisy neighbours.

Future enhancements could potentially be implemented to simulate slower devices (like a device profile configuration against a monitor that simulates a slower device).

Network throttling is equally important and monitors should rarely be running at line speed (certainly not when we’re gathering performance data) as it can generate such variability and misleading results.

Monitor configurations should be used to define what network speed the test runs with, controlling upload speed, download speed and latency as a minimum set of variables.

In all these cases, there is a need to ensure that the platform has sufficient CPU and network bandwidth to deliver the total concurrent monitors being run from that shared infrastructure, to ensure that it does not create a bottleneck that impacts the performance metrics of the tests.

[drewpost]

100% agree on the importance of this from a product perspective.

Device simulation is my biggest use case. Both with synthetic monitors and also ad-hoc generated runs based on identified anomalous RUM cohorts.

We should also consider the UX of this @katrin-freihofner - From a user's perspective is it better to have mixed speeds within a single monitor or would we have one speed per monitor consistently? Also, how do we communicate when speed settings have changed when viewing historical data

[paulb-elastic]

I’d like to split this out to track and prioritise the features individually.

(I am excluding the platform consistency (CPU) from this as that’s a platform requirement for GA)

From the Synthetic Agent perspective, there are several things we need to throttle/emulate:

• UA String (to emulate a different device type, often combined with a change of the viewport)
• Viewport (to emulate a different device type, often combined with a change of the UA String)
• Network (to emulate more realistic, throttled connections such as DSL, 3G, 4G etc.)
• CPU (to emulate older devices)

Three separate issues have been raised to cover these as follows:

• Emulate device type (this covers both the UA String and Viewport as they are so closely related for setting the device type) Emulate device type (UA String and Viewport) #209
• Network throttling Network throttling #210
• CPU Throttling CPU Throttling #211