[Heartbeat][Browser] Heartbeat as a Load Tester

[BenB196]

Describe the enhancement:

Note: I recognize that this request is outside of the scope of monitoring and probably outside the scope of what Heartbeat was probably ever considered for, so feel free to close if it isn't something that will be implemented.

---

I'd like to request the ability to configure Heartbeat with the browser monitor to act as a load tester for applications.

With Heartbeat supporting browser (synthetic) monitoring I thought that if I already have tests written to monitor the functionality of my apps, why not use these same tests to put load on these apps for load testing.

To better explain the concept below is rough idea of what I think the config for it could be:

heartbeat.run_once: true # I think you'd want Heartbeat to not continuously run, so setting this would probably be ideal.
heartbeat.monitors:
- type: browser
  id: load-tester
  name: Load Tester
  schedule: '@every 1m'
  source: #...
  params: #...
  screenshots: #...
  #...
  load_testing:
    instances: 100 # The number of `instance` tests heartbeat will spin up for load testing
    ramp: # How to spin up the `instances` used in the load testing
      duration: 100s # The duration that all `instances` should be running within
      distribution: 'even'/`random` # How the `instances` are started within the `duration` time.
                                # `even` Would be `instances`/`ramp.duration`, in this example 1 instance per second.
                                # `random` Heartbeat would randomly choose when to start instance as long as...
                                # they are all started within `ramp.duration` window.
    reporting: # Determines how much data is sent back to Elastic from the tests.
      failures: true/false # If true, all failed `instances` report back their data.
                           # If false, failed `instances` follow regular reporting logic.
      instances: 10/'5%' # How many instances should report back their collected information.
                         # Can either be a hard number (10) or a percentage (5%)
      distribution: 'random'/'duration'/'percentiles' # Determines how heartbeat will select which instance data to send back.
                                                      # `random` Heartbeat chooses instance at random
                                                      # `duration` Hearbeat chooses instances with the highest durations
                                                      # `percentile` Heartbeat chooses instances with durations...
                                                      # within a set of percentile ranges
      percentiles: ['99','95','75','50','25'] # List of percentile ranges heartbeat will use for `percentile` distribution.

Describe a specific use case for the enhancement or feature:

I have apps which I monitor with Heartbeat browser. I want to use those same tests to load test the apps, and have Heartbeat collect and send the information back to Elastic for analysis.

[elasticmachine]

Pinging @elastic/uptime (Team:Uptime)

[drewpost]

Hi @BenB196. I'm the PM for our Synthetics and RUM products at Elastic. First off, this is totally cool. I agree that it's a pretty natural progression from having a synthetic agent perform regular testing runs to ramping that up to perform load testing and building the bespoke reporting around it.

It's not something on our near term roadmap as we're currently getting flat-out building a brand new synthetics app with managed testing nodes, point and click script recording etc that will be launching this year. That being said, I certainly think there's place for load testing capabilities in the medium term roadmap.

I'd love to know a bit more about your use case. Do you see this as something run manually on an ad-hoc basis, something you run with every build as part of a CI/CD pipeline or something else? Feel free to reply on here or to ping my email [email protected] and I can pepper you with more questions!

[BenB196]

Hi @drewpost,

I think the use case mainly aligns with what I discussed in #25972. Where I see a good amount of value, allowing for the abstraction of using playwright with Heartbeat to get information into Elastic for easier visualization + analytics.

Though I really think it could actually be expanded to a few additional use cases.

1. CI/CD pipelines. Run a load test as part of a test or suite of tests that can be used to gauge the performance of a change. Here the developer would be able to see the test information within the tests time range (plus other filters) so that they can not only see if their change had a performance impact, but where exactly in the frontend it happens. (i.e.: Did the addition or upgrade of a JS library cause issues, did someone upload an image that has been optimized and takes a long time to download, etc...)
2. Ad-hoc. Sometimes for special events, sites get informed that a large traffic load is expected, so management wants to validate ahead of time that the site can handle the load. This case, you could run the load test with the agent, but also provide a report to management/developers to show what type of performance they can expect for the event, and whether things need to be changed for it to meet expectations.
3. Scheduled Tasks. (This kind of falls into CI/CD depending on how you look at it). (i.e.: A nightly load tests) can be run on a system to regularly to gather statistical data regarding the performance of a system under load, which might not have been caught in a system change (i.e.: CI/CD pipeline), or a change in a downstream dependency (i.e.: another system's API). You could use something like Elastic's ML to detect these anomalies as the tasks should occur on a fairly regular cadence.

All the above use cases I think would also work well alongside both RUM and APM to provide a great in-depth way for developers to see how their changes affect performance, as well as provide a platform that could be used to build reports and other tools to show improvement of systems, as well as investigate problems.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[BenB196]

This is still a neat feature request (I think)

[andrewvc]

I agree with you @BenB196 , but I think it's best we close it for now. We've got a deep backlog of stuff we're working on, and I don't see it as fitting in with our near or medium term plans. If that changes we can always open a new issue.

Thanks for the thoughtful contribution as always however!