Support Step Load Pattern (up & down)

[delulu]

Locust currently only support constant load pattern, that is to specify a user load that does not change during the load testing.

But in most scenarios of load testing, we would like to increase the user load by step, so we can monitor that how server's performance changes with different user load, and find out the max rps that server can handle without significantly performance drop.

Also Step Load pattern is supported in other popular test framework like JMeter, Visual Studio Load testing. I would like that Locust can also support load testing in Step Load pattern and I believe with this new feature Locust can reach a bigger market.

I've already make code changes for this new feature, but I would be glad to make any changes that are good, so free to let me know what you're thinking.

Expected behavior

Actual behavior

[delulu]

@cgoldberg please help mark it as a new feature, thx!

[delulu]

BTW, is there any regular meeting for new feature review?

[delulu]

@heyman for awareness.

[cgoldberg]

I'll review #1002

[vemedara]

I'll review #1002

Any update on general availability of step load pattern support with locust?

[cmendesce]

I would like to add a scenario to this request.

The step pattern is just one specific case of load pattern/profile. There are many load patterns available in academia and technical blogs. Some of them can be found in section 2.2 here.

An engineer would like to know how his system behaves in different scenarios with frequent increase and decrease users, so apply those scenarios would be a nightmare. I wrote a simple script to dynamically change users in locust. It calls the rest API adding/removing users according to a JSON file like this:

{
  "name": "simple scenario",
  "steps": [
  {
    "time": 0,
    "users": 200,
    "rate": 100
  },
  {
    "time": 30,
    "users": 400,
    "rate": 100
  },
  {
    "time": 60,
    "users": 0,
    "rate": 1
  }]
}

What do you guys think about to include it together with this feature?

[cgoldberg]

What do you guys think about to include it together with this feature?

are you planning on submitting a PR?

[mshegolev]

It will be very useful.

[vemedara]

It's a great idea, this will be really useful.

[cmendesce]

I'm planning to submit the code. Is there a date to this PR get merged? If not, I can use this code to start it.

[heyman]

A few thoughts:

I'm not very keen on changing the default UI for starting a test. I'd prefer if we instead had a link/button that would change the dialog to step/scenario mode.

I do think @cmendesce has a point in that we should support more scenarios than just stepping up.

We should also have support for this when running Locust in headless mode (with the --no-web option). Perhaps something like this:

locust --no-web -r 50 -c "1000:10m, 5000:1h, 3000:5m, 1000:5m, 0"

Which would mean:

• Spawn 1000 users and run for 10 minutes
• Increase to 5000 users and run for 1hour
• Decrease to 3000 users and run for 5 minutes
• Decrease to 1000 users and run for 5 minutes
• Kill all users

If we used the above format (though there might be scenarios that it wouldn't support) one could perhaps enter a string in the same format in the "Number of users to simulate" input box

[delulu]

@heyman Thank you for your feedback.

Actually it will change the default UI only with --step-load enabled. I prefer not to change current locust way, so all these new changes will only take effect when --step-load is enabled.

As for the webui change, I'm not a ui expert and if you think it's necessary to have a link/button to change the dialog to step/scenario mode, please go ahead, I'm Okay on it.

And for the support of more flexible scenarios, it sounds good and the format looks reasonable. But I'm afraid to make it too complex and customer has to consider more when running a load test.

So I refer to the step load pattern supported in Visual Studio Load testing, and simplify the design.

I think it's good to split the concerns, if you want to support more flexible scenarios maybe it's more feasible to keep the step load pattern, add a new load pattern for more complex scenarios.

[cyberw]

Fixed in #1002 (at least ramp UP)