Custom Request/sec exit code

[odannyc]

Hello,

I'm running Locust on my CI server (jenkins) and would like to know if there is any way to specify when my test is fail or pass.
For example if my test does less than 100req/sec I want it to fail.

Is this possible?

[cgoldberg]

is any way to specify when my test is fail or pass

Locust returns an exit code of 1 when any failed requests were reported.

For example if my test does less than 100req/sec I want it to fail.

Exit status code base on throughput (rps) has not been implemented.

[cyberw]

I'm going to reopen this. Not saying I'm going to build it, but I think it would be neat to have, and I think it is a very common use case in CI. Throughput is the most important metric, but response times could be very useful as well.

[heyman]

I think this makes sense, and I think we should provide a hook for letting end-users set the exit code from within their test scripts.

Maybe we could add a process_exit_code attribute to Environment. Something like this maybe:

# locustfile.py
from locust import events

@events.init.add_listener
def _(self, environment, **kw):
    environment.process_exit_code = lambda env: env.stats.total.total_rps < 100 and 1 or 0

Or we could change so that if a listener for the quitting event returns True or False, it'll set the exit code to 0/1 (and ignore the default behaviour):

# locustfile.py
from locust.import events

@events.quitting.add_listener
def _(self, environment, **kw):
    return environment.stats.total.total_rps >= 100

Though I'm not sure how we would handle multiple event listeners returning different bools, and it would also require us to rework the event system to support events that return values.

[cyberw]

The process_exit_code attribute on environment seems more flexible. This, in combination with a good example should be enough (so no need for a specific parameter or anything).

Maybe we could even remove the —exit-code-on-failure or whatever it is called.

[heyman]

The process_exit_code attribute on environment seems more flexible. This, in combination with a good example should be enough (so no need for a specific parameter or anything).

Yeah. I guess we could even let it be either a callable (in which case it could be used like my first example in the previous post) or just an integer in which case one could also set it in a listener for the quitting event like this:

from locust import events

@events.quitting.add_listener
def _(environment, **kw):
    environment.process_exit_code = environment.stats.total.total_rps < 100 and 1 or 0

[cyberw]

Setting an integer should be fine. I dont see any case where you’d need to set it at any other time than during runtime or at ”quitting time” (and quit time would be pretty much the same time as the callable gets run anyway)

[heyman]

Setting an integer should be fine. I dont see any case where you’d need to set it at any other time than during runtime or at ”quitting time” (and quit time would be pretty much the same time as the callable gets run anyway)

True. I made a PR for this feature: #1396