Ability to control the Locust process' exit code

[heyman]

PR for #587

[codecov]

Codecov Report

Merging #1396 into master will decrease coverage by 0.06%.
The diff coverage is 18.75%.

@@            Coverage Diff             @@
##           master    #1396      +/-   ##
==========================================
- Coverage   81.07%   81.00%   -0.07%     
==========================================
  Files          26       26              
  Lines        2335     2338       +3     
  Branches      361      362       +1     
==========================================
+ Hits         1893     1894       +1     
- Misses        349      351       +2     
  Partials       93       93              

Impacted Files	Coverage Δ
locust/event.py	90.00% <ø> (ø)
locust/main.py	18.84% <0.00%> (-0.19%)	⬇️
locust/env.py	96.55% <100.00%> (+0.06%)	⬆️
locust/runners.py	80.20% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3bbc669...48fd3f4. Read the comment docs.

[cyberw]

Looks good! Could you add an example on how to set the return code based on rps or avg response time as well? And maybe do a log entry in the example as well.

[heyman]

I've extended the example to also check for average response time as well as response time percentile.

I'm hesitant on adding an RPS check to the example because I think it's a slightly weird condition to use (since it's very dependent on both the number of simulated users as well the actual code of the User classes). It shouldn't be hard to figure out for anyone who still wants to do it, but I don't think it should be encouraged by an example in the official docs.

And maybe do a log entry in the example as well.

Good idea!

[cyberw]

Ok to merge! I still think an RPS check is a good idea, especially for CI purposes. Yes, RPS depends on your test plan, but that is constant over CI runs. There are cases where RPS might drop even though response times do not rise (e.g. delays reading test data, load gen overload etc) which you would definitely want to catch.

But if you are -1 on that, I can just comment it in the ticket (which was about RPS, just like the slack question that made me reopen it :)

[cyberw]

Lets worry about (possibly) adding rps thresholds to the documentation later. Merging.