Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

locust should exit when a load shape returns None in headless mode #1653

Closed
Apteryks opened this issue Dec 10, 2020 · 6 comments · Fixed by #1683
Closed

locust should exit when a load shape returns None in headless mode #1653

Apteryks opened this issue Dec 10, 2020 · 6 comments · Fixed by #1683
Labels
bug

Comments

@Apteryks
Copy link

Apteryks commented Dec 10, 2020 •
edited
Loading

Hello!

Describe the bug

When using a LoadTestShape class, the test finishes when its tick method returns None, but the Locust process keeps running, even when in --headless mode.

Expected behavior

When Locust is run in --headless mode, it should automatically stop running when a LoadTestShape class tick method signals a test is complete by returning None.

Actual behavior

The test is finished, but the Locust process does not exit.

Steps to reproduce

I'm using Docker to run Locust.

file: repro.py

import logging

from locust import HttpUser, LoadTestShape, task


# Straight from:
# https://docs.locust.io/en/latest/generating-custom-load-shape.html?highlight=LoadTestShape
class MyCustomShape(LoadTestShape):
    time_limit = 20
    spawn_rate = 2

    def tick(self):
        run_time = self.get_run_time()

        if run_time < self.time_limit:
            # User count rounded to nearest hundred.
            user_count = round(run_time, -2)
            return (user_count, self.spawn_rate)

        return None


class User(HttpUser):

    @task
    def main(self):
        logging.info('Running main task...')

Then run the reproducer like:

docker run -v "$PWD:/mnt/locust" -w "/mnt/locust" locustio/locust -f repro.py --headless --host http://www.example.com --only-summary
[2020-12-10 16:19:21,864] 39c75587a12e/WARNING/locust.main: System open file limit '1024' is below minimum setting '10000'. It's not high enough for load testing, and the OS didn't allow locust to increase it by itself. See https://github.com/locustio/locust/wiki/Installation#increasing-maximum-number-of-open-files-limit for more info.
[2020-12-10 16:19:21,865] 39c75587a12e/INFO/locust.runners: Shape test starting. User count and spawn rate are ignored for this type of load test
[2020-12-10 16:19:21,865] 39c75587a12e/INFO/locust.main: No run time limit set, use CTRL+C to interrupt.
[2020-12-10 16:19:21,865] 39c75587a12e/INFO/locust.main: Starting Locust 1.4.1
[2020-12-10 16:19:21,866] 39c75587a12e/INFO/locust.runners: Shape worker starting
[2020-12-10 16:19:21,866] 39c75587a12e/INFO/locust.runners: Shape test updating to 0 users at 2.00 spawn rate
[2020-12-10 16:19:21,866] 39c75587a12e/INFO/root: Terminal was not a tty. Keyboard input disabled
[2020-12-10 16:19:21,866] 39c75587a12e/INFO/locust.runners: Spawning 0 users at the rate 2 users/s (0 users already running)...
[2020-12-10 16:19:21,867] 39c75587a12e/INFO/locust.runners: All users spawned: User: 0 (0 total running)
[2020-12-10 16:19:41,887] 39c75587a12e/INFO/locust.runners: Shape test stopping

Observe that Locust does not return control to the shell and keeps running.

Environment

Dockerized. Everything needed to reproduce is described above.
@Apteryks Apteryks added the bug label Dec 10, 2020
@Apteryks Apteryks changed the title locust should stop when a load shape returns Node in headless mode locust should exit when a load shape returns Node in headless mode Dec 10, 2020
@Apteryks Apteryks changed the title locust should exit when a load shape returns Node in headless mode locust should exit when a load shape returns None in headless mode Dec 10, 2020
@cyberw
Copy link
Collaborator

cyberw commented Dec 10, 2020

Your setup doesnt actually spawn any users? time_limit needs to be more than 100. Does it work if you change that?

Sounds like might have both have a documentation issue as well as an issue with shape tests not stopping if they never actually spawned any users. @max-rocket-internet You got time to take a look?

@Apteryks
Copy link
Author

Hi @cyberw! Even by changing the to 600 (the actual value from the example), it still won't exit:

time docker run -v "$PWD:/mnt/locust" -w "/mnt/locust" locustio/locust -f repro.py --headless --host http://www.example.com --only-summary
[...]
[2020-12-11 03:29:53,863] 1c4c6d090d1d/INFO/root: Running main task...
[2020-12-11 03:29:53,863] 1c4c6d090d1d/INFO/root: Running main task...
[2020-12-11 03:29:53,863] 1c4c6d090d1d/INFO/root: Running main task...
[2020-12-11 03:29:53,873] 1c4c6d090d1d/INFO/locust.runners: 600 Users have been stopped, 0 still running
  C-c C-cKeyboardInterrupt
2020-12-11T03:38:30Z
[2020-12-11 03:38:30,950] 1c4c6d090d1d/INFO/locust.main: Running teardowns...
[2020-12-11 03:38:30,951] 1c4c6d090d1d/INFO/locust.main: Shutting down (exit code 0), bye.
[2020-12-11 03:38:30,951] 1c4c6d090d1d/INFO/locust.main: Cleaning up runner...
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

Response time percentiles (approximated)
 Type     Name                                                              50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|



real	18m54.950s
user	0m4.398s
sys	0m4.471s

@cyberw
Copy link
Collaborator

cyberw commented Dec 11, 2020

Ok, then my hunch was incorrect. Can you run docker run locustio/locust -V and check that you are on the latest version? (1.4.1)

@Apteryks
Copy link
Author

@cyberw yes, I confirm:

$ docker run locustio/locust -V
locust 1.4.1

@cyberw
Copy link
Collaborator

cyberw commented Dec 14, 2020 •
edited
Loading

I can reproduce this as well (on windows of all things, but I doubt that is relevant :)

@max-rocket-internet can you take a look? You can see the Shape test stopping logging happening but it doesnt shut down until I CTRL+C.

locust -f repro.py --headless
[2020-12-14 19:59:16,286] DESKTOP-TGHVUHE/INFO/locust.runners: Shape test starting. User count and spawn rate are ignored for this type of load test
[2020-12-14 19:59:16,287] DESKTOP-TGHVUHE/INFO/locust.main: No run time limit set, use CTRL+C to interrupt.
[2020-12-14 19:59:16,287] DESKTOP-TGHVUHE/INFO/locust.main: Starting Locust 1.4.1
[2020-12-14 19:59:16,287] DESKTOP-TGHVUHE/INFO/locust.runners: Shape worker starting
[2020-12-14 19:59:16,287] DESKTOP-TGHVUHE/INFO/locust.runners: Shape test updating to 0 users at 2.00 spawn rate
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

[2020-12-14 19:59:16,288] DESKTOP-TGHVUHE/INFO/locust.runners: Spawning 0 users at the rate 2 users/s (0 users already running)...
[2020-12-14 19:59:16,288] DESKTOP-TGHVUHE/INFO/locust.runners: All users spawned: User: 0 (0 total running)
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

[2020-12-14 19:59:36,475] DESKTOP-TGHVUHE/INFO/locust.runners: Shape test stopping
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\users\larsh\appdata\local\programs\python\python39\lib\site-packages\gevent\_ffi\loop.py", line 269, in python_check_callback
    def python_check_callback(self, watcher_ptr): # pylint:disable=unused-argument
KeyboardInterrupt
2020-12-14T18:59:59Z
[2020-12-14 19:59:59,409] DESKTOP-TGHVUHE/INFO/locust.main: Running teardowns...
[2020-12-14 19:59:59,412] DESKTOP-TGHVUHE/INFO/locust.main: Shutting down (exit code 0), bye.
[2020-12-14 19:59:59,413] DESKTOP-TGHVUHE/INFO/locust.main: Cleaning up runner...
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

Response time percentiles (approximated)
 Type     Name                                                              50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|

@max-rocket-internet
Copy link
Contributor

OK cool, I'll make a PR

@max-rocket-internet max-rocket-internet mentioned this issue Jan 20, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Shutdown workers when using LoadTestShape and headless mode
3 participants
@cyberw @Apteryks @max-rocket-internet