Locust does not stop all users

[marcinh]

Describe the bug

When there is more than 1 user defined in locust file, Locust does not stop users correctly - there are always 10%-20% left in running state. I'm adding the scenario, that causes problem - there is a background task that periodically checks the number of users and print sit to console.

Expected behavior

All users stopped correctly

Actual behavior

Not all users stopped

Steps to reproduce

Run scenario attached (containing 2 user classes)

Environment

• OS: Windows/MAC
• Python version: 3.9
• Locust version: 2.5
• Locust command line that you ran: locust -f bug_scenario.py
• Locust file contents (anonymized if necessary):

from logging import getLogger
from random import randrange

from gevent import sleep, spawn
from locust import task, constant_pacing, SequentialTaskSet, events, User
from locust.env import Environment

logger = getLogger('Scenario')


def add_request_event(request_type, name, duration, length=0, response=None, context=None, exception=None):
    events.request.fire(
        request_type=request_type,
        name=name,
        response_time=duration,
        response_length=length,
        response=response,
        context=context or {},
        exception=exception
    )


def run_environment_checker(environment: Environment):
    while True:
        user_count = environment.runner.user_count
        logger.info(f'User count: {user_count}')
        sleep(3)


@events.init.add_listener
def hadle_environment_checker(environment: Environment, **_):
    spawn(lambda: run_environment_checker(environment))


class TestScenario(SequentialTaskSet):

    def __init__(self, parent):
        super().__init__(parent)

    def on_start(self):
        logger.info('on_start')

    @task
    def open_homepage(self):
        add_request_event(
            request_type='PUT',
            name='3d',
            duration=randrange(1000),
            length=randrange(1000)
        )

    @task
    def open_dashboard(self):
        add_request_event(
            request_type='GET',
            name='2d',
            duration=randrange(1000),
            length=randrange(1000)
        )

    @task
    def get_open_positions(self):
        add_request_event(
            request_type='POST',
            name='1d',
            duration=randrange(1000),
            length=randrange(1000)
        )

    def on_stop(self):
        logger.info('on_stop')


class ExampleUser(User):
    tasks = [TestScenario]
    wait_time = constant_pacing(10)


class ExampleUser2(ExampleUser):
    pass

[cyberw]

Your test looks a little strange. Why are you calling randrange(1000) in on_stop? Also, TaskSets that dont call self.interrupt() will never exit, so you probably shouldnt use a TaskSet at all.

I think the problem is not actually that the users dont stop, but there is an issue with environment.runner.user_count - it sometimes returns the wrong value (particuarly noticeable when the ramp up finishes).

Please use a more basic User (no TaskSet) and print something in on_stop, so we can see what the problem is.

[marcinh]

So I updated on_start and on_stop method to log when they're called, and in this case on_stop is not called for all users and additionally after hitting stop button I can see increased number of request in WebUI.

This scenario is just simplification of scenario we are using for load test without any internal code that is not relevant. I don't get why don't calling self.interrupt() is anyhow related to this problem - the TaskSet will run until I stop it.

[cyberw]

Hmm... I dont know what could be causing this. Maybe the cause is somehow in your "internal" code? Sorry, I dont have much time to spend on this right now. If you can write a failing test case I will investigate further (I know that is asking a lot, but I'm doing this on my free time :)

[marcinh]

The scenario attached to this bug is causing the problem without any internal code - not sure what else I could write. With single user defined, everything seems to be ok. When we add second user, stopping is not working correctly.

[cyberw]

Sorry, I dont have time to investigate further. If you can express your problem in a unit test I will give it another look.

[cyberw]

Also, I did try running your locust file, adding an on_stop for the second user class, and then launching it like this: locust -u 2 -t 1 --headless -s 10. Both on_stop-methods were called for me.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[github-actions]

This issue was closed because it has been stalled for 10 days with no activity. This does not necessarily mean that the issue is bad, but it most likely means that nobody is willing to take the time to fix it. If you have found Locust useful, then consider contributing a fix yourself!

[marcinh]

Hi, sorry for late response. I finally got some time to investigate this a little bit more. So I created a unit test, to cover this case:

    def test_bug_1947(self):
        class BaseUser1(User):
            wait_time = constant(1)

            @task
            def task_a(self):
                sleep(0.1)

        class BaseUser2(BaseUser1):
            wait_time = constant(1)

        runner = Environment(user_classes=[BaseUser1, BaseUser2]).create_local_runner()
        runner.start(user_count=user_count, spawn_rate=4)

        sleep(5)

        self.assertEqual(user_count, runner.user_count)

        runner.stop()
        sleep(5)
        self.assertEqual(0, runner.user_count)

Every time there are 2 users left after stopping.
I think this has something to do with Users inheritance. Tomorrow I'll investigate it further

[marcinh]

I think problematic is line 253 in runners.py:

                if isinstance(user, self.user_classes_by_name[user_class]):
                    to_stop.append(user)

if we change it to:

                if isinstance(user, self.user_classes_by_name[user_class]) and user.__class__.__name__ == user_class:
                    to_stop.append(user)

or:

                if type(user) == self.user_classes_by_name[user_class]:
                    to_stop.append(user)

test starts passing