Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl IP/stop does not actually terminate Locust swarms #1661

Closed
freelerobot opened this issue Dec 15, 2020 · 7 comments
Closed

curl IP/stop does not actually terminate Locust swarms #1661

freelerobot opened this issue Dec 15, 2020 · 7 comments
Labels
bug non-critical stale Issue had no activity. Might still be worth fixing, but dont expect someone else to fix it

Comments

@freelerobot
Copy link

freelerobot commented Dec 15, 2020

Expected
Per suggestion in #831, I have a script that starts Locust through a POST request, then terminates Locust through a GET /stop.

There are two issues with this:

  1. Depending on the spawn_rate, calling GET /stop only terminates httpusers that are already spawned. I expected this would terminate the entire test's users.
  2. If you initiate a swarm with a HTTP request, the UI doesn't have a "STOP" button. The only way to terminate that swarm is with a GET /stop.

To repro

  1. curl -XPOST -d "user_count=100&spawn_rate=10" http://$LOADGEN_IP/swarm where user_count/spawn_rate are our custom fields
  2. sleep for 5 seconds, then curl http://$LOADGEN_IP/stop
  3. Notice, after running step 2 UI indicates STATUS: STOPPED, and HTTPResponse is successful:
{
  "message": "Test stopped",
  "success": true
}
  1. However, according to our reliable telemetry, Locust is still swarming

Environment

  • Locust version: 1.3.1
@cyberw
Copy link
Collaborator

cyberw commented Dec 16, 2020

Interesting. It is possible that this issue arose from making the stopping of user greenlets async a while back.

I think the best way to fix this is to stop the spawning greenlet from runner.stop() before actually stopping user greenlets. Do you agree @heyman ?

@heyman
Copy link
Member

heyman commented Jan 21, 2021

I don't have that code fresh in my head (and at the moment don't have time to start digging), but that sounds reasonable to me @cyberw

We should be able to write a test for this I think.

@cyberw
Copy link
Collaborator

cyberw commented Jan 21, 2021

I'll have a look some time soon.

@mboutet
Copy link
Contributor

mboutet commented Jan 21, 2021

From my understanding of the codebase, I believe that these problems could be alleviated using synchronisation primitives offered by gevent (e.g. Synchronized queues, Locking primitives). For instance, some events such as start, stop, spawn events could be queued up and locking primitives could be used to ensure that only one of those mutually exclusive events (start and stop are two exclusive events IMO) is happening at a given time. Right now, these greenlets all run concurrently without real knowledge of each others resulting in flaky situations like the one in this issue.

@cyberw
Copy link
Collaborator

cyberw commented Mar 15, 2021

I wont have time to look at this, I think... Some else is welcome to try!

@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions bot added the stale Issue had no activity. Might still be worth fixing, but dont expect someone else to fix it label May 15, 2021
@github-actions
Copy link

This issue was closed because it has been stalled for 10 days with no activity. This does not necessarily mean that the issue is bad, but it most likely means that nobody is willing to take the time to fix it. If you have found Locust useful, then consider contributing a fix yourself!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug non-critical stale Issue had no activity. Might still be worth fixing, but dont expect someone else to fix it
Projects
None yet
Development

No branches or pull requests

4 participants