feat: run loadgen for a few seconds during installation #609
Conversation
google-cla
bot
added
the
cla: yes
freelerobot
added
the
do not merge
terraform/install.sh
Outdated
| TRIES=0 | ||
| while [[ $(curl -sL -w "%{http_code}" "http://$loadgen_ip" -o /dev/null --max-time 1) -ne 200 && \ | ||
| # while [[ $(curl -sL -w "%{http_code}" "http://$loadgen_ip" -o /dev/null --max-time 1) -ne 200 && \ | ||
| while [[ $(curl -XPOST -d "user_count=100&spawn_rate=10" http://$loadgen_ip/swarm -o /dev/null -w "%{http_code}" --max-time 1) -ne 200 && \ |
There was a problem hiding this comment.
is /swarm idempotent? Can we keep trying to hit it multiple times and always expect a 200 response?
There was a problem hiding this comment.
It is indempotent
| # Ends initial load generation | ||
| if [[ $(curl http://$loadgen_ip/stop -o /dev/null -w "%{http_code}") -ne 200 ]]; then | ||
| log "Failed to stop initial load generation" | ||
| fi |
There was a problem hiding this comment.
Are you sure we want to shut down the load generation? Maybe we should keep it running so users continuously see things come in
There was a problem hiding this comment.
I'm a fan of this suggestion. This actually solves the Locust bug issue. and doesn't slow down install time
There was a problem hiding this comment.
Hm another Locust bug. If we start the swarm with a post request, and keep it running, users don't get the STOP button in the UI. Meaning they won't be able to able to create their own swarms (which we want them to do) 😬😬😬
The fundamental issue here is the Locust page is all jquery. I'll reduce the swarm size and increase the swarm run time by a little bit.
freelerobot
removed
the
do not merge
|
Ok this is mergeable |
simonz130
added
priority: p2
fix: #601
Run load generator for ~10-15s during installation.
Adds ~5s to previous install time.
Load specs:
user_count=50
spawn_rate=10
At the end of 5 seconds we delete users that have already been created.
Note:
There's a race condition where
curl http://loadgen_ip/stopto stop loadgen returns a successful http response (and UI indicates the swarm has terminated..)BUT this only terminates users that have already been spawned, not the users that were originally scheduled to spawn! Bug filed with Locust here.
For us this might be nice as sandbox users will continue to get traces/metrics from any undeleted remaining users from Locust.
Context:
We wanted to provide initial traffic telemetry for users to view && still provide Locust loadgen UI.
Implemented the workaround suggested by this Locust issue
Testing:
In Span Viewer, use filter
/http/user_agent:python-requests