Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: do not always retry load tests requests #3300

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

olevski
Copy link
Member

@olevski olevski commented Feb 6, 2023

Fixes #3240

NOTE: This only affects the load tests. These do not currently run in any CI and are not deployed anywhere.

I cleaned up some unnecessary logging and added some metrics to track the duration of specific requests.

Example output:

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: class-scenario.js
     output: -

  scenarios: (100.00%) 1 scenario, 4 max VUs, 32m0s max duration (incl. graceful stop):
           * lecture: 1 iterations for each of 4 VUs (maxDuration: 30m0s, gracefulStop: 2m0s)


running (02m48.3s), 0/4 VUs, 4 complete and 0 interrupted iterations
lecture ✓ [======================================] 4 VUs  02m45.3s/30m0s  4/4 iters, 1 per VU

     █ setup

       ✓ request succeeded with 2XX
       ✓ getting project info response has no error

     █ fork

       ✓ request succeeded with 2XX

     █ launch session

       ✓ request succeeded with 2XX

     █ shutdown server

       ✓ request succeeded with 2XX

     █ remove project

       ✓ request succeeded with 2XX

     checks.........................: 100.00% ✓ 218      ✗ 0  
     data_received..................: 804 kB  4.8 kB/s
     data_sent......................: 66 kB   392 B/s
     group_duration.................: avg=38.22s   min=258.37ms med=30.38s   max=1m31s    p(90)=1m31s    p(95)=1m31s   
     http_req_blocked...............: avg=3.46ms   min=0s       med=1µs      max=88.56ms  p(90)=2µs      p(95)=31.42ms 
     http_req_connecting............: avg=1ms      min=0s       med=0s       max=67.78ms  p(90)=0s       p(95)=6.06ms  
     http_req_duration..............: avg=213.34ms min=9.86ms   med=173.57ms max=2.41s    p(90)=272.91ms p(95)=369.76ms
       { expected_response:true }...: avg=213.34ms min=9.86ms   med=173.57ms max=2.41s    p(90)=272.91ms p(95)=369.76ms
     http_req_failed................: 0.00%   ✓ 0        ✗ 312
     http_req_receiving.............: avg=241.16µs min=26µs     med=77µs     max=11.67ms  p(90)=178.3µs  p(95)=1.31ms  
     http_req_sending...............: avg=174.84µs min=55µs     med=180µs    max=350µs    p(90)=232.9µs  p(95)=251.34µs
     http_req_tls_handshaking.......: avg=2.32ms   min=0s       med=0s       max=38.98ms  p(90)=0s       p(95)=24.22ms 
     http_req_waiting...............: avg=212.92ms min=9.6ms    med=173ms    max=2.41s    p(90)=272.65ms p(95)=369.54ms
     http_reqs......................: 312     1.854364/s
     iteration_duration.............: avg=2m12s    min=2.96s    med=2m45s    max=2m45s    p(90)=2m45s    p(95)=2m45s   
     iterations.....................: 4       0.023774/s
     session_create_req_duration....: avg=2.27s    min=2.14s    med=2.28s    max=2.41s    p(90)=2.39s    p(95)=2.4s    
     session_delete_req_duration....: avg=302.63ms min=273.48ms med=305.55ms max=325.95ms p(90)=324.66ms p(95)=325.31ms
     session_get_req_duration.......: avg=185.81ms min=140.13ms med=176.61ms max=754.04ms p(90)=220.88ms p(95)=236.04ms
     session_startup_duration.......: avg=49s      min=49s      med=49s      max=49s      p(90)=49s      p(95)=49s     
     vus............................: 3       min=0      max=4
     vus_max........................: 4       min=4      max=4

@olevski olevski requested a review from a team as a code owner February 6, 2023 17:59
@olevski olevski requested a review from li-il-li February 6, 2023 17:59
@coveralls
Copy link
Collaborator

coveralls commented Feb 16, 2023

Pull Request Test Coverage Report for Build 6875247711

Warning: This coverage report may be inaccurate.

We've detected an issue with your CI configuration that might affect the accuracy of this pull request's coverage report.
To ensure accuracy in future PRs, please see these guidelines.
A quick fix for this PR: rebase it; your next report should be accurate.

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 16 unchanged lines in 8 files lost coverage.
  • Overall coverage decreased (-0.05%) to 82.786%

Files with Coverage Reduction New Missed Lines %
renku/core/dataset/dataset.py 1 79.01%
renku/core/dataset/providers/dataverse.py 1 88.86%
renku/core/migration/models/v3.py 1 97.24%
renku/domain_model/dataset.py 1 92.84%
renku/domain_model/image.py 1 95.65%
renku/ui/cli/init.py 1 96.9%
renku/ui/cli/service.py 1 28.57%
renku/core/dataset/providers/renku.py 9 86.69%
Totals Coverage Status
Change from base Build 6823291196: -0.05%
Covered Lines: 25441
Relevant Lines: 30731

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Remove retries from load tests
3 participants