Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚇🩹 Fix wrong comparison in pr_benchmark workflow #1097

Merged
merged 2 commits into from
Jun 11, 2022

Conversation

s-weigand
Copy link
Member

@s-weigand s-weigand commented Jun 11, 2022

In #1060 we just found that there is an error in the comparison of the last tag and the main branch in the pr_benchmark workflow. Instead of comparing the last tag to main, it compares it against the PR HEAD

Change summary

Checklist

  • ✔️ Passing the tests (mandatory for all PR's)
  • 🚧 Added changes to changelog (mandatory for all PR's)

@s-weigand s-weigand requested a review from a team as a code owner June 11, 2022 15:37
@github-actions
Copy link
Contributor

Binder 👈 Launch a binder notebook on branch s-weigand/pyglotaran/fix/benchmark-workflow

@github-actions
Copy link
Contributor

github-actions bot commented Jun 11, 2022

Benchmark is done. Checkout the benchmark result page.
Benchmark differences below 5% might be due to CI noise.

Benchmark diff v0.6.0 vs. main

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [6c3c390e]       [73ebe461]
     <v0.6.0>                   
       53.9±0.4ms       53.7±0.2ms     1.00  BenchmarkOptimize.time_optimize(False, False, False)
        58.6±30ms       57.9±0.7ms     0.99  BenchmarkOptimize.time_optimize(False, False, True)
       53.8±0.3ms       53.7±0.2ms     1.00  BenchmarkOptimize.time_optimize(False, True, False)
       57.2±0.3ms       57.8±0.5ms     1.01  BenchmarkOptimize.time_optimize(False, True, True)
       64.6±0.5ms       65.1±0.4ms     1.01  BenchmarkOptimize.time_optimize(True, False, False)
         148±50ms       68.0±0.3ms    ~0.46  BenchmarkOptimize.time_optimize(True, False, True)
       65.1±0.4ms       64.9±0.3ms     1.00  BenchmarkOptimize.time_optimize(True, True, False)
        68.5±30ms         171±60ms    ~2.49  BenchmarkOptimize.time_optimize(True, True, True)
             205M             201M     0.98  IntegrationTwoDatasets.peakmem_optimize
       1.72±0.07s       1.74±0.08s     1.01  IntegrationTwoDatasets.time_optimize

Benchmark diff main vs. PR

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [73ebe461]       [50612667]
       53.7±0.2ms       53.4±0.7ms     0.99  BenchmarkOptimize.time_optimize(False, False, False)
       57.9±0.7ms         58.5±4ms     1.01  BenchmarkOptimize.time_optimize(False, False, True)
       53.7±0.2ms       54.0±0.8ms     1.00  BenchmarkOptimize.time_optimize(False, True, False)
       57.8±0.5ms         59.3±3ms     1.03  BenchmarkOptimize.time_optimize(False, True, True)
       65.1±0.4ms       64.8±0.4ms     1.00  BenchmarkOptimize.time_optimize(True, False, False)
       68.0±0.3ms         71.5±4ms     1.05  BenchmarkOptimize.time_optimize(True, False, True)
       64.9±0.3ms       64.3±0.2ms     0.99  BenchmarkOptimize.time_optimize(True, True, False)
         171±60ms        67.8±20ms    ~0.40  BenchmarkOptimize.time_optimize(True, True, True)
             201M             204M     1.01  IntegrationTwoDatasets.peakmem_optimize
       1.74±0.08s       1.66±0.07s     0.95  IntegrationTwoDatasets.time_optimize

@codecov
Copy link

codecov bot commented Jun 11, 2022

Codecov Report

Merging #1097 (5061266) into main (73ebe46) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##            main   #1097   +/-   ##
=====================================
  Coverage   87.1%   87.1%           
=====================================
  Files        101     101           
  Lines       5339    5339           
  Branches     995     995           
=====================================
  Hits        4654    4654           
  Misses       530     530           
  Partials     155     155           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 73ebe46...5061266. Read the comment docs.

jsnel
jsnel previously approved these changes Jun 11, 2022
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.7% 0.7% Duplication

@jsnel jsnel merged commit 60115bd into glotaran:main Jun 11, 2022
@s-weigand s-weigand deleted the fix/benchmark-workflow branch June 11, 2022 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants