🚧 Forward port Improve result comparison workflow and v0.4 changelog

[jsnel]

Result comparison was too sensitive to numerical fluctuation, the tolerances needed to be adjusted on both the v0.4 maintenance branch as well as main.

PR targeting v0.4 maintenance:

• 🚇Update test_result_consistency #936

This PR also forward ports missing changelog entries from the v0.4 maintenance branch

Checklist

• ✔️ Passing the tests (mandatory for all PR's)
• 👌 Closes issue (mandatory for ✨ feature and 🩹 bug fix PR's)

Closes issues

Closes #937

[sourcery-ai]

Sourcery Code Quality Report

❌  Merging this PR will decrease code quality in the affected files by 1.07%.

Quality metrics	Before	After	Change
Complexity	7.99 ⭐	8.24 ⭐	0.25 👎
Method Length	80.36 🙂	83.57 🙂	3.21 👎
Working memory	11.71 😞	12.47 😞	0.76 👎
Quality	58.25% 🙂	57.18% 🙂	-1.07% 👎

Other metrics	Before	After	Change
Lines	369	375	6

Changed files	Quality Before	Quality After	Quality Change
.github/test_result_consistency.py	58.25% 🙂	57.18% 🙂	-1.07% 👎

Here are some functions in these files that still need a tune-up:

File	Function	Complexity	Length	Working Memory	Quality	Recommendation
.github/test_result_consistency.py	data_var_test	11 🙂	316 ⛔	24 ⛔	27.94% 😞	Try splitting into smaller methods. Extract out complex expressions
.github/test_result_consistency.py	map_result_files	15 🙂	140 😞	9 🙂	52.01% 🙂	Try splitting into smaller methods
.github/test_result_consistency.py	get_compare_results_path	8 ⭐	130 😞	11 😞	55.42% 🙂	Try splitting into smaller methods. Extract out complex expressions
.github/test_result_consistency.py	coord_test	8 ⭐	97 🙂	13 😞	56.37% 🙂	Extract out complex expressions
.github/test_result_consistency.py	map_result_parameters	3 ⭐	65 🙂	10 😞	71.13% 🙂	Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

• ⭐ excellent
• 🙂 good
• 😞 poor
• ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.

---

Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

[github-actions]

👈 Launch a binder notebook on branch jsnel/pyglotaran/feature/improve_result_comparison_workflow

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[codecov]

Codecov Report

Merging #938 (057e516) into main (2c2c605) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##            main    #938   +/-   ##
=====================================
  Coverage   85.1%   85.1%           
=====================================
  Files         85      85           
  Lines       4799    4799           
  Branches     921     921           
=====================================
  Hits        4085    4085           
  Misses       562     562           
  Partials     152     152           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c2c605...057e516. Read the comment docs.

[github-actions]

Benchmark is done. Checkout the benchmark result page.
Benchmark differences below 5% might be due to CI noise.

Benchmark diff v0.5.0 vs. main

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [89046e0a]       [057e516c]
     <v0.5.0>                   
         74.5±2ms         75.8±2ms     1.02  BenchmarkOptimize.time_optimize(False, False, False)
          157±3ms         159±20ms     1.01  BenchmarkOptimize.time_optimize(False, False, True)
       73.5±0.7ms         74.9±1ms     1.02  BenchmarkOptimize.time_optimize(False, True, False)
         160±10ms          164±3ms     1.02  BenchmarkOptimize.time_optimize(False, True, True)
         90.9±1ms         93.2±3ms     1.03  BenchmarkOptimize.time_optimize(True, False, False)
         103±40ms         104±40ms     1.01  BenchmarkOptimize.time_optimize(True, False, True)
         93.0±1ms         94.1±2ms     1.01  BenchmarkOptimize.time_optimize(True, True, False)
         100±20ms         118±40ms    ~1.17  BenchmarkOptimize.time_optimize(True, True, True)
             194M             193M     1.00  IntegrationTwoDatasets.peakmem_optimize
       2.37±0.07s       2.32±0.07s     0.98  IntegrationTwoDatasets.time_optimize

Benchmark diff main vs. PR

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [2c2c6053]       [057e516c]
       76.4±0.7ms         75.8±2ms     0.99  BenchmarkOptimize.time_optimize(False, False, False)
         173±30ms         159±20ms     0.92  BenchmarkOptimize.time_optimize(False, False, True)
         76.4±1ms         74.9±1ms     0.98  BenchmarkOptimize.time_optimize(False, True, False)
         154±20ms          164±3ms     1.06  BenchmarkOptimize.time_optimize(False, True, True)
       92.3±0.9ms         93.2±3ms     1.01  BenchmarkOptimize.time_optimize(True, False, False)
          100±5ms         104±40ms     1.04  BenchmarkOptimize.time_optimize(True, False, True)
       93.9±0.7ms         94.1±2ms     1.00  BenchmarkOptimize.time_optimize(True, True, False)
          101±2ms         118±40ms    ~1.17  BenchmarkOptimize.time_optimize(True, True, True)
             195M             193M     0.99  IntegrationTwoDatasets.peakmem_optimize
       2.33±0.06s       2.32±0.07s     1.00  IntegrationTwoDatasets.time_optimize

[s-weigand]

LGTM 👍

[jsnel]

Although the "Run Examples / Compare Results (pull_request)" workflow is failing here, we know from the maintenance branch (PR #936 ) that it fixes the comparison issues, but it is dependent on the Hotfix pending in PR #927.