✨ Add optimization history to result and iteration column to parameter history

[s-weigand]

This change allows inspecting the stdout output from scipy.optimize.least_squares during optimization and saving the current iteration to the parameter history, as well as creating a pd.DataFrame for plotting and inspection.

The OptimizationHistory class is written in a way that allows using it like a view of pd.DataFrame e.g.

result.optimization_history.cost.plot()

However, since the wrapping functionality is dynamic at runtime you lose help information and autocompleting from static analyzers, which why the underlying pd.DataFrame is also exposed via the data property which gives you the static analyzer goodies.

result.optimization_history.data.cost.plot()

Change summary

• ✨ Added TeeContext
• 🧹 Don't save index in when saving ParameterHistory to csv
• ✨ Added iteration to parameter history
• 👌⌨️ Improved typing of glotaran.project.dataclass_helpers.fromdict
• ✨ Added OptimizationHistory to Result
• 🧪 Added roundtrip test to result yml saving
• 📚 Added plot of optimization_history to quickstart
• 🔧 Remove more metadata from notebooks with nbstripout
• 👌 Disallow usage of 'iteration' as parameter label
• ♻️ Made get_current_optimization_iteration a static method of Optimizer
• ♻️ Moved optimization_history module from project to optimization pkg

Links to the changed sections

• Plot of optimization history

Checklist

• ✔️ Passing the tests (mandatory for all PR's)
• 🚧 Added changes to changelog (mandatory for all PR's)
• 👌 Closes issue (mandatory for ✨ feature and 🩹 bug fix PR's)
• 🧪 Adds new tests for the feature (mandatory for ✨ feature and 🩹 bug fix PR's)
• 📚 Adds documentation of the feature

Closes issues

Might allow #233 or one of the related issues to be closed

[sourcery-ai]

Sourcery Code Quality Report

❌  Merging this PR will decrease code quality in the affected files by 0.23%.

Quality metrics	Before	After	Change
Complexity	3.47 ⭐	3.40 ⭐	-0.07 👍
Method Length	49.26 ⭐	50.49 ⭐	1.23 👎
Working memory	6.61 🙂	6.61 🙂	0.00
Quality	76.26% ⭐	76.03% ⭐	-0.23% 👎

Other metrics	Before	After	Change
Lines	3290	3369	79

Changed files	Quality Before	Quality After	Quality Change
glotaran/builtin/io/folder/folder_plugin.py	55.30% 🙂	54.05% 🙂	-1.25% 👎
glotaran/builtin/io/yml/yml.py	73.74% 🙂	73.74% 🙂	0.00%
glotaran/builtin/io/yml/test/test_save_result.py	84.85% ⭐	80.19% ⭐	-4.66% 👎
glotaran/optimization/optimizer.py	67.62% 🙂	67.93% 🙂	0.31% 👍
glotaran/parameter/parameter.py	82.12% ⭐	82.11% ⭐	-0.01% 👎
glotaran/parameter/parameter_group.py	74.25% 🙂	74.22% 🙂	-0.03% 👎
glotaran/parameter/parameter_history.py	93.47% ⭐	92.13% ⭐	-1.34% 👎
glotaran/parameter/test/test_parameter_history.py	71.76% 🙂	70.92% 🙂	-0.84% 👎
glotaran/project/dataclass_helpers.py	58.51% 🙂	58.46% 🙂	-0.05% 👎
glotaran/project/project.py	83.50% ⭐	83.50% ⭐	0.00%
glotaran/project/result.py	71.84% 🙂	71.83% 🙂	-0.01% 👎
glotaran/typing/types.py	81.68% ⭐	80.13% ⭐	-1.55% 👎
glotaran/utils/regex.py	91.21% ⭐	89.68% ⭐	-1.53% 👎

Here are some functions in these files that still need a tune-up:

File	Function	Complexity	Length	Working Memory	Quality	Recommendation
glotaran/project/result.py	Result.markdown	10 🙂	242 ⛔	14 😞	39.37% 😞	Try splitting into smaller methods. Extract out complex expressions
glotaran/parameter/parameter.py	Parameter.markdown	13 🙂	186 😞	13 😞	41.94% 😞	Try splitting into smaller methods. Extract out complex expressions
glotaran/project/dataclass_helpers.py	asdict	18 🙂	142 😞	13 😞	42.15% 😞	Try splitting into smaller methods. Extract out complex expressions
glotaran/project/dataclass_helpers.py	file_loader_factory	18 🙂	136 😞		47.45% 😞	Try splitting into smaller methods
glotaran/builtin/io/yml/yml.py	YmlProjectIo.save_model	15 🙂	117 🙂	13 😞	47.50% 😞	Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

• ⭐ excellent
• 🙂 good
• 😞 poor
• ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.

---

Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

[github-actions]

👈 Launch a binder notebook on branch s-weigand/pyglotaran/optimization-history

[github-actions]

Benchmark is done. Checkout the benchmark result page.
Benchmark differences below 5% might be due to CI noise.

Benchmark diff v0.6.0 vs. main

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [6c3c390e]       [7c203e3b]
     <v0.6.0>                   
+      62.3±0.4ms          104±1ms     1.66  BenchmarkOptimize.time_optimize(False, False, False)
        99.1±30ms        124±0.6ms    ~1.25  BenchmarkOptimize.time_optimize(False, False, True)
+      62.4±0.3ms        103±0.7ms     1.65  BenchmarkOptimize.time_optimize(False, True, False)
        73.3±20ms          124±1ms    ~1.69  BenchmarkOptimize.time_optimize(False, True, True)
+      75.0±0.7ms        177±0.7ms     2.37  BenchmarkOptimize.time_optimize(True, False, False)
+       80.0±20ms        198±0.8ms     2.48  BenchmarkOptimize.time_optimize(True, False, True)
+        76.6±2ms        177±0.6ms     2.31  BenchmarkOptimize.time_optimize(True, True, False)
+       81.4±40ms        198±0.6ms     2.44  BenchmarkOptimize.time_optimize(True, True, True)
             204M             204M     1.00  IntegrationTwoDatasets.peakmem_optimize
-      2.02±0.07s       1.63±0.05s     0.81  IntegrationTwoDatasets.time_optimize

Benchmark diff main vs. PR

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [7c203e3b]       [a769410f]
          104±1ms          108±2ms     1.04  BenchmarkOptimize.time_optimize(False, False, False)
        124±0.6ms          133±4ms     1.08  BenchmarkOptimize.time_optimize(False, False, True)
        103±0.7ms          109±4ms     1.05  BenchmarkOptimize.time_optimize(False, True, False)
          124±1ms          131±2ms     1.06  BenchmarkOptimize.time_optimize(False, True, True)
        177±0.7ms          181±1ms     1.02  BenchmarkOptimize.time_optimize(True, False, False)
        198±0.8ms        201±0.8ms     1.02  BenchmarkOptimize.time_optimize(True, False, True)
        177±0.6ms        181±0.9ms     1.02  BenchmarkOptimize.time_optimize(True, True, False)
        198±0.6ms          202±1ms     1.02  BenchmarkOptimize.time_optimize(True, True, True)
             204M             204M     1.00  IntegrationTwoDatasets.peakmem_optimize
       1.63±0.05s       1.61±0.02s     0.99  IntegrationTwoDatasets.time_optimize

[codecov]

Codecov Report

Base: 87.0% // Head: 87.2% // Increases project coverage by +0.1% 🎉

Coverage data is based on head (a769410) compared to base (7c203e3).
Patch coverage: 91.3% of modified lines in pull request are covered.

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #1134     +/-   ##
=======================================
+ Coverage   87.0%   87.2%   +0.1%     
=======================================
  Files        101     103      +2     
  Lines       5203    5274     +71     
  Branches     912     916      +4     
=======================================
+ Hits        4531    4599     +68     
- Misses       525     527      +2     
- Partials     147     148      +1     

Impacted Files	Coverage Δ
glotaran/builtin/io/yml/yml.py	90.6% <ø> (ø)
glotaran/parameter/parameter_group.py	88.8% <ø> (ø)
glotaran/optimization/optimizer.py	81.3% <73.6%> (+1.7%)	⬆️
glotaran/optimization/optimization_history.py	93.5% <93.5%> (ø)
glotaran/utils/tee.py	95.2% <95.2%> (ø)
glotaran/builtin/io/folder/folder_plugin.py	100.0% <100.0%> (ø)
glotaran/parameter/parameter.py	95.3% <100.0%> (ø)
glotaran/parameter/parameter_history.py	93.8% <100.0%> (+0.1%)	⬆️
glotaran/project/dataclass_helpers.py	83.8% <100.0%> (ø)
glotaran/project/project.py	100.0% <100.0%> (ø)
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[s-weigand]

@jsnel I triggered a binder build of the PR branch so you don't need to wait to try it out.

[joernweissenborn]

Lgtm

[jsnel]

Nice functionality! Some small changes suggested.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.8% Duplication

[jsnel]

Looks good to me now.