Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🩹 Fix parameter expression parsing #843

Merged
merged 2 commits into from
Sep 30, 2021

Conversation

s-weigand
Copy link
Member

@s-weigand s-weigand commented Sep 30, 2021

The naive approach using expression.replace lead to problems when an expression for a parameter was part of another expression for a parameter.

Example:
The expression:
"100 - $inputs1.s1 - $inputs1.s3 - $inputs1.s8 - $inputs1.s12"
should have been transformed to:
"100 - group.get('inputs1.s1').value - group.get('inputs1.s3').value - group.get('inputs1.s8').value - group.get('inputs1.s12').value"
but instead was transformed to:
"100 - group.get('inputs1.s1').value - group.get('inputs1.s3').value - group.get('inputs1.s8').value - group.get('inputs1.s1').value2"

Since "inputs1.s1" partially matches "inputs1.s12"

I also moved the regex compiling out of the class, so it only gets executed when the module is loaded.

Change summary

  • 🩹♻️🧪 Fixed Parameter expression transformation
  • 👌 Added missing space in exception text of update_parameter_expression

Checklist

  • ✔️ Passing the tests (mandatory for all PR's)
  • 🧪 Adds new tests for the feature (mandatory for ✨ feature and 🩹 bug fix PR's)

The naive approach using expression.replace lead to problems when an expression for a parameter was part of another expression for a parameter.

Example:
The expression:
"100 - $inputs1.s1 - $inputs1.s3 - $inputs1.s8 - $inputs1.s12"
should have been transformed to:
"100 - group.get('inputs1.s1').value - group.get('inputs1.s3').value - group.get('inputs1.s8').value - group.get('inputs1.s12').value"
but instead was transformed to:
"100 - group.get('inputs1.s1').value - group.get('inputs1.s3').value - group.get('inputs1.s8').value - group.get('inputs1.s1').value2"

Since "inputs1.s1" partially matches "inputs1.s12".

I also moved the regex compiling out of the class, so it only gets executed when the module is loaded.
@s-weigand s-weigand added Type: Serious Bug Crashes, Broken code, Security Issues Priority: High Nasty bugs leading to incorrect results or crashes labels Sep 30, 2021
@s-weigand s-weigand added this to the v0.5.0 milestone Sep 30, 2021
@github-actions
Copy link
Contributor

Binder 👈 Launch a binder notebook on branch s-weigand/pyglotaran/fix-parameter-expression-parsing

@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Sep 30, 2021

Sourcery Code Quality Report

✅  Merging this PR will increase code quality in the affected files by 0.01%.

Quality metrics Before After Change
Complexity 2.70 ⭐ 2.68 ⭐ -0.02 👍
Method Length 40.87 ⭐ 40.87 ⭐ 0.00
Working memory 6.13 ⭐ 6.12 ⭐ -0.01 👍
Quality 78.84% 78.85% 0.01% 👍
Other metrics Before After Change
Lines 1365 1374 9
Changed files Quality Before Quality After Quality Change
glotaran/parameter/parameter.py 88.47% ⭐ 88.43% ⭐ -0.04% 👎
glotaran/parameter/parameter_group.py 69.08% 🙂 69.08% 🙂 0.00%
glotaran/parameter/test/test_parameter.py 78.55% ⭐ 78.54% ⭐ -0.01% 👎

Here are some functions in these files that still need a tune-up:

File Function Complexity Length Working Memory Quality Recommendation
glotaran/parameter/parameter_group.py ParameterGroup.from_dataframe 28 😞 267 ⛔ 13 😞 26.41% 😞 Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
glotaran/parameter/parameter_group.py ParameterGroup.__repr__ 6 ⭐ 72 🙂 12 😞 63.49% 🙂 Extract out complex expressions
glotaran/parameter/parameter_group.py ParameterGroup.markdown 3 ⭐ 118 🙂 10 😞 63.64% 🙂 Extract out complex expressions
glotaran/parameter/parameter.py Parameter.__init__ 1 ⭐ 67 🙂 15 😞 64.94% 🙂 Extract out complex expressions
glotaran/parameter/parameter_group.py ParameterGroup.get_label_value_and_bounds_arrays 4 ⭐ 79 🙂 11 😞 66.12% 🙂 Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

  • ⭐ excellent
  • 🙂 good
  • 😞 poor
  • ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.


Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

@sonarcloud
Copy link

sonarcloud bot commented Sep 30, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@github-actions
Copy link
Contributor

Benchmark is done. Checkout the benchmark result page.
Benchmark differences below 5% might be due to CI noise.

Benchmark diff v0.4.1 vs. main

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [21ba272a]       [4626de2c]
     <v0.4.1>                   
-      42.5±0.4ms       29.6±0.2ms     0.70  BenchmarkOptimize.time_optimize(False, False, False)
-         261±1ms       34.1±0.4ms     0.13  BenchmarkOptimize.time_optimize(False, False, True)
-      58.3±0.2ms       48.3±0.2ms     0.83  BenchmarkOptimize.time_optimize(False, True, False)
-      59.8±0.1ms       51.9±0.3ms     0.87  BenchmarkOptimize.time_optimize(False, True, True)
-      42.3±0.3ms       38.1±0.2ms     0.90  BenchmarkOptimize.time_optimize(True, False, False)
-         260±1ms        42.7±50ms     0.16  BenchmarkOptimize.time_optimize(True, False, True)
       58.7±0.1ms       58.7±0.1ms     1.00  BenchmarkOptimize.time_optimize(True, True, False)
       60.6±0.1ms        69.1±50ms    ~1.14  BenchmarkOptimize.time_optimize(True, True, True)
             181M             179M     0.99  IntegrationTwoDatasets.peakmem_create_result
             199M             196M     0.98  IntegrationTwoDatasets.peakmem_optimize
-         184±3ms          150±3ms     0.82  IntegrationTwoDatasets.time_create_result
-      3.87±0.01s       1.58±0.04s     0.41  IntegrationTwoDatasets.time_optimize

Benchmark diff main vs. PR

Parametrized benchmark signatures:

BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)

All benchmarks:

       before           after         ratio
     [38cdb713]       [4626de2c]
       29.3±0.1ms       29.6±0.2ms     1.01  BenchmarkOptimize.time_optimize(False, False, False)
        35.1±40ms       34.1±0.4ms     0.97  BenchmarkOptimize.time_optimize(False, False, True)
       47.6±0.4ms       48.3±0.2ms     1.02  BenchmarkOptimize.time_optimize(False, True, False)
       51.4±0.4ms       51.9±0.3ms     1.01  BenchmarkOptimize.time_optimize(False, True, True)
       37.8±0.2ms       38.1±0.2ms     1.01  BenchmarkOptimize.time_optimize(True, False, False)
        42.3±50ms        42.7±50ms     1.01  BenchmarkOptimize.time_optimize(True, False, True)
       58.0±0.4ms       58.7±0.1ms     1.01  BenchmarkOptimize.time_optimize(True, True, False)
        94.8±40ms        69.1±50ms    ~0.73  BenchmarkOptimize.time_optimize(True, True, True)
             177M             179M     1.01  IntegrationTwoDatasets.peakmem_create_result
             193M             196M     1.02  IntegrationTwoDatasets.peakmem_optimize
          150±2ms          150±3ms     1.00  IntegrationTwoDatasets.time_create_result
        1.42±0.1s       1.58±0.04s    ~1.11  IntegrationTwoDatasets.time_optimize

@codecov
Copy link

codecov bot commented Sep 30, 2021

Codecov Report

Merging #843 (4626de2) into main (38cdb71) will decrease coverage by 0.0%.
The diff coverage is 100.0%.

Impacted file tree graph

@@           Coverage Diff           @@
##            main    #843     +/-   ##
=======================================
- Coverage   84.9%   84.9%   -0.1%     
=======================================
  Files         77      77             
  Lines       4373    4371      -2     
  Branches     786     785      -1     
=======================================
- Hits        3713    3711      -2     
  Misses       521     521             
  Partials     139     139             
Impacted Files Coverage Δ
glotaran/parameter/parameter_group.py 88.7% <ø> (ø)
glotaran/parameter/parameter.py 98.7% <100.0%> (-0.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38cdb71...4626de2. Read the comment docs.

Copy link
Member

@joernweissenborn joernweissenborn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Member

@jsnel jsnel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Reviewed and tested out locally. 👍

@jsnel jsnel merged commit ca1a75e into glotaran:main Sep 30, 2021
@jsnel jsnel deleted the fix-parameter-expression-parsing branch September 30, 2021 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: High Nasty bugs leading to incorrect results or crashes Type: Serious Bug Crashes, Broken code, Security Issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants