Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

❌ [TEST-PR] Wip/projects against d2b297 #829

Closed
wants to merge 44 commits into from

Conversation

jsnel
Copy link
Member

@jsnel jsnel commented Sep 18, 2021

Analysis of feature/projects had it been merged into staging targeting the commit it branched off from

s-weigand and others added 30 commits August 14, 2021 12:11
…otaran.model (#734)

* Changed calculate_matrix function to return xarrays

* Added relation and constraint to basemodel

* Added functions to apply relations and constraints

* Removed LabelAndMatrix class

* Added relations and constraints tests to base model

* Added penalties to base model

* Adapted models to changes

* Address xarray deprecation warnings

Fixed DeprecationWarning: Using a DataArray object to construct a variable is ambiguous, please extract the data using the .data property. This will raise a TypeError in 0.19.0.

Co-authored-by: Jörn Weißenborn <[email protected]>
Co-authored-by: Joris Snellenburg <[email protected]>
* 🔧 Allowed mypy to run on a subset of glotaran

modules that will be checked are:
- utils
- plugin_system
- deprecation

* 🔧 Partially activated docstring QA tools and fixed issues

modules that will be checked are:
- utils
- plugin_system
- deprecation
This PR removes models like kinetic-image in favor of independent megacomplex model. Therefor, there are a lot of internal changes. 

Key Changes for users:

* use `default-megacomplex` instead of `model-type` in yml spec
* simulations can now be done by setting `global_megacomplex` in a dataset model
* heterogenous dataset models can be analyzed in one optimization, e.g. a spectral and a temporal dataset
* the user now needs to set `group=true` in a scheme to have a grouped analysis.
* Added benchmark for Problem class

* removed print

* ♻️ 'Refactored by Sourcery'

* 🧹 Moved glotaran/analysis/test/test_relations.py

to benchmark/pytest/analysis/test_problem.py

Use 'pytest benchmark/pytest/' to run the benchmarks

* Numerous performance tweaks

* Don't weight data with ones if no weight supplied

* Fic duplicate call to create result_dataset

* Removed dead code

* Switched back to pure numpy in problem

* Fixed example

* Cleanunp

* Update glotaran/analysis/problem_ungrouped.py

Co-authored-by: Sebastian Weigand <[email protected]>

* Update glotaran/analysis/problem_ungrouped.py

Co-authored-by: Sebastian Weigand <[email protected]>

* Update glotaran/analysis/util.py

Co-authored-by: Sebastian Weigand <[email protected]>

Co-authored-by: Sourcery AI <>
Co-authored-by: s-weigand <[email protected]>
* Added Full Model functionality (also known as spectrotemporal models)

* Added finalizing functions for full models and changed spectral decay test to full model test
* Added flake8-print to precommit
* Fixed parameter history and set correct optizemet parameter prior to resultdata creation
* Don't apply clp relations and constraints on full models
* Added auto inference of grouping
* Update glotaran/test/test_spectral_decay_full_model.py
* Correction of cost calculation


Co-authored-by: Joris Snellenburg <[email protected]>
Co-authored-by: Sebastian Weigand <[email protected]>
* Small fix for baseline megacomplex

* Save baseline as name 'baseline'
* Added SpectralShapeGaussian.
* Replaced property energy_spectrum of SpectralMegacomplex with invert and axis_scale.
* Made invert and axis_scale dataset properties.
* Added guard and meaningful exception message for spectral skewness
* Fixed scaling
* Made SpectralShapeSkewedGaussian decendent of SpectralShapeGaussian and
added fallback for skewness == 0.
* Move sanatize.py to utils module and correct typo sanatize -> sanitize (incl rename to sanitize.py)
* Move regex patterns to seperate module in utils
* Add sanity_scientific_notation_conversion
Convert scientific notation string (e.g. 1E7) to proper floats
* Fixed spectral megacomplex test parameters and added test for inverted axis
* Removed model_item._from_list
* Added convenience in model_item.from_dict for automatically convert float or int typed properties which are parsed as strings.
* Made amplitude of shape optional

Co-authored-by: Joris Snellenburg <[email protected]>
Various fixes and improvements to the glotaran command line interface.

* Changed CLI save plugin to folder
* Added outputformat option to CLI
* Added basic test for CLI
* Rename CLI entrypoint to main and add more CLI tests
* 👌 CLI use same default for non_negative_least_squares as scheme
* 👌 CLI dedent pluginlist output
* 🩹 CLI fixed result outformat accepting none supported formats

Co-authored-by: Joris Snellenburg <[email protected]>
Co-authored-by: s-weigand <[email protected]>
* ✨ Added 'deprecate_dict_entry' to deprecate dict keys and/or values
* 🧹📚 Changed  Warns OverDueDeprecation to Raises and added missing Raises OverDueDeprecation to deprecate_module_attribute and deprecate_submodule
* 🩹🧪 Reimplemented check_deprecations and with deprecate_dict_entry and renamed it to 'model_spec_deprecations'
This change also ensures that the deprecation warning is thrown when users calls 'load_model' so python will show the warning
* ♻️🧪 Changed deprecation_warning_on_call_test_helper to test for file emitting the warning and return the WarningsRecorder
* 📚 Added docs for deprecate_dict_entry
* 🧹 Removed unused record variables
* 🧪 Added test for deprecated 'spectral_constraints'
* 👌 Changed DeprecationWarning to GlotaranApiDeprecationWarning (GlotaranApiDeprecationWarning is a subclass of UserWarning so it won't be filtered automatically by python)
* 📚 Updated docs to reflect the usage of GlotaranApiDeprecationWarning
@github-actions
Copy link
Contributor

Binder 👈 Launch a binder notebook on branch glotaran/pyglotaran/wip/projects_against_d2b297

@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Sep 18, 2021

Sourcery Code Quality Report

✅  Merging this PR will increase code quality in the affected files by 0.66%.

Quality metrics Before After Change
Complexity 4.23 ⭐ 4.17 ⭐ -0.06 👍
Method Length 49.76 ⭐ 48.47 ⭐ -1.29 👍
Working memory 8.12 🙂 8.01 🙂 -0.11 👍
Quality 71.59% 🙂 72.25% 🙂 0.66% 👍
Other metrics Before After Change
Lines 11358 11475 117
Changed files Quality Before Quality After Quality Change
benchmark/benchmarks/integration/ex_two_datasets/benchmark.py 85.49% ⭐ 85.49% ⭐ 0.00%
benchmark/pytest/analysis/test_problem.py 84.94% ⭐ 85.02% ⭐ 0.08% 👍
glotaran/analysis/nnls.py 89.32% ⭐ 89.32% ⭐ 0.00%
glotaran/analysis/optimize.py 45.91% 😞 45.71% 😞 -0.20% 👎
glotaran/analysis/problem.py 80.36% ⭐ 80.44% ⭐ 0.08% 👍
glotaran/analysis/problem_grouped.py 57.20% 🙂 58.06% 🙂 0.86% 👍
glotaran/analysis/problem_ungrouped.py 67.75% 🙂 67.78% 🙂 0.03% 👍
glotaran/analysis/simulation.py 64.20% 🙂 64.20% 🙂 0.00%
glotaran/analysis/util.py 66.64% 🙂 62.73% 🙂 -3.91% 👎
glotaran/analysis/variable_projection.py 71.18% 🙂 71.18% 🙂 0.00%
glotaran/analysis/test/models.py 66.88% 🙂 66.10% 🙂 -0.78% 👎
glotaran/analysis/test/test_constraints.py 57.15% 🙂 57.15% 🙂 0.00%
glotaran/analysis/test/test_grouping.py 50.01% 🙂 50.01% 🙂 0.00%
glotaran/analysis/test/test_optimization.py 34.60% 😞 34.60% 😞 0.00%
glotaran/analysis/test/test_penalties.py 64.87% 🙂 64.87% 🙂 0.00%
glotaran/analysis/test/test_problem.py 64.49% 🙂 64.49% 🙂 0.00%
glotaran/analysis/test/test_relations.py 57.85% 🙂 57.85% 🙂 0.00%
glotaran/analysis/test/test_simulation.py 64.39% 🙂 64.39% 🙂 0.00%
glotaran/builtin/io/folder/folder_plugin.py 71.72% 🙂 61.82% 🙂 -9.90% 👎
glotaran/builtin/io/folder/test/test_folder_plugin.py 86.72% ⭐ 93.57% ⭐ 6.85% 👍
glotaran/builtin/io/netCDF/netCDF.py 79.78% ⭐ 87.18% ⭐ 7.40% 👍
glotaran/builtin/io/yml/yml.py 52.74% 🙂 76.57% ⭐ 23.83% 👍
glotaran/builtin/io/yml/test/test_save_result.py 88.48% ⭐ 95.76% ⭐ 7.28% 👍
glotaran/builtin/megacomplexes/baseline/init.py % % %
glotaran/builtin/megacomplexes/baseline/baseline_megacomplex.py 90.42% ⭐ 90.42% ⭐ 0.00%
glotaran/builtin/megacomplexes/coherent_artifact/init.py % % %
glotaran/builtin/megacomplexes/decay/init.py % % %
glotaran/builtin/megacomplexes/decay/decay_megacomplex.py 70.00% 🙂 69.57% 🙂 -0.43% 👎
glotaran/builtin/megacomplexes/decay/util.py 53.83% 🙂 53.73% 🙂 -0.10% 👎
glotaran/builtin/megacomplexes/spectral/init.py % % %
glotaran/builtin/megacomplexes/spectral/spectral_megacomplex.py 64.03% 🙂 64.03% 🙂 0.00%
glotaran/cli/init.py % % %
glotaran/cli/main.py 89.63% ⭐ 89.63% ⭐ 0.00%
glotaran/cli/commands/optimize.py 11.73% ⛔ 11.73% ⛔ 0.00%
glotaran/cli/commands/pluginlist.py 86.81% ⭐ 86.81% ⭐ 0.00%
glotaran/cli/commands/util.py 73.47% 🙂 73.47% 🙂 0.00%
glotaran/cli/commands/test/test_util.py 98.00% ⭐ 98.00% ⭐ 0.00%
glotaran/cli/test/test_cli.py 87.31% ⭐ 87.31% ⭐ 0.00%
glotaran/deprecation/init.py % % %
glotaran/deprecation/deprecation_utils.py 71.06% 🙂 71.06% 🙂 0.00%
glotaran/deprecation/modules/builtin_io_yml.py 51.78% 🙂 61.56% 🙂 9.78% 👍
glotaran/deprecation/modules/test/init.py 80.32% ⭐ 80.32% ⭐ 0.00%
glotaran/deprecation/modules/test/test_builtin_io_yml.py 64.77% 🙂 67.70% 🙂 2.93% 👍
glotaran/deprecation/modules/test/test_changed_imports.py 91.59% ⭐ 91.59% ⭐ 0.00%
glotaran/deprecation/modules/test/test_glotaran_root.py 91.65% ⭐ 91.65% ⭐ 0.00%
glotaran/deprecation/modules/test/test_project_result.py 90.19% ⭐ 90.19% ⭐ 0.00%
glotaran/deprecation/modules/test/test_project_sheme.py 71.23% 🙂 71.23% 🙂 0.00%
glotaran/deprecation/test/test_deprecation_utils.py 86.65% ⭐ 86.65% ⭐ 0.00%
glotaran/examples/sequential.py 46.88% 😞 44.27% 😞 -2.61% 👎
glotaran/examples/test/test_example.py 100.00% ⭐ 100.00% ⭐ 0.00%
glotaran/io/init.py 87.99% ⭐ 87.99% ⭐ 0.00%
glotaran/io/interface.py 97.71% ⭐ 97.52% ⭐ -0.19% 👎
glotaran/model/init.py % % %
glotaran/model/constraint.py 99.50% ⭐ 99.50% ⭐ 0.00%
glotaran/model/dataset_model.py 81.03% ⭐ 81.16% ⭐ 0.13% 👍
glotaran/model/interval_property.py 89.18% ⭐ 89.18% ⭐ 0.00%
glotaran/model/megacomplex.py 72.53% 🙂 72.79% 🙂 0.26% 👍
glotaran/model/model.py 72.67% 🙂 74.57% 🙂 1.90% 👍
glotaran/model/property.py 43.98% 😞 48.19% 😞 4.21% 👍
glotaran/model/relation.py 87.02% ⭐ 87.02% ⭐ 0.00%
glotaran/model/weight.py 79.96% ⭐ 79.96% ⭐ 0.00%
glotaran/model/test/test_model.py 71.05% 🙂 70.88% 🙂 -0.17% 👎
glotaran/parameter/parameter.py 88.47% ⭐ 88.47% ⭐ 0.00%
glotaran/parameter/parameter_group.py 69.30% 🙂 69.30% 🙂 0.00%
glotaran/plugin_system/base_registry.py 77.99% ⭐ 77.99% ⭐ 0.00%
glotaran/plugin_system/data_io_registration.py 92.67% ⭐ 92.43% ⭐ -0.24% 👎
glotaran/plugin_system/megacomplex_registration.py 86.14% ⭐ 86.14% ⭐ 0.00%
glotaran/plugin_system/project_io_registration.py 89.03% ⭐ 87.08% ⭐ -1.95% 👎
glotaran/plugin_system/test/test_base_registry.py 92.47% ⭐ 92.47% ⭐ 0.00%
glotaran/plugin_system/test/test_megacomplex_registration.py 95.12% ⭐ 95.12% ⭐ 0.00%
glotaran/plugin_system/test/test_project_io_registration.py 91.22% ⭐ 91.22% ⭐ 0.00%
glotaran/project/init.py % % %
glotaran/project/result.py 69.15% 🙂 76.39% ⭐ 7.24% 👍
glotaran/project/scheme.py 90.75% ⭐ 90.05% ⭐ -0.70% 👎
glotaran/project/test/test_result.py 80.10% ⭐ 75.03% ⭐ -5.07% 👎
glotaran/project/test/test_scheme.py 76.59% ⭐ 76.97% ⭐ 0.38% 👍
glotaran/test/test_spectral_decay_full_model.py 55.21% 🙂 55.21% 🙂 0.00%
glotaran/utils/ipython.py 91.99% ⭐ 91.99% ⭐ 0.00%
glotaran/utils/regex.py 92.78% ⭐ 92.78% ⭐ 0.00%
glotaran/utils/test/test_ipython.py 86.12% ⭐ 86.12% ⭐ 0.00%

Here are some functions in these files that still need a tune-up:

File Function Complexity Length Working Memory Quality Recommendation
glotaran/cli/commands/optimize.py optimize_cmd 29 😞 457 ⛔ 29 ⛔ 11.73% ⛔ Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
glotaran/analysis/test/test_optimization.py test_optimization 19 😞 509 ⛔ 15 😞 25.13% 😞 Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
glotaran/model/property.py ModelProperty.validate 44 ⛔ 187 😞 13 😞 25.34% 😞 Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
glotaran/parameter/parameter_group.py ParameterGroup.from_dataframe 28 😞 267 ⛔ 13 😞 26.41% 😞 Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
glotaran/analysis/optimize.py _create_result 15 🙂 256 ⛔ 23 ⛔ 27.13% 😞 Try splitting into smaller methods. Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

  • ⭐ excellent
  • 🙂 good
  • 😞 poor
  • ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.


Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

@sonarcloud
Copy link

sonarcloud bot commented Sep 18, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 13 Code Smells

No Coverage information No Coverage information
2.9% 2.9% Duplication

@jsnel jsnel changed the title 🗑️ [TEST-PR] Wip/projects against d2b297 ❌ [TEST-PR] Wip/projects against d2b297 Sep 19, 2021
@jsnel
Copy link
Member Author

jsnel commented Oct 24, 2021

This was an experimental PR which has been superseded by a number of other smaller PRs:

@jsnel jsnel closed this Oct 24, 2021
@jsnel jsnel deleted the wip/projects_against_d2b297 branch October 24, 2021 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants