feat: allow for defining model comparison scenarios in YAML files #330
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…or yaml parse/validation issues
…ions to separate files and add "Compare" prefix to ScenarioInfo and DatasetInfo types
…les by id or custom alias * add inputId to InputVar interface * add inputAliases map to ModelSpec interface * make inputGroups and datasetGroups optional in ModelSpec * implement conversion of parsed specs to resolved CompareScenario instances
…blic "Spec" types
… by stage [skip ci]
…lution of scenario groups
…it from scenario if needed)
…view title/subtitle from scenario
…y -> uid and derive from settings, add separate CompareScenarioDefKey type [skip ci]
…key/specL/specR to CompareScenario [skip ci]
…n check-core (instead of in check-ui-shell)
… changes [skip ci]
[skip ci]
…ement context graphs
[skip ci]
…ios" checkbox [skip ci]
[skip ci]
[skip ci]
[skip ci]
… the "all inputs" row at top
…iltering datasets used for comparison
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #315
This includes a large set of changes to the model-check packages. It makes it possible to define comparison tests in yaml files similar to they way you can define check tests. (Previously the only way to define these was in code, which made them difficult to configure and update.) It includes support for defining custom "comparison views" which show a particular set of graphs for a given scenario.
It also includes many improvements to the model-check report interface, which now has a tabbed interface, improved navigation, and better annotations to show when tests are misconfigured. See issue for screenshots.
We've been using this branch internally for En-ROADS and C-ROADS for a few months and it's been working well, so I'm ready to merge this. No impact on other SDE packages. There are breaking changes in
check-coreandcheck-ui-shellpackages but those are still considered implementation details, so I'm not flagging those. (The changes will impact anyone defining custom model-check tests or bundles, but I'm not sure anyone outside of Climate Interactive uses these features yet; if you do, get in touch on the discussion board and I can help you migrate.)