Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for defining model comparison scenarios in YAML files #315

Closed
chrispcampbell opened this issue Mar 8, 2023 · 1 comment · Fixed by #330 or #331
Closed

Allow for defining model comparison scenarios in YAML files #315

chrispcampbell opened this issue Mar 8, 2023 · 1 comment · Fixed by #330 or #331

Comments

@chrispcampbell
Copy link
Contributor

Currently we have an asymmetry between the "check" and "compare" halves of the model-check tool. The "check" tests can be defined in YAML files, but so far we have no support for "compare" tests defined in YAML.

We are trying to define many more comparison tests for En-ROADS and C-ROADS, but the custom scenarios are hard to define in code, and it would be better if we could define them in YAML files (and eventually, finish the browser-based tool for defining both "check" and "compare" tests).

As a related enhancement (best implemented at the same time as adding the YAML support), I'm going to add support for "user scenarios" in the "Comparisons" section of the model-check report. We already have all the underlying support for setting up custom input scenarios, but these user scenarios will allow for customizing what we show in the report. For example, you can set up a user scenario that sets a few specific inputs in the model, and then in the detail page for that scenario, it can show specified user-oriented graphs (as the user would see in a simulator). We already have most of the support for the UI part of this implemented as well, so this is just a matter of shuffling some things around for better presentation.

@chrispcampbell
Copy link
Contributor Author

Here are a few screenshots (taken from the sample-check-app demo) that show the improved UI for the model-check report.

Tabbed interface

image

New tab for user-defined "comparison views"

image

New and improved "comparisons by scenario" tab

... that better handles/annotates special cases, like when an input is added/removed/renamed or has its range changed.

image

New and improved "comparisons by output" tab

... that includes improved annotations for added/removed/renamed variables.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment