Explore results after running a model

[suvayu]

Background

As mentioned in #2492, one of the user functions of the DB editor is inspecting results after a model run. Visualisations currently accessible to the user is insufficient as it doesn't allow for exploration.

Implementation goals

Ability to:

• explore the result across its different dimensions (or indices)
• plot as distribution (histograms/bar chart) / sequence (trend lines, e.g. for a time series)
• plot them against each other to identify patterns
• facet across more than 2 dimensions (i.e. when you create a panel across the other dimensions). Example from other projects:
  • Seaborn gallery
  • Bokeh gallery

[suvayu]

@nelliputkonen I created this to track the "results checking" issue. Please feel free to edit the top post if I missed any points.

[manuelma]

In my opinion a lot of the goals are already achieved to some extent, so we will probably need to be more specific - but I guess that's part of the issue, to really narrow down the needs. It's good to be general as a starting point.

[nelliputkonen]

Thanks Suvayu and sorry it took so long to add my comments. In addition to what you have written, I would add the following functionalities (my barrel of wishes):

• In general: Data Store properties widget should have a choice what 'tools' are available in the DB editor. At least distuinghin between input data addition and result checking (if DB is just input data, result checking functionality does not need to be visible). Some DBs are both (output of one model but also input of another), so this should be multiple choice.
• Two new docs (visible by default if result database): Plot dock and Values dock -> When you click on a result table in Parameter value list, Plot dock shows the plotting options of the values and Values the table view. This way you don’t have to double-click/right-click+select on the “Map” value of the result table and they open in new windows. (We should still preserve that option in case the user wants to have multiple result tables open at the same time.)
• Make it easier to hide unnecessary docks: Parameter definition, Entity alternative, Parameter value list, Metadata and Item metadata (maybe about the Mode of working).
• Make it more visible for the user to choose whether to keep old results or not. Currently there is 'purge before writing', but there could also be dialog (that one can turn off). NELLI: Also add option to rewrite results of the same scenario, but keep others in the DB.
• You typically only want to view results from one model and certain result runs. Currently if you have results from multiple models in the same database, the list of entities includes the entities from all the models. Also the list of alternatives can get quite long. At the moment one can filter by alternatives (in v0.8). However, maybe there could also be a more sticky choice (checkbox) that applies the alternative filter also when not keeping ctrl-pressed.
• Disable possibility to edit values in result databases. (Additional checkbox in Data Store properties)
• Enable workflow developers to make most important results for the modelling tool in question easily available, for example:
  o Entity tree could have automatic entity class: “results” where developers can specify model specific entities like “basic results” (which could include result tables for grouped outputs, generation mix, costs, emissions etc.) and “flexibility indicators” (curtailment, loss-of-load etc.). Node and unit level result could still be accessed just like now. JUHA; This is probably mostly possible, it just needs the developer to do the classes. And maybe to have entity_class ordering possible.
• This is a workflow editor issue, but when exporting results (to e.g. Excel), in addition to a scenario filter, an alternative filter would be useful (implemented in v0.8)
• It's very hard to delete certain result runs completely from the result DB (but keeping others). If you remove scenarios and alternatives, the entities that used to belong to those runs, still remain in the entity tree. (If you have had two models in the database, that can be dozens to hundreds of entities). There should be a way to remove them.

[soininen]

This is a workflow editor issue, but when exporting results (to e.g. Excel), in addition to a scenario filter, an alternative filter would be useful

Alternative filters have already been implemented in 0.8-dev branch.

[suvayu]

Couple of questions from today's short meeting:

• Currently we can plot the time series in the results, what other alternative visualisations are useful when looking at this kind of data?
• Currently working with the SpineOpt tutorial which has a time series for a day (24 points). Can someone share (here or privately) example results that are longer? e.g.
  • a year, or multi-year
  • finer resolution

[jkiviluo]

I edited Nelli's list, since some things are already possible, but not visible enough, I tried to express that.

[jkiviluo]

Currently we can plot the time series in the results, what other alternative visualisations are useful when looking at this kind of data?

• Statistics of time series could be nice (max, min, average, sum, st. dev, number of empty values).
• Heatmaps
• Distribution (ordered data)

For longer data, copy paste should work. Here's some example data:
https://github.com/vttresearch/north_european_model/tree/main/timeseries/Basic_processing/Elec_demand/input

[suvayu]

Distribution (ordered data)

I'm not sure I follow this one. I'm guessing it's not a histogram. What is meant by "ordered" here?

[nelliputkonen]

Here is a dummy database of FlexTool Egypt national model results for one full year in hourly resolution:
https://drive.google.com/file/d/1AMx-YYlVr3tHmal7RJWcM3cVYHLp3VKC/view?usp=sharing
(Let me know once you have it, I can then remove it from my personal Drive)

I think Juha means by "Distribution (ordered data)" possibility to rearrange the datapoints based on value e.g. smallest to largest.

One other nice visualization option could be stacked bars (or stacked area if you can make it handle negative values better than Excel), and/or combinations of stacked + line:

PS. I added couple of points to my original list edited by Juha.

[suvayu]

Perfect! Everything is clear now :)

You can remove the file now :)