Validation of results using tests

[ckaldemeyer]

Within the development-phase of renpass-gis, results have to be validated. Since this is a general procedure for most models, the idea was to implement this using tests. If suitable, these tests can then be implemented into oemof_base afterwards.

Here are some quick ideas to start with (any further suggestions are welcome):

• Bus balances: check if zero for all timesteps (regarding a small deviation "epsilon")
• Transformers
  • Check if outputs divided through their efficiency equal their input for all timesteps
  • Check if full load hours are between 0 and 8760
• Storages
  • Check storage balance over time (sum(inflow)>sum(outflow))
  • Check if SOC(t=0) is SOC(t=t_max)
• Merit-Order-Checks
  • Check if order sort(marginal_prices) equals order of sort(full load hours)
  • Merit-Order plots (price vs. quantity): Check if all transformers on the right side of the required quantity (demand) are zero

@oemof/oemof-main : To you have any suggestions where to implement this? Using nose directly in the test folder?

@simonhilpert : Did I forget something?
@uvchik : I remember you telling me that you already had testing procedures at the RLI. Feel free to contribute!

[uvchik]

I would differ between programming test and system validation tests. For me there are three kinds of test:

1.) The normal nose test should be very fast because you execute them quite often to check if the program itself works. The optimisation itself should not be part of these tests.

2.) Validation test to check if an optimisation still gets the same results after changing the code. These test should be done before a merge or at least before we will release. There should be a handful of representative models.

3.) Plausibility test to check the results of a new optimisation. They are optional but a good help if the models become more and more complicated.

I think your suggestions are of the third kind.

[uvchik]

Maybe it is to much for the restructuring meeting but we should adopt a roadmap there. Shall we change the milestone to that meeting?

[cswh]

perhaps it would be better suited for the oemof developer meeting in may. what do you think?

[uvchik]

It is an important part of the quality management and people already broke other peoples app by pushing untested code. But it will be a matter of time, so let's see.

[uvchik]

I would shift that topic to the re-factoring meeting.

[uvchik]

@ckaldemeyer Do you think this is issue is covered by PR #160 and #154? Otherwise you should specify the remaining topics.

[uvchik]

@ckaldemeyer : Sorry I misunderstood the main point. I guess the main point is to implement plausibility tests. Do you think it should be a part of solph or a part of the outputlib?

[uvchik]

Sorry, just hit the wrong button 😄

[ckaldemeyer]

@ckaldemeyer : Sorry I misunderstood the main point. I guess the main point is to implement plausibility tests. Do you think it should be a part of solph or a part of the outputlib?

Another idea: would a method in the EnergySystem class make sense?

If not, I would tend to do it in solph!

[uvchik]

I think it depends on your approach but if it is part of the EnergySystem it should work for oemof in general. So think implementing it within solph will be easier.

[ckaldemeyer]

I think it depends on your approach but if it is part of the EnergySystem it should work for oemof in general. So think implementing it within solph will be easier.

👍

[ckaldemeyer]

Maybe we can shedule this to the release after the refactoring-release as I won't be able to commit myself before September..

[ckaldemeyer]

I guess this can be closed, or not?

[ckaldemeyer]

Or at least be split in two kinds of tests

[ckaldemeyer]

I gues this can be closed, or not?

[uvchik]

We do have code tests, example test.

We do not have plausibility tests and as I understand it this was your main concern according to your first comment.

The ideas of you first comment are quite interesting. Maybe we should add wiki to save these ideas but make it possible to close ancient issues.

https://github.com/oemof/oemof/wiki/ideas

[uvchik]

@oemof/oemof-developer-group Please keep it in mind that there is a wiki like this.