Enhanced generate_weights that takes warming levels and NaN in consideration

[RondeauG]

Addressing a Problem?

Not all simulations/experiments will reach a given warming level. If multiple warming levels exist in a single dataset (with dimension horizon == ["+2C", "+3C", "+4C"]), some combinations will be purely NaNs.

This becomes a problem for ensemble statistics and specifically generate_weights, since that function does not look at the data and, thus, cannot infer that a model run is "invalid" and should not be counted.

Example:

• I have NorESM1 data for RCPs 2.6, 4.5, 6.0 and 8.5, and for warming levels +2C, +3C and 4C. Every warming level is in the same dataset, with NaNs when a given experiment doesn't reach a given warming level.
• +2C is reached by all RCPs.
• +3C is reached by RCPs 6.0 and 8.5 only.
• +4C is reached by RCP8.5 only.

Current result:

• weights == [0.25, 0.25, 0.25, 0.25] for all warming levels.

Expected result

• weights == [0.25, 0.25, 0.25, 0.25] for +2C
• weights == [0, 0, 0.5, 0.5] for +3C
• weights == [0, 0, 0, 1] for +4C

Potential Solution

Add an argument that, when True, performs a isnull/count/whatever is fastest along the other dimensions (or a specific dimension?), puts a weight of 0 if all entries are NaN, and ignores those realizations when generating weights.

Since this will be much slower than the current method, I would make it optional.

Contribution

• I would be willing/able to open a Pull Request to contribute this feature.

[juliettelavoie]

I might even make is even more general. The new argument could be the name of the dimension we want to keep.