[Numpy2] Preprocessor distance_metric returns value 0 instead of -- (masked element) with numpy==2.0.0 for metrics that use np.sqrt or da.sqrt

[valeriupredoi]

• test: tests/unit/preprocessor/_compare_with_refs/test_compare_with_refs.py
• test result: when an all-masked array is used as data input, distance_metric returns data element 0 instead of masked or --, with numpy=2 and certain (not all!) metrics
• see https://github.com/ESMValGroup/ESMValCore/actions/runs/9584890389/job/26429529577?pr=2395
• Output of interest:

FAILED tests/unit/preprocessor/_compare_with_refs/test_compare_with_refs.py::test_distance_metric[weighted_pearsonr-nan-1.0-Pearson's r-pearsonr_tas-1] - AssertionError: 
FAILED tests/unit/preprocessor/_compare_with_refs/test_compare_with_refs.py::test_distance_metric_masked_data[emd-1.98625-0.0-EMD-emd_tas-K-True] - AssertionError: 
FAILED tests/unit/preprocessor/_compare_with_refs/test_compare_with_refs.py::test_distance_metric_fully_masked_data[rmse-2.34520788-0.0-RMSE-rmse_tas-K-True] - AssertionError: 
FAILED tests/unit/preprocessor/_compare_with_refs/test_compare_with_refs.py::test_distance_metric_fully_masked_data[rmse-2.34520788-0.0-RMSE-rmse_tas-K-False] - AssertionError: 
FAILED tests/unit/preprocessor/_compare_with_refs/test_compare_with_refs.py::test_distance_metric_fully_masked_data[weighted_rmse-2.0-0.0-RMSE-rmse_tas-K-True] - AssertionError: 
FAILED tests/unit/preprocessor/_compare_with_refs/test_compare_with_refs.py::test_distance_metric_fully_masked_data[weighted_rmse-2.0-0.0-RMSE-rmse_tas-K-False] - AssertionError:

could be Scipy could be something else, this needs a bit of due dilligence 🍺

[valeriupredoi]

This is a bug introduced with Numpy=2.0: in

ESMValCore/esmvalcore/preprocessor/_compare_with_refs.py

Line 454 in d6eaac2

rmse = npx.sqrt(npx.ma.average(squared_error, axis=axis, weights=weights))

the sqrt is not computed correctly in the case of masked arrays: one needs to account both for Numpy and Dask:

    npx = get_array_module(squared_error)

    # need masked sqrt for numpy >=2.0
    # and dask.array.reductions.safe_sqrt for Dask
    # otherwise results will be computed ignoring masks
    if npx.__name__ == "dask.array":
        da_squared_error = npx.ma.average(squared_error,
                                          axis=axis,
                                          weights=weights)
        rmse = npx.reductions.safe_sqrt(da_squared_error)
    else:
        rmse = npx.ma.sqrt(
            npx.ma.average(squared_error, axis=axis, weights=weights)
        )

this is fixed in #2395

ie if np.sqrt() ignores masks then da.sqrt() will too, see numpy/numpy#25635 that may be fixed, or not - the issue is pretty old now wrt Numpy2 timeline, so better we have the patch in our code