🚧 Forward port Improve result comparison workflow and v0.4 changelog (#…

…938)

* 🚧 Forward port of #936 test_result_consistency

🚇Update test_result_consistency (#936)

* 🚇Update test_result_consistency
Update float_resolution for comparison of spectral values.
Should resolve issues with result comparison signaled in #927

* 🚇👌 Further refinement of test_result_consistency
Refine calculation of epsilon for residual check
Should resolve issues with result comparison signaled in #896

* 🚇👌 Failing tests now also show the tolerances used for the comparison
d2bff4

* 📚 Forward port changelog from v0.4 maintenance branch

.github/test_result_consistency.py

@@ -180,7 +180,7 @@ def data_var_test(
     eps = np.finfo(np.float32).eps
     rtol = 1e-5  # default value of allclose
     if expected_var_name.endswith("residual"):  # type:ignore[operator]
-        eps = expected_result["data"].values.max() * 1e-8
+        eps = max(eps, expected_result["data"].values.max() * eps)
 
     if "singular_vectors" in expected_var_name:  # type:ignore[operator]
         # Sometimes the coords in the (right) singular vectors are swapped
@@ -211,6 +211,11 @@ def data_var_test(
             np.abs(eps * expected_values_scaled),
             np.ones(expected_var_value.data.shape) * eps,
         )
+    elif "spectra" in expected_var_name:
+        float_resolution = np.maximum(
+            np.ones(expected_var_value.data.shape) * eps * np.max(np.abs(expected_var_value.data)),
+            np.ones(expected_var_value.data.shape) * eps,
+        )
     else:
         float_resolution = np.maximum(
             np.abs(eps * expected_var_value.data),
@@ -230,6 +235,7 @@ def data_var_test(
         f"{float(np.sum(abs_diff))} and shape: {expected_var_value.shape}\n"
         "Mean difference: "
         f"{float(np.sum(abs_diff))/np.prod(expected_var_value.shape)}\n"
+        f"Using: \n - {rtol=} \n - {eps=} \n - {float_resolution=}"
     )
 
     coord_test(

changelog.md

@@ -77,6 +77,28 @@
 - 🧹 Move megacomplex integration tests from root level to megacomplexes (#894)
 - 🩹 Fix artifact download in pr_benchmark_reaction workflow (#907)
 
+## 0.4.2 (2021-12-12)
+
+### 🩹 Bug fixes
+
+- 🩹🚧 Backport of bugfix #927 discovered in PR #860 related to initial_concentration normalization when saving results (#935).
+
+### 🚧 Maintenance
+
+- 🚇🚧 Updated 'gold standard' result comparison reference ([old](https://github.com/glotaran/pyglotaran-examples/commit/9b8591c668ad7383a908b853339966d5a5f7fe43) -> [new](https://github.com/glotaran/pyglotaran-examples/commit/fc5a5ca0c7fd8b224c85027b510a15717c696c7b))
+- 🚇 Refine test_result_consistency (#936).
+
+## 0.4.1 (2021-09-07)
+
+### ✨ Features
+
+- Integration test result validation (#760)
+
+### 🩹 Bug fixes
+
+- Fix unintended saving of sub-optimal parameters (0ece818, backport from #747)
+- Improve ordering in k_matrix involved_compartments function (#791)
+
 ## 0.4.0 (2021-06-25)
 
 ### ✨ Features