TestNumpyFunc3.test_interp is flakey

[drew-parsons]

Debian CI has run tests of the debian experimental build of pythran 0.11.0+ds-2. TestNumpyFunc3.test_interp shows up as flakey, in the 7c, 7, 6c and 8c variants. Occasionally the test passes, depending also on which BLAS implementation is being used.

Test logs can be found at https://ci.debian.net/packages/p/pythran/ in the unstable links for each arch. The version I'm reporting are the 0.11.0+ds-2 tests, taking src:pythran from experimental (I'll shortly load up 0.11.0+ds-3 to debian unstable, which will "pass" by skipping test_interp)

I note the error shows mostly in the complex variants, with the discrepancy in the imaginary part.
Here is a summary of the failures:

amd64

pythran-openblas

TestNumpyFunc3.test_interp_8c
test_numpy_func3.py:813:
__init__.py:147: in assertAlmostEqual
    self.assertLessEqual(abs(e) / sys.float_info.epsilon, 3.)
E   AssertionError: 32.0 not less than or equal to 3.0
Python result:  (0.42774179176426386-0.009130985223261723j)
Pythran result:  (0.4277417917642641-0.009130985223261834j)
    
    
arm64

actually it passed all tests second time round
first attempt:
TestNumpyFunc3.test_interp_6c
test_numpy_func3.py:799:.
    self.assertLessEqual(abs(e) / sys.float_info.epsilon, 3.)
E   AssertionError: 8.5 not less than or equal to 3.0
Python result:  (-0.10416069757759507-0.04432392677299891j)
Pythran result:  (-0.10416069757759507-0.04432392677299879j)


armhf

upstream-tests
TestNumpyFunc3.test_interp_7c (py3.10)
test_numpy_func3.py:806:.
    self.assertLessEqual(abs(e) / sys.float_info.epsilon, 3.)
E   AssertionError: 16.0 not less than or equal to 3.0
Python result:   (0.9140634758789765-0.020893787344676396j)
Pythran result:  (0.9140634758789763-0.020893787344676507j)


i386

TestNumpyFunc3.test_interp_7
    self.assertLessEqual(abs(e) / sys.float_info.epsilon, 3.)
E   AssertionError: 3.5 not less than or equal to 3.0
Python result:  -0.1723323050796359
Pythran result:  -0.1723323050796357


ppc64el

TestNumpyFunc3.test_interp_7c
test_numpy_func3.py:806:.
    self.assertLessEqual(abs(e) / sys.float_info.epsilon, 3.)
E   AssertionError: 5.0 not less than or equal to 3.0
Python result:  (0.09563233312896172+0.8559554545778792j)
Pythran result:  (0.09563233312896158+0.8559554545778792j)


s390x

generic blas
TestNumpyFunc3.test_interp_7c
test_numpy_func3.py:806:.
    self.assertLessEqual(abs(e) / sys.float_info.epsilon, 3.)
E   AssertionError: 7.5 not less than or equal to 3.0
Python result:  (0.04630951688408735+0.012376385953737155j)
Pythran result:  (0.04630951688408724+0.012376385953736913j)

[drew-parsons]

Also test_interp_6, _7 and _8 (real number variants) fail on i386.

[drew-parsons]

More broadly than test_interp, pythran breaks other tolerance conditions in tests using scipy on nonstandard architectures. #2002 notes some tolerance failures on armhf. statsmodels fails test_mle with scipy 1.8.1 and pythran 0.11.0 on s390x