[BUG] Flaky pylibraft distance test

[cjnolet]

We've seen a flaky distance test fail several times now in pylibraft. It only seems to fail in one of the configurations and it seems to pass if re-run enough times (though I've seen it fail twice in a row recently).

Here's an example of the failing test configuration (it always seems to tbe for the cosine distance):

=========================== short test summary info ============================
FAILED test/test_distance.py::test_distance[float64-F-False-cosine-40-100] - assert False
 +  where False = <function allclose at 0xfffee4bec160>(array([[0.        , 0.28490248, 0.20478542, ..., 0.31273605, 0.19879491,\n        0.19658006],\n       [0.28490248, 0.  ...  ,\n        0.17852762],\n       [0.19658006, 0.17743437, 0.16078043, ..., 0.30887233, 0.17852762,\n        0.        ]]), array([[0.        , 0.28490248, 0.20478542, ..., 0.31273605, 0.19879491,\n        0.19658006],\n       [0.28490248, 0.  ...  ,\n        0.17852762],\n       [0.81601286, 0.8116284 , 0.80781457, ..., 0.30887233, 0.17852762,\n        0.        ]]), rtol=0.0001)
 +    where <function allclose at 0xfffee4bec160> = np.allclose
============ 1 failed, 1091 passed, 24 skipped, 4 xfailed in 42.90s ============

This is taken from this CI log