Flaky test: pymc3/tests/test_distributions_random.py::TestMvNormal::test_with_cov_rv

[michaelosthege]

It works on my machine.

Failed on master CI: https://github.com/pymc-devs/pymc3/runs/1585173690#step:8:2326

The message is the same as in this other flaky tests #4346

ValueError: 0-dimensional argument does not have enough dimensions for all core dimensions ('i_0_0',)

Trackeback:

``` =================================== FAILURES =================================== ____________ TestMvNormal.test_with_cov_rv[(10, 3)-(10, 3)-(1, 1)] _____________

self = <pymc3.tests.test_distributions_random.TestMvNormal object at 0x7f49b5c17d90>
sample_shape = (10, 3), dist_shape = (10, 3), mu_shape = (1, 1)

@pytest.mark.parametrize(
    ["sample_shape", "dist_shape", "mu_shape"],
    generate_shapes(include_params=False),
    ids=str,
)
def test_with_cov_rv(self, sample_shape, dist_shape, mu_shape):
    with pm.Model() as model:
        mu = pm.Normal("mu", 0.0, 1.0, shape=mu_shape)
        sd_dist = pm.Exponential.dist(1.0, shape=3)
        chol, corr, stds = pm.LKJCholeskyCov(
            "chol_cov", n=3, eta=2, sd_dist=sd_dist, compute_corr=True
        )
        mv = pm.MvNormal("mv", mu, cov=pm.math.dot(chol, chol.T), shape=dist_shape)

      prior = pm.sample_prior_predictive(samples=sample_shape)

pymc3/tests/test_distributions_random.py:1637:

---

pymc3/sampling.py:1919: in sample_prior_predictive
values = draw_values([model[name] for name in names], size=samples)
pymc3/distributions/distribution.py:790: in draw_values
value = draw_value(next, point=point, givens=temp_givens, size=size)
pymc3/distributions/distribution.py:954: in _draw_value
return param.random(point=point, size=size)
pymc3/model.py:104: in call
return getattr(self.obj, self.method_name)(*args, **kwargs)
pymc3/distributions/multivariate.py:262: in random
mu, param = draw_values([self.mu, param_attribute], point=point, size=size)
pymc3/distributions/distribution.py:833: in draw_values
value = _draw_value(param, point=point, givens=givens.values(), size=size)
pymc3/distributions/distribution.py:1005: in _draw_value
output = func(*input_vals)
/usr/share/miniconda/envs/pymc3-dev-py38/lib/python3.8/site-packages/numpy/lib/function_base.py:2108: in call
return self._vectorize_call(func=func, args=vargs)
/usr/share/miniconda/envs/pymc3-dev-py38/lib/python3.8/site-packages/numpy/lib/function_base.py:2182: in _vectorize_call
res = self._vectorize_call_with_signature(func, args)
/usr/share/miniconda/envs/pymc3-dev-py38/lib/python3.8/site-packages/numpy/lib/function_base.py:2211: in _vectorize_call_with_signature
broadcast_shape, dim_sizes = _parse_input_dimensions(
/usr/share/miniconda/envs/pymc3-dev-py38/lib/python3.8/site-packages/numpy/lib/function_base.py:1873: in _parse_input_dimensions
_update_dim_sizes(dim_sizes, arg, core_dims)

---

dim_sizes = {}, arg = array(0.), core_dims = ('i_0_0',)

def _update_dim_sizes(dim_sizes, arg, core_dims):
    """
    Incrementally check and update core dimension sizes for a single argument.

    Arguments
    ---------
    dim_sizes : Dict[str, int]
        Sizes of existing core dimensions. Will be updated in-place.
    arg : ndarray
        Argument to examine.
    core_dims : Tuple[str, ...]
        Core dimensions for this argument.
    """
    if not core_dims:
        return

    num_core_dims = len(core_dims)
    if arg.ndim < num_core_dims:

      raise ValueError(

            '%d-dimensional argument does not have enough '
            'dimensions for all core dimensions %r'
            % (arg.ndim, core_dims))

E ValueError: 0-dimensional argument does not have enough dimensions for all core dimensions ('i_0_0',)

/usr/share/miniconda/envs/pymc3-dev-py38/lib/python3.8/site-packages/numpy/lib/function_base.py:1836: ValueError
=============================== warnings summary ===============================
pymc3/tests/test_distributions_random.py: 36 warnings
/home/runner/work/pymc3/pymc3/pymc3/distributions/discrete.py:1157: DeprecationWarning: Constant has been deprecated. We recommend using a Deterministic object instead.
warnings.warn(

pymc3/tests/test_distributions_random.py::TestScalarParameterSamples::test_lkj
pymc3/tests/test_distributions_random.py::TestScalarParameterSamples::test_lkj
pymc3/tests/test_distributions_random.py::TestScalarParameterSamples::test_lkj
/home/runner/work/pymc3/pymc3/pymc3/distributions/multivariate.py:1356: DeprecationWarning: Parameters in LKJCorr have been rename: shape parameter n -> eta dimension parameter p -> n. Please double check your initialization.
warnings.warn(

pymc3/tests/test_distributions_random.py: 12 warnings
/home/runner/work/pymc3/pymc3/pymc3/sampling.py:465: FutureWarning: In an upcoming release, pm.sample will return an arviz.InferenceData object instead of a MultiTrace by default. You can pass return_inferencedata=True or return_inferencedata=False to be safe and silence this warning.
warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/warnings.html

----------- coverage: platform linux, python 3.8.6-final-0 -----------

</details>

[Sayam753]

Ouch. I cannot reproduce it as well.
This looks similar to #4323. So, what's the plan of action to identify the source of bug?

[michaelosthege]

It's related to a shape, not to some random number generator stuff. The only reason I can think of is some kind of interference between tests, due to a caching problem in Theano.

Unless someone comes up with a reproducible example, I think the best strategy is to push ahead with the work on Theano-PyMC.
With the RandomVariable Op we'll throw out most of the predictive sampling code, and with it a ton of bugs like this one 🐛🐛,

[brandonwillard]

This error has just appeared in a recent run of #4471. In this case, it was TestMvNormal::test_with_chol_rv that failed in exactly the same way.

I have also tried to reproduce it locally—to no avail—on my more up-to-date Ubuntu machine by setting all the same Aesara/Theano-PyMC options (e.g. floatX=float32) and running the test repeatedly for the failing inputs (i.e. sample_shape = 3, dist_shape = (10, 3), mu_shape = (1, 1) in this case) under new RNG seeds.

[ricardoV94]

I believe this test has been refactored/replaced in V4. Closing it.