Conditional prediction can make predictions worse?

[smithja16]

Hi there,

I'm having an unexpected issue where the predictive performance of my joint conditional predictions is often considerably worse than non-conditional predictions. I condition on ~half my species to predict the other half. I use a hurdle model, and predict each component separately. The binomial component behaves as expected - the conditional predictions are about the same or slightly better. But the lognormal component will vary wildly (for some species, not all) between conditional and non-conditional predictions - see figure.

I did not expect joint conditional prediction to make predictions worse, but I've found limited explanation of how the process works for distributions other than binomial. Does this mean that the estimated residual corrections are uncertain, or that the 'known' species abundances are poorly represented by the model's expected value? I've tried conditioning on species with residual correlations with higher posterior support, but this makes little difference. What is it about 'continuous' distributions that seem to allow this behaviour from conditional prediction?

Thanks!

Figure shows true abundance (black line), joint prediction (yellow line), and joint conditional prediction (blue line)