Changed Categorical to work with multidim p at the logp level reloaded

[lucianopaz]

"Reopening" #3383 with the dimshuffle comment from @junpenglao.

[lucianopaz]

@twiecki, issue #3141 (in particular this reply) shows a potential model where the no normalization condition could be dangerous. Maybe I could still force the normalization here for backwards compatibility. What do you think?

[twiecki]

@lucianopaz I think auto-normalization is a good idea.

[lucianopaz]

@lucianopaz I think auto-normalization is a good idea.

Ok, this change means that the sumto1 check can be dropped.

[lucianopaz]

Ok, I think this should now be ready, @twiecki @junpenglao . In the end

• Categorical.p is not normalized inside __init__.
• Categorical.logp checks the raw p for negative values and returns -inf if any p entry is negative (does not include zero p).
• Categorical.logp will always normalize the elements of p to sum 1.
• Categorical.mode and hence its associated default test points will have shape equal to p.shape[:-1] . If this shape were (1,), it is squeezed.

These changes end up reaching many open issues:

• Wrong test values in Categorical distribution #3156 now raises an error if shape is not provided to the Categorical. When shape is given, the issue is now resolved. I think this means that we can close this issue
• In the discussion of Zero probability in Categorical distribution #3272, @junpenglao points out that at that time we did not allow elements of p equal to 0. The present code allows for zero entries, and logp returns -inf if a values accesses a zero probability category. Is this problematic? The snippet posted by @zplizzi still is stuck at the initial value, but this seems to be a problem of the step method not being able to cope with -inf logps.
• pm.Categorical behaves differently in a model versus as pm.Categorical.dist #3141 points out an ambiguity between Categorical.dist.random, and Categorical and then sample. This ambiguity is no longer a problem because if the shape is not provided, the dist will raise an error. We also now normalize always, so all the other snippets work as back then. I think that we can close this issue after merging this PR.
• Trouble sampling from Dirichlet with multiple groups #1153 now samples in under 10 seconds, so we can close it.
• Categorical distribution bound not enforced early enough #1093 seems to have already been fixed with the tt.clip(...), which were already in place before this PR.

[twiecki]

This is fantastic, thanks for the careful work, also in identifying the appropriate issues.