Type hierarchy for Angular, Spherical and Manifold distributions.

[simonbyrne]

This has come up in a couple of issues (#192, #223), so I thought I would propose creating some new abstract types:

• AngularDistribution <: Distribution{Univariate,Continuous}: e.g. von Mises distribution, wrapped normal. The problem is that they can be defined on any interval of length 2pi, so what I was thinking is that we define pdf over the whole real line, such that the integral is 1 over any such interval. We could define cdf and rand on [0,2pi].
• SphericalDistribution <: Distribution{Multivariate,Continuous}: e.g. Fisher-von Mises, Bingham, etc. We need to be a bit careful in how we deal with support (i.e. allow for some error in the condition norm(x) == 1).
• StiefelDistribution <: Distribution{Matrixvariate,Continuous}: MatrixBingham etc. for distributions over orthogonal matrices.
• perhaps also PDMatDistribution <: Distribution{Matrixvariate,Continuous} for Wishart and InverseWishart?

[simonbyrne]

Actually, having thought more about this, these should be new "support" types (instead of Continuous). My rationale is that Continuous support really means "absolutely continuous with respect to the Lebesgue measure", and Discrete means "absolutely continuous with respect to the counting measure on the integers". These new types would then mean that the distributions are absolutely continuous wrt to the uniform measure on the relevant spaces.

I would also propose that we provide an "unchecked" method for computing the densities for computational efficiency: perhaps pdf_unsafe?

Also, one more to consider would be Dirichlet <: Distribution{Multivariate,Simplex}. However strict adherence to this standard would require a dividing the usual density by sqrt(d), which could scare/confuse people.

[lindahua]

My concern is that introducing a new type for each kind of manifold would complicate things a lot, which may not be necessary from the standpoint of API design.

[simonbyrne]

@lindahua I concede that it would add some complications, though I think the benefits outweigh the risks. The main advantage is that it would allow us to leverage dispatch. Some examples:

• we would only need to define insupport on the abstract type.
• for MCMC, you need a proposal scheme that generates samples in the support of the distribution. e.g. a random-walk metropolis scheme on a Continuous can just add a randn(), on a Discrete can move by integer amounts, on an Angular can add a randn(), then mod2pi, Spherical can do rotations, etc.

[lindahua]

Maybe we should introduce an abstract type SampleSpace and add a bunch of subtypes.

[simonbyrne]

@lindahua That might be a better approach. If we're going to do that, it would also be useful to have different types to distinguish between distributions defined on the whole real line, and distributions defined on the positive reals, and distributions defined on compact intervals.