pdfs up to proportionality

[simonbyrne]

In many cases, we don't need to compute all the factors of a pdf/logpdf:

• In likelihoods, we can ignore any factors not involving the parameters (e.g. the factorial in the Poisson)
• Many sampling techniques don't require computing the normalisation constant, i.e. factors not involving the point at which the density is evaluated (e.g. the Bessel function in the von Mises distribution)
• Posteriors are typically only known up to proportionality, so we would only need the prior (ignoring the normalisation constant), and the likelihood (ignoring non-parameter factors)

It would be good to have a consistent interface for all of this: we might also want to think about how this relates to autodiff functionality.

[johnmyleswhite]

How about something like updf?

[andreasnoack]

In some of the algorithms for random number generation, for efficiency reasons, some numbers should only to be calculated once given the parameter values. In C you could use static variables to store the information, but I don't know the best solution in Julia. At some point I experienced with generating these numbers, at the time the random variable object was constructed. That solution could be an alternative to an un-normalised pdf.

[StefanKarpinski]

Possibly related might be the idea of non-normalized distributions which might be useful for working with mixture models. In particular, signed, non-normalized distributions (i.e. finite signed measures) form a vector space.

[johnmyleswhite]

They form a vector space? How do you handle multiplication by large negative constants?

[StefanKarpinski]

That's what signedness allows. See finite signed measure.

[sbos]

I would also like to have standard interface for normalizing constants, e.g. lognorm and norm

[lindahua]

For exponential family distributions, the normalization constant has a name: partition function.

lognorm and norm would be easily confused with the typical meaning of norms.

[samuela]

+100 This is really important.

@sbos It's also worth noting that in many cases calculating the partition function is analytically and/or computationally intractable. This doesn't generally affect the standard distributions that the package targets, but it may be worth keeping in mind as the project evolves.

[lindahua]

I am going to implement this (particularly for exponential family distributions). Will make a proposal with RFC pretty soon.

[simonbyrne]

Having thought more about this, I decided what was really needed is a ratio of pdfs, so i was thinking:

pdfratio(d1::Distribution, d2::Distribution, x::Real) = pdf(d1,x) / pdf(d2,x)
pdfratio(d::Distribution, x1::Real, x2::Real) = pdf(d,x1) / pdf(d,x2)
pdfratio(d1::Distribution, d2::Distribution, x1::Real, x2::Real) = pdf(d1,x1) / pdf(d2,x2)