Define fallback implementations for mean, var, and entropy #1875
Conversation
We have the internal `expectation` function that uses `quadgk` to compute integrals, as well as a fallback implementation for `kldivergence` that uses `expectation`, so it seems reasonable to similarly define fallbacks for other quantities trivially computable using `expectation`: `mean`, `var`, and `entropy`. We could probably do `skewness` and `kurtosis` as well. NOTE: Skipping CI until I've actually done more with this since we run a lot of CI on this repo [ci skip]
|
Copied from #1874 (comment): My worry (that was also expressed in issues such as #968) is that generally numerical integration is challenging and a fallback might lead to silently incorrect results. It seems such a fallback would be wrong (or at least problematic) e.g. if the moments are not finite (such as e.g. for So my general feeling is that numerical integration should maybe be restricted to a smaller subset of distributions, or maybe even only be available as a separate function. In case we want to use it more broadly, I think it would also be safer to error if the integration error estimate is too large, to reduce the probability of silently incorrect results. |
| @@ -214,7 +214,7 @@ skewness(d::UnivariateDistribution) | |||
|
|
|||
| Compute the entropy value of distribution `d`. | |||
| """ | |||
| entropy(d::UnivariateDistribution) | |||
| entropy(d::UnivariateDistribution) = expectation(x -> -log(pdf(d, x)), d) | |||
There was a problem hiding this comment.
| entropy(d::UnivariateDistribution) = expectation(x -> -log(pdf(d, x)), d) | |
| entropy(d::UnivariateDistribution) = expectation(x -> -logpdf(d, x), d) |
pdf can overflow
|
Generally agree with @devmotion . I've hit a similar issue for |
|
Yep, fair enough. |
We have the internal
expectationfunction that usesquadgkto compute integrals, as well as a fallback implementation forkldivergencethat usesexpectation, so it seems reasonable to similarly define fallbacks for other quantities trivially computable usingexpectation:mean,var, andentropy. We could probably doskewnessandkurtosisas well.To do:
expectation-based fallbacks(n.b. I skipped CI on the initial commit purposefully as I haven't given this a ton of thought yet and wanted to see whether anybody was strongly for or against it before doing meaningful work here)