Add entropy dispatch for distributions

[uditjainstjis]

Description

Adds an entropy dispatcher under pymc.distributions.moments, mirroring the existing mean dispatch (#7530). entropy(rv) returns the differential entropy for continuous distributions and the Shannon entropy (in nats) for discrete ones, following the scipy.stats convention.

Closed-form entropies are registered per RV Op for the univariate distributions — Normal, Uniform, Exponential, Laplace, AsymmetricLaplace, Cauchy, HalfCauchy, HalfNormal, Gamma, InverseGamma, Beta, Logistic, LogNormal, Gumbel, Pareto, Weibull, StudentT, Triangular, Moyal, VonMises, Bernoulli, Geometric, DiscreteUniform, Categorical — plus MvNormal and Dirichlet. ChiSquared is covered automatically via its Gamma parametrization. Distributions without a closed-form entropy remain unregistered and raise NotImplementedError.

This keeps the helper self-contained in PyMC with no new dependency on pytensor-distributions, consistent with @ricardoV94's note in the issue that these derived stats "will remain PyMC specific", and with how mean (#7530) is already implemented. Happy to reroute through the distributions package instead if that's preferred — see my comment on the issue. Opening as a draft since the issue was parked pending the (now-shipped) v0 release.

Related Issue

• Closes Implement entropy dispatches for distributions #8085

Checklist

• Checked that the pre-commit linting/style checks pass (ruff check/format, mypy — no new errors)
• Included tests that prove the new feature works (every value verified against scipy.stats's entropy; batched/tiled cases and the NotImplementedError path covered)
• Added necessary documentation (module + function docstrings)

Type of change

• New feature / enhancement

[welcome]

]
💖 Thanks for opening this pull request! 💖 The PyMC community really appreciates your time and effort to contribute to the project. Please make sure you have read our Contributing Guidelines and filled in our pull request template to the best of your ability.

[read-the-docs-community]

Documentation build overview

📚 pymc | 🛠️ Build #33736160 | 📁 Comparing 329b038 against latest (3b661c7)

  🔍 Preview build  

1 file changed

± glossary.html

[uditjainstjis]

Checking in on this one — it's been parked on the design question of whether the entropy implementations belong here or in a separate distributions package.

Is there a decision I should build against, or something smaller I could split out in the meantime? The 26 distributions are backed by 44 scipy-verified tests, so I'd rather land it in whatever shape you want than leave it sitting.

[ricardoV94]

I'd suggest the pytensor-distributions, we already have the POC PR happening in #8376