Logp inference fails for exponentiated distributions

[jessegrabowski]

Description

Suppose I want to make a log-normal "by hand":

def d(mu, sigma, size=None):
    return pt.exp(pm.Normal.dist(mu=mu, sigma=sigma, size=size))

with pm.Model() as m:
    y = pm.CustomDist('y', 0, 1, dist=d, observed=[0., 1.])

m.logp().eval() # array(nan)

This should return -inf, since 0 is outside the support of the (exponential) distribution. The same is true for any other support-constraining transformation.

Non-trivial use-case would be in a mixture model with components LogNormal and DiracDelta(0). This works fine, but if I want to explore a more fat-tailed distribution for the nonzero component (like LogStudentT), it fails.

[ricardoV94]

mixture model with components LogNormal and DiracDelta(0)

This is not a mixture (mass vs density don't go well together), not sure why you mentioned it.

Re bounds, They are always a bit tricky specially when you chain multiple transformations. However I thought we were returning ninf for this case. Is it also nan for negative numbers?

[jessegrabowski]

This is not a mixture (mass vs density don't go well together), not sure why you mentioned it.

That's the mathematical form of any zero-inflated model, why wouldn't I mention it?

Negative numbers return -inf, so it's specifically only for zero.

[ricardoV94]

zero inflated model is not a mixture it's two likelihoods. You always know which component a value belongs to add the expression doesn't involve weighting the components (you can't weigh the density and the pmf, the pmf has infinity density at the support point I guess).

Re zero, transforms right now rely on jacobian (just a heck) to try to enforce the domain. When it returns nan it's considered outside of the donation. I guess the log transform jacobian doesn't return Nan for zero, but ninf?

[jessegrabowski]

Bruh.

Where should I be looking to answer the question that actually matters? Here I don't see any kind of special logic. pt.log(0) returns -inf "natively", which is all that seems to be implicated.

[ricardoV94]

Those are hurdle models and rely on truncating the continuous component so it excludes zero. It's a hack to recycle mixture functionality, we should implement something specifically for it.

Requiring the truncation adds significant cost to the logp.

Re zero, should we consider logp zero when log det of jacobian returns -inf as well, now we just consider nan

[ricardoV94]

Sorry all you showed are discrete which is fine, those are mixtures. But we have hurdle classes that seems to be what you are looking for. Those are not mixtures but we use mixture with the truncation trick under the hood

[ricardoV94]

Here is the line I'm talking about

https://github.com/pymc-devs/pymc/blob/main/pymc%2Flogprob%2Ftransforms.py#L227