Transformed random variables initialisation

[emilemathieu]

Hi all !

In HMC, VarInfo is initialised with vi = model() which consequently calls the default assume, then init(dist), then inittrans(dist) if dist is a subtype of TransformDistribution.

I don't understand why don't we sample from the prior (rand(dist)) for initialisation, instead of this uniform sampling (through some mapping) which can be quite far from the prior (and posterior). Do we need this for the transformations in HMCto be valid, or is this for performance ?

Maybe it's to allow to perform inference when we can only evaluate pointwise distributions' pdf and cannot sample from them ?

Best,
Emile

[emilemathieu]

@yebai do you have an answer for this ?

[yebai]

This is for better numerical stability for the HMC sampler- we initially used samples from the prior as initialisation, but for transformed variables such as Dirichlet, the gradient often vanishes. This is also what the HMC sampler in Stan does by default

[xukai92]

Yes, it's mainly about numerical stability. Note for transformed variables, we directly initialise them in their transformed space in stead of original space, which ensures that their initialisation gives no error in their transformed space.

[emilemathieu]

Thanks for your answers !
How could I know whether such a transformation is needed for Distributions.rand(d::SBS_DP) = d.T_surplus*rand(Beta(1, d.alpha)) ?

I overrode this initialisation for this distribution (1fb6ab8) since the samples were far away from the prior (d.alpha) wasn't taken into account.