[BNP] Bayesian Nonparametrics project

[emilemathieu]

This is a macro issue to follow the development of Turing being able to handle nonparametric models.

• Defining the class of models we aim to handle: Q-class cf [BNP] Q-class #374

  • Implement DP, PYP via stick breaking: Here
  • Q-class total mass: rand(ExpTiltedSigma(sigma, tau)): Here
  • Q-class total mass: logpdf(ExpTiltedSigma(sigma, tau))
  • Q-class new weight with sigma=0.5: rand : Here
  • Q-class new weight with sigma=0.5: logpdf
  • Q-class new weight with sigma<0.5: rand and logpdf

• Choose & implement model / metrics for the tests cf [BNP] Model & metric for test #371

• Infinite array data structure for weights and locations of the RPM

  • Can use tzeros(Float64, N) with N being the number of datapoints for both locations and weights.
  • Can use Dict{Int, Any}() for locations ?
  • Need a specific data structure InfiniteArray ?

• SMC working on those models which implies handling array of random length cf Handling several vectors of random variables with random size #367 (comment)

  • Chain wrapper issue [Chain] flatten fails with different random variables in samples #386
  • cache issue for getidcs

• CSMC working

  • Bug: missing prior of conditioned random variables in particles-weight [Particle Gibbs] [Bug] Missing global parameter prior in particles weight #383
  • Handling random variables sampled in random order cf Handling several vectors of random variables with random size #367 (comment). Solutions proposed Handling several vectors of random variables with random size #367 (comment)
  • Unnecessary operation in forkc cf Trace Replay #358 (comment) ?
  • Unknown random variables that do not belongs to CSMC's space cf [CSMC] Case where random variable is unknown and belongs to a different sampler #373

• HMC working on those models.

  • Computing gradient cf [HMC] [IMM] gradient issue #375
  • Initialisation of random variables in HMC cf Transformed random variables initialisation #368 => override init for SizeBiasedSamplingDistribution with rand(dist) 1fb6ab8
  • Do not update unused clusters in HMC ? cf [HMC] Update random variables independent of the joint likelihood ? #372

• Combining samplers:

  • Particle Gibbs: Gibbs(N, CSMC(:z), HMC(:a, :p), HMC(...) )
  • Pseudo marginal: PMMH(N, SMC(...), MH(...))

• New syntax: Avoid having to write the all stick-breaking / stick-biased process (by using a macro for instance)

[emilemathieu]

I created a branch with a proof of concept: https://github.com/yebai/Turing.jl/blob/project-bnp/test/rpm.jl/imm.jl.

[trappmartin]

Hi,
As discussed with @yebai I'll be looking into approaches to store parameters of RPMs in the Mamba.Chain struct within the next weeks. I'll keep you informed about the progress.

Cheers,
Martin

[yebai]

@emilemathieu @ae-foster @mlomeli1 Just to add, @trappmartin is a new member on the Turing team. He has previously implemented https://github.com/trappmartin/BNP.jl and now helping us to get BNP integrated into Turing:

• resolve Abandoning "Mamba.Chain" #413 first
• create a working example of DPmix

[emilemathieu]

Hi @trappmartin ! Great news !
I can't exactly remember where we were in https://github.com/TuringLang/Turing.jl/tree/project-bnp, but I believe I had a SMC and Gibbs sampler working on a PYP and NIGP mixture model.

We could have a skype session to talk about your plans, and if you have questions on what we did on that matter.

Cheers,
Emile

PS: @yebai we had the above mentioned examples working but not in the master branch indeed.
PS2: @trappmartin @yebai Also at the time there was still a cache issue with getidcs in VarInfo, that I hard fixed by commenting some lines which is obviously damaging performances. Something was wrong also I believe with checkgid in VarInfo.

[mlomeli1]

Hi @trappmartin , that's great news! If you have any questions I am happy to talk too.

[trappmartin]

Hi,
Thanks a lot for the warm welcome.

I’m happy to talk via Skype once I have a better overview of the codes.

[yebai]

I’m happy to talk via Skype once I have a better overview of the codes.

@trappmartin the current code is not super well documented, so pls feel free to ask any questions related to the code.