[Particle Gibbs] [Bug] Missing global parameter prior in particles weight

[emilemathieu]

In CSMC when used within a Gibbs sampler, particles' weight should include (log)priors of conditioned random variables.

For instance in the following mixture model, normal random variables :mu1 and :mu2 logpdfs should be added in particles' weight. Fortunately for this particular case, all particles are missing the same value which consequently does not bias the resampling step. Yet for models where the number of conditioned random variable used in the CSMC step is random, this is invalid.

[emilemathieu]

Proposition of fix: #384

[emilemathieu]

@twgr would you have a simple model that should fail with the current implementation, so as to write a test ?

[twgr]

Here is the right up I did of the problem last night which includes a suggestion about a test.

At present then a CSMC sweep simply ignores any of the global variables
in terms of the probabilistic semantics. This is fine when they are
defined unconditionally in the model (hence the tests have been passing)
but incorrect when they are sampled conditioned on other variables.
When this is the case there are extra likelihood terms - we need to
factor the trace probability using the old assume statement. For
example, if we a model that defines p(x1,x2,y1,y2) =
p(x1)p(y1|x1)p(x2|x1)p(y2|x1,x2) and we do a CSMC sweep fixing x2 then
we are targeting p(x1|x2,y1,y2) \propto p(x1,x2,y1,y2) and so we still
need the old p(x2|x1) term which is now a likelihood. Note that this
has slipped through the net because if you instead fix x1, then the
p(x1) term is a constant across particles and so can be ignored. In
general though we cannot as p(x2|x1) is different for each particle
because of having different x1s.

We should probably also add a test to confirm this case. It is
interesting that the gibbs.jl/gibbs.jl test does not catch the bug as it
should manifest for this problem. My hypothesis for what is happening
here is that even with the bug you might concevieably get the correct
marginal distributions for the variables, but not the right joint
distribution - each gibbs could actually end up targeting the marginal
rather than the conditional. This needs more investigation though

[emilemathieu]

Indeed the test shouldn't pass, I'll investigate.

Otherwise would you have a model stressing this error @twgr ?

[emilemathieu]

So the test indeed didn't pass, because check_numerical is only printing the difference between empirical and analytical means and is not a true test (returning an error). I have no idea why such a function is used.

In the PR #384 I fixed it by using the macro @test (which is used in most tests). With the fix this test now pass ! 🙌

[yebai]

So the test indeed didn't pass, because check_numerical is only printing the difference between empirical and analytical means and is not a true test (returning an error). I have no idea why such a function is used.

check_numerical was supposed to only for visual inspection for failed tests. The reason that numerical tests didn't throw an error is that these tests sometimes fail stochastically even the code is correct. In the end, what we did was moving most numerical tests on Travis to builds that allow failures (e.g. builds like Bench on Travis).

[emilemathieu]

I think a good way to do it is to that check locally that the test pass most of the time. Then fix the stochasticity with srand, check that it still passes and then push.