Should we have a context to indicate that we're not performing inference?

[torfjelde]

Sometimes we know that we're not performing inference in a model, in which case it can be useful to have the model perform some additional computations.

For example, people often end up using Dirac to save quantities that are deterministic when conditioned on the variables in a Chains (e.g. TuringLang/Turing.jl#2059), but this can then cause issues since samplers will also consider these as random variables to be sampled. generated_quantities provides a partial solution to this, but

1. It doesn't have the same behavior as ~, which people often want.
2. It requires an additional pass over the chain after sampling.

Addressing (2) is non-trivial as it requires changing both the internals of varinfos in addition to how transitions are constructed in Turing.jl.

But addressing (1) could be done by adding a simple NotInferringContext + a macro @isinferring, which simply translates into a call isinferring(__context__, __varinfo__) or something. We could then either provide an alternative to generated_quantities which makes use of this, or we could include this in generated_quantities somehow (I'm leaning towards the former).

An alternative way of addressing (1) is to just provide a convenient way to convert a Matrix{<:NamedTuple} into a Chains, which I have existing code to do, but the annoying bit with this is that it's "asking a lot" of the user (they need to explicitly filter out the variables in return they don't want in the chain, and then also make sure to return a NamedTuple, etc.). Hence I think I'm leaning towards supporting both approaches 🤷

[SamuelBrand1]

Bumping this in the context of prediction.

[torfjelde]

It was attempted in #589 but got downvoted for now, as it was deemed undesirable to add more macros (see discussion in that PR)

[SamuelBrand1]

A thing to note is that if you want to do within chain parallel then this is a handy feature.

A reason is that its much easier to use @addlogprob! in patterns with (say) @threads over a loop of data observations; but then you sacrifice the generative quality of the Turing PPL. Having an "Not doing inference right now" context squares that circle.

[torfjelde]

Indeed. I've had to use similar approaches in the past due to the computational complexity of the model.

@yebai worth revisiting #589 ?

[yebai]

over a loop of data observations; but then you sacrifice the generative quality of the Turing PPL. Having an "Not doing inference right now" context squares that circle.

Not sure I understand this. Can you clarify what you mean by "sacrifice the generative quality of the Turing PPL"? @SamuelBrand1

[torfjelde]

I believe @SamuelBrand1 means that you can drop the usage of ~, which loses the niceties that it brings, e.g. sample from prior, in favour of the faster @addlogprob! path during inference (like we did extensively in the Epimap project to make it work @yebai )

[SamuelBrand1]

Just saw this, but yes exactly!

[yebai]

@torfjelde and I have some further discussion today. While I agree this is a useful feature, I am not yet convinced there is a safe and robust design.

I suggest that we use something more specific than post inference processing, e.g., @evaluating_logprob, to determine whether the current evaluation is for computing log prob. This @evaluating_logprob approach has the advantage of not being closely coupled to a specific inference algorithm or post-inference analysis.

Suggestions on a better design are welcome!

[yebai]

I think we could use something like Turing.@evaluation_mode() to determine the current evaluation mode, e.g., log density, prediction, etc.

See, also, TuringLang/Turing.jl#2239

Cc @devmotion @sunxd3 @mhauru for more comments.

[yebai]

Cc @seabbs