Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions src/utils.jl
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,67 @@ const NO_DEFAULT = NoDefault()
@addlogprob!(ex)

Add the result of the evaluation of `ex` to the joint log probability.

# Examples

This macro allows you to [include arbitrary terms in the likelihood](https://github.com/TuringLang/Turing.jl/issues/1332)

```jldoctest; setup = :(using Distributions)
julia> myloglikelihood(x, μ) = loglikelihood(Normal(μ, 1), x);

julia> @model function demo(x)
μ ~ Normal()
@addlogprob! myloglikelihood(x, μ)
end;

julia> x = [1.3, -2.1];

julia> loglikelihood(demo(x), (μ=0.2,)) ≈ myloglikelihood(x, 0.2)
true
```

and to [reject samples](https://github.com/TuringLang/Turing.jl/issues/1328):

```jldoctest; setup = :(using Distributions, LinearAlgebra)
julia> @model function demo(x)
m ~ MvNormal(zero(x), I)
if dot(m, x) < 0
@addlogprob! -Inf
# Exit the model evaluation early
return
end
x ~ MvNormal(m, I)
return
end;

julia> logjoint(demo([-2.1]), (m=[0.2],)) == -Inf
true
```

!!! note
The `@addlogprob!` macro increases the accumulated log probability regardless of the evaluation context,
i.e., regardless of whether you evaluate the log prior, the log likelihood or the log joint density.
If you would like to avoid this behaviour you should check the evaluation context.
It can be accessed with the internal variable `__context__`.
For instance, in the following example the log density is not accumulated when only the log prior is computed:
```jldoctest; setup = :(using Distributions)
julia> myloglikelihood(x, μ) = loglikelihood(Normal(μ, 1), x);

julia> @model function demo(x)
μ ~ Normal()
if DynamicPPL.leafcontext(__context__) !== PriorContext()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The leafcontext is not exported and may change. So this example might be fragile. In the future, wt would be better to avoid calling these low-level internal APIs to just get the current (primitive) context. But this should be addressed in separate PRs.

cc @rikhuijzer you might want to take a look at how context works in DynamicPPL.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was not super happy about bringing up leafcontext here - the purpose of the macro is to ensure that users don't have to use internal variable names and functions, so this is really not in line with our goals. Maybe we should just add a warning that it's always accumulated and one has to work with internal unstable fumctionality if that's not OK?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is okay to keep the example - it is useful for the users, but add a (big) warning that this interface is internal, and may change in the future.

A side note: we would like to improve the context design, together with the model macro implementation, in the longer term. Hopefully, these improvements will remove the requirement of using leafcontext.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another alternative (or addition) would be separate macro (or a switch?) if the manual log probability corresponds to an observe statement. (One could also add something for assume statements explicitly, but I guess it is less common since then one also should add a sample to the varinfo - so maybe the observe behaviour should be the default?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not feel critical to have multiple versions of @addlogprob! for now. Most people only use it inside MH or HMC samplers. But there is a need, we can add support for that later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm mainly concerned about the, in most cases, incorrect results of logprior currently - which is the main reason for this annoying leafcontext approach. I think this could be fixed by identifying it with a custom observe statement instead of an arbitrary modification of the log probability. One could eg. change the macro syntax to

@addlogprob! val [assume=false] [observe=true]

Then it would still be possible to unconditionally modify the log probability but one could also limit it easily to evaluations that include the likelihood or the prior.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in any case this is out of scope for this PR, I think.

@addlogprob! myloglikelihood(x, μ)
end
end;

julia> x = [1.3, -2.1];

julia> logprior(demo(x), (μ=0.2,)) ≈ logpdf(Normal(), 0.2)
true

julia> loglikelihood(demo(x), (μ=0.2,)) ≈ myloglikelihood(x, 0.2)
true
```
"""
macro addlogprob!(ex)
return quote
Expand Down