Skip to content

Conversation

@phipsgabler
Copy link
Member

Implements the changes discussed in #49: make VarName use a tuple of tuples instead of a string for the indexing value. For that, I mostly reused the existing implementations of @vinds and @vsym.

While at that, I also refactored the methods realing to VarName a bit and moved some leftovers to their right place in varname.jl.

Updating all the rest of the code base and the tests is still to be done.

@codecov

This comment has been minimized.

@phipsgabler
Copy link
Member Author

I'm now getting a couple of equal errors from ForwardDiff. Any ideas?

sample: Error During Test at /home/philipp/.julia/dev/DynamicPPL/test/compiler.jl:300
  Got exception outside of a @test
  BoundsError: attempt to access 12-element Array{ForwardDiff.Chunk,1} at index [0]
  Stacktrace:
   [1] getindex at ./array.jl:744 [inlined]
   [2] ForwardDiff.Chunk(::Int64, ::Int64) at /home/philipp/.julia/packages/ForwardDiff/vt5F1/src/prelude.jl:17
   [3] Chunk at /home/philipp/.julia/packages/ForwardDiff/vt5F1/src/prelude.jl:16 [inlined]
   [4] gradient_logp(::ForwardDiffAD{40}, ::Array{Float64,1}, ::VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64}, ::Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}, ::Sampler{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},Main.Turing.Inference.SamplerState{VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64}}}) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/core/ad.jl:102
   [5] gradient_logp at /home/philipp/.julia/dev/DynamicPPL/test/Turing/core/ad.jl:69 [inlined]
   [6] ∂logπ∂θ at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/hmc.jl:401 [inlined]
   [7] ∂H∂θ at /home/philipp/.julia/packages/AdvancedHMC/haUrH/src/hamiltonian.jl:28 [inlined]
   [8] phasepoint(::AdvancedHMC.Hamiltonian{AdvancedHMC.Adaptation.UnitEuclideanMetric{Float64,Tuple{Int64}},Main.Turing.Inference.var"#logπ#49"{VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64},Sampler{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},Main.Turing.Inference.SamplerState{VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64}}},Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}},Main.Turing.Inference.var"#∂logπ∂θ#48"{VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64},Sampler{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},Main.Turing.Inference.SamplerState{VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64}}},Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}}}, ::Array{Float64,1}, ::Array{Float64,1}) at /home/philipp/.julia/packages/AdvancedHMC/haUrH/src/hamiltonian.jl:59
   [9] phasepoint at /home/philipp/.julia/packages/AdvancedHMC/haUrH/src/hamiltonian.jl:129 [inlined]
   [10] sample_init at /home/philipp/.julia/packages/AdvancedHMC/haUrH/src/sampler.jl:13 [inlined]
   [11] #HMCState#52(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Type{Main.Turing.Inference.HMCState}, ::Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}, ::Sampler{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},Main.Turing.Inference.SamplerState{VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64}}}, ::Random._GLOBAL_RNG) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/hmc.jl:562
   [12] Main.Turing.Inference.HMCState(::Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}, ::Sampler{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},Main.Turing.Inference.SamplerState{VarInfo{NamedTuple{(:s, :m),Tuple{DynamicPPL.Metadata{Dict{VarName{:s,Tuple{}},Int64},Array{InverseGamma{Float64},1},Array{VarName{:s,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{VarName{:m,Tuple{}},Int64},Array{Normal{Float64},1},Array{VarName{:m,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64}}}, ::Random._GLOBAL_RNG) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/hmc.jl:533
   [13] Sampler(::HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric}, ::Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}, ::DynamicPPL.Selector) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/hmc.jl:310
   [14] (::Main.Turing.Inference.var"#79#80"{Gibbs{(:m, :s),Tuple{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},PG{(:s,)}}},Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}})(::HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric}) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/gibbs.jl:73
   [15] map at ./tuple.jl:140 [inlined]
   [16] Sampler(::Gibbs{(:m, :s),Tuple{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},PG{(:s,)}}}, ::Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}, ::DynamicPPL.Selector) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/gibbs.jl:64
   [17] Sampler at /home/philipp/.julia/dev/DynamicPPL/src/sampler.jl:44 [inlined]
   [18] #sample#2(::Type, ::Nothing, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(sample), ::Random._GLOBAL_RNG, ::Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}, ::Gibbs{(:m, :s),Tuple{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},PG{(:s,)}}}, ::Int64) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/Inference.jl:155
   [19] sample at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/Inference.jl:154 [inlined]
   [20] #sample#1 at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/Inference.jl:141 [inlined]
   [21] sample(::Model{var"##inner_function#543#14",NamedTuple{(),Tuple{}},ModelGen{(),var"###gdemo_d#569",NamedTuple{(),Tuple{}}},Val{()}}, ::Gibbs{(:m, :s),Tuple{HMC{ForwardDiffAD{40},(:m,),AdvancedHMC.Adaptation.UnitEuclideanMetric},PG{(:s,)}}}, ::Int64) at /home/philipp/.julia/dev/DynamicPPL/test/Turing/inference/Inference.jl:141
   [22] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/compiler.jl:302
   [23] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Test/src/Test.jl:1107
   [24] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/compiler.jl:301
   [25] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Test/src/Test.jl:1107
   [26] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/compiler.jl:12
   [27] include at ./boot.jl:328 [inlined]
   [28] include_relative(::Module, ::String) at ./loading.jl:1105
   [29] include(::Module, ::String) at ./Base.jl:31
   [30] include(::String) at ./client.jl:424
   [31] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/runtests.jl:9
   [32] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Test/src/Test.jl:1107
   [33] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/runtests.jl:9
   [34] include at ./boot.jl:328 [inlined]
   [35] include_relative(::Module, ::String) at ./loading.jl:1105
   [36] include(::Module, ::String) at ./Base.jl:31
   [37] include(::String) at ./client.jl:424
   [38] top-level scope at none:6
   [39] eval(::Module, ::Any) at ./boot.jl:330
   [40] exec_options(::Base.JLOptions) at ./client.jl:263
   [41] _start() at ./client.jl:460

@devmotion
Copy link
Member

devmotion commented Mar 23, 2020

The logs indicate that the error is caused by N = 0 in
https://github.com/JuliaDiff/ForwardDiff.jl/blob/3314011068b2b0d7bd3039c71eb75b6ca76bff96/src/prelude.jl#L17. This arises from

chunk = ForwardDiff.Chunk(min(length(θ), chunk_size))
, I assume since length(θ) = 0. So there's an empty array passed to gradient_logp?

@devmotion
Copy link
Member

If that is actually the reason for the errors, then it might be caused by
the statement spl.state.vi[spl] in https://github.com/phipsgabler/DynamicPPL.jl/blob/c614cef5a3128d9f883cd9fba795d780c03fe4ad/test/Turing/inference/hmc.jl#L546.

@phipsgabler phipsgabler changed the title [WIP] Begin refactoring of varnames to use tuples [WIP] Refactor VarName Mar 25, 2020
@phipsgabler
Copy link
Member Author

I just missed updating of the in type piracy. It works now, in principle, but I'll update the inspace function once more this evening and then the PR should be finished.

@phipsgabler
Copy link
Member Author

I updated some more tests, and am now getting the following error from this line:

  ArgumentError: index ["p[1,1]"] not found
  Stacktrace:
   [1] axisindexes(::Type{AxisArrays.Categorical}, ::Array{String,1}, ::Array{String,1}) at /home/philipp/.julia/packages/AxisArrays/FPe0d/src/indexing.jl:329
   [2] axisindexes at /home/philipp/.julia/packages/AxisArrays/FPe0d/src/indexing.jl:199 [inlined]
   [3] macro expansion at /home/philipp/.julia/packages/AxisArrays/FPe0d/src/indexing.jl:394 [inlined]
   [4] _to_index(::AxisArrays.AxisArray{Real,3,Array{Real,3},Tuple{AxisArrays.Axis{:iter,StepRange{Int64,Int64}},AxisArrays.Axis{:var,Array{String,1}},AxisArrays.Axis{:chain,UnitRange{Int64}}}}, ::Tuple{AxisArrays.Unsupported,AxisArrays.Categorical,AxisArrays.Unsupported}, ::Colon, ::Array{String,1}, ::Colon) at /home/philipp/.julia/packages/AxisArrays/FPe0d/src/indexing.jl:350
   [5] to_index at /home/philipp/.julia/packages/AxisArrays/FPe0d/src/indexing.jl:347 [inlined]
   [6] getindex(::AxisArrays.AxisArray{Real,3,Array{Real,3},Tuple{AxisArrays.Axis{:iter,StepRange{Int64,Int64}},AxisArrays.Axis{:var,Array{String,1}},AxisArrays.Axis{:chain,UnitRange{Int64}}}}, ::Function, ::Array{String,1}, ::Function) at /home/philipp/.julia/packages/AxisArrays/FPe0d/src/indexing.jl:123
   [7] getindex(::Chains{Real,Missing,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(),Tuple{}}}, ::Function, ::Array{String,1}, ::Function) at /home/philipp/.julia/packages/MCMCChains/eOnm5/src/chains.jl:167
   [8] getindex at /home/philipp/.julia/packages/MCMCChains/eOnm5/src/chains.jl:147 [inlined]
   [9] #check_numerical#4(::Float64, ::Float64, ::typeof(check_numerical), ::Chains{Real,Missing,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(),Tuple{}}}, ::Array{String,1}, ::Array{Int64,1}) at /home/philipp/.julia/dev/DynamicPPL/test/test_utils/numerical_tests.jl:47
   [10] (::var"#kw##check_numerical")(::NamedTuple{(:atol,),Tuple{Float64}}, ::typeof(check_numerical), ::Chains{Real,Missing,NamedTuple{(:internals, :parameters),Tuple{Array{String,1},Array{String,1}}},NamedTuple{(),Tuple{}}}, ::Array{String,1}, ::Array{Int64,1}) at ./none:0
   [11] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/varinfo.jl:374
   [12] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Test/src/Test.jl:1107
   [13] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/varinfo.jl:345
   [14] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Test/src/Test.jl:1107
   [15] top-level scope at /home/philipp/.julia/dev/DynamicPPL/test/varinfo.jl:20

@devmotion
Copy link
Member

I'd suggest printing the names of the chain that was constructed in the step before by running names(chain).


"""
@varname(var)
subsumes(u::VarName, v::VarName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this function used/needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It replaces the former string matching in in(::VarName, ::Tuple), and is used in inspace. Like, checking that x[1][2] is in a space containing x[:] or something. If you have a better name than subsumes, I'd be happy to rename it!

Copy link
Contributor

@mohamed82008 mohamed82008 Apr 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, but I don't see this function used very much here. So I was wondering if you have another use case for it. Most of the time we only need to check if a VarName's symbol is in a tuple of symbols.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If spaces were allowed to contain just symbols, inspace would be trivial indeed. But since it also allowes indexed VarNames, there must be a way to determine compatibility. Like what the string matching in the previous _in function did.

Check whether `vn`'s variable symbol is in `space`.
"""
inspace(vn, space::Tuple{}) = true # empty space is treated as universal set
inspace(vn, space::Tuple) = vn in space
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this use the symbol of vn?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those two are the cases when vn is not a VarName. I left those because I think some parts are still not consistently using VarName. It is consistent with the old usage of in, which only added the special check for VarName.

@devmotion
Copy link
Member

devmotion commented Apr 8, 2020

@phipsgabler Can you add the fix in #61 (comment) (i.e., basically add Base.replace_ref_end!) and a test for it to this PR? Otherwise I'll open another PR after this PR has been merged (I don't want to mess with VarName right now on master).

@phipsgabler
Copy link
Member Author

phipsgabler commented Apr 8, 2020

@devmotion Sure. What would be a good test? I somehow don't get the following to work:

@model testmodel(z) = begin
    m ~ Normal()
    z[1:end] ~ MvNormal(fill(m, length(z)), 1.0)
end
model = testmodel()
@test all(z -> isapprox(z, 0; atol = 0.1), mean(model() for _ in 1:1000))

(Kind of embarrassing that I still don't really know how to actually work with Turing...)

@devmotion
Copy link
Member

devmotion commented Apr 8, 2020

Oh you have to provide z (or otherwise initialize it in the model). The test doesn't have to be elaborate, just make sure it runs reasonably.

@phipsgabler
Copy link
Member Author

OK, I think I'm finished here.

@yebai yebai merged commit 1c071df into TuringLang:master Apr 10, 2020
@yebai
Copy link
Member

yebai commented Apr 10, 2020

Good work @phipsgabler!

@phipsgabler phipsgabler deleted the phg/varinfo-indices branch April 10, 2020 08:19
@devmotion
Copy link
Member

Really good, it's great we do not work with strings anymore internally 👍

@phipsgabler
Copy link
Member Author

MCMCChains still does, somehow, I think...

@devmotion
Copy link
Member

It needs TuringLang/MCMCChains.jl#197, that should solve most String/Symbol issues IIRC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants