Tracker 0.2.9 does not work

[devmotion]

Tests of DistributionsAD (master branch) fail with Tracker 0.2.9 but pass with Tracker 0.2.8 (just tested locally).

[devmotion]

@willtebbutt @DhairyaLGandhi Any idea what might have caused this breakage? Tracker-related code (a lot of type piracy 😢) is contained in https://github.com/TuringLang/DistributionsAD.jl/blob/master/src/tracker.jl

[ChrisRackauckas]

DiffEqSensitivity doesn't do type-piracy and that one fails, so that's a sign.

[willtebbutt]

@devmotion this is unfortunate. Could you point us towards the tests in question?

[devmotion]

You can, e.g., compare https://github.com/TuringLang/DistributionsAD.jl/pull/106/checks?check_run_id=971611652 (Tracker 0.2.9) with https://github.com/TuringLang/DistributionsAD.jl/runs/929313813 (Tracker 0.2.8). The test summary of the new version is

Test Summary:                 | Pass  Fail  Error  Broken  Total
distributions                 | 1824  1798     78     310   4010
  Univariate distributions    |  207   257             44    508
  Multivariate distributions  |  643   437     30      89   1199
  Matrixvariate distributions |  974  1104     48     177   2303

This illustrates that it affects a large range of univariate, multivariate, and matrixvariate distributions.

The test file is https://github.com/TuringLang/DistributionsAD.jl/blob/master/test/ad/distributions.jl (it is used for testing the other AD backends such as Zygote, ForwardDiff, and ReverseDiff), and the first failing distributions seem to be https://github.com/TuringLang/DistributionsAD.jl/blob/master/test/ad/distributions.jl#L44-L45.

[DhairyaLGandhi]

I've opened FluxML/Tracker.jl#77 to debug if it's a change in behaviour we need to worry about. Does DistributionsAD make use of internal functions from Tracker? There was a change in the fields for TrackedArray but hopefully that's not what would cause this.

[ChrisRackauckas]

The DiffEqSensitivity test doesn't do any type piracy nor does it depend on DistributionsAD and it also fails, so that points to something internal to Tracker.

[devmotion]

There was a change in the fields for TrackedArray but hopefully that's not what would cause this.

That caused some errors in Bijectors, but I've already fixed them: TuringLang/Bijectors.jl@33048d5

[devmotion]

Does DistributionsAD make use of internal functions from Tracker?

A quick search didn't reveal anything yesterday.

[ChrisRackauckas]

For reference:

using DiffEqSensitivity, OrdinaryDiffEq, Tracker, ForwardDiff, FiniteDiff

function fb(du,u,p,t)
  du[1] = dx = p[1]*u[1] - p[2]*u[1]*u[2]*t
  du[2] = dy = -p[3]*u[2] + p[4]*u[1]*u[2]
end

p = [1.5,1.0,3.0,1.0]; u0 = [1.0;1.0]
prob = ODEProblem(fb,u0,(0.0,10.0),p)

function G(p)
  tmp_prob = remake(prob,u0=convert.(eltype(p),prob.u0),p=p)
  sol = solve(tmp_prob,Tsit5(),abstol=1e-14,reltol=1e-14,sensealg=DiffEqBase.SensitivityADPassThrough(),saveat=t)
  A = Array(sol)
  sum(((2 .- A).^2)./2)
end
G([1.5,1.0,3.0,1.0])
res2 = ForwardDiff.gradient(G,[1.5,1.0,3.0,1.0])
res3 = FiniteDiff.finite_difference_gradient(G,[1.5,1.0,3.0,1.0])
res4 = Tracker.gradient(G,[1.5,1.0,3.0,1.0])[1]

Gives

4-element Array{Float64,1}:
   228.68445307683288
  -237.74392662478962
   394.6268717258131
 -1223.4191564606485

4-element Array{Float64,1}:
   228.68445287934475
  -237.74392679565383
   394.6268717123165
 -1223.4191569287455

Tracked 4-element Array{Float64,1}:
  46.827325320171816
 -67.37120714519541
   4.689749988111205
 -12.586582069291765

One is not like the others.

[devmotion]

I just tried FluxML/Tracker.jl#77 and it seems to fix the issues (FluxML/Tracker.jl#76 (comment)). Have you tried the MWE with the PR as well?

[DhairyaLGandhi]

Right, so FluxML/Tracker.jl#72 would need some work, we can revert it, fix downstream packages and open up a new PR for iterative backprop.

[willtebbutt]

Oh I see -- we had multiple PRs merged in the same patch release. I had assumed that it was something to do with me merging FluxML/Tracker.jl#1, but I guess not.