DiffEqFlux could handle infinite loss better.

[freemin7]

I've been (miss)using DiffEqFlux for some project with a differential equation whose parameter space has solutions with finite escape time.

Loss is infinite in top-level scope at base/none in at base/none in #train!#12 at Flux/qXNjB/src/optimise/train.jl:69 in macro expansion at Juno/TfNYn/src/progress.jl:124 in macro expansion at Flux/qXNjB/src/optimise/train.jl:71 in gradient at Tracker/RRYy6/src/back.jl:164 in #gradient#24 at Tracker/RRYy6/src/back.jl:164 in gradient_ at Tracker/RRYy6/src/back.jl:98 in losscheck at Tracker/RRYy6/src/back.jl:154

It currently throws an error under these conditions. The issue came up in this tread of mine: https://discourse.julialang.org/t/tracking-initial-condition-to-optimize-starting-value-too-diffeqflux/26596

A solution in this case was proposed:
Since ADAM is inherently stochastic, it could have functionality to discard Infs and try a different randomized points to try and not hit the bad area. Machine learning problems generally don’t have this bifurcation behavior so it doesn’t show up as much, so I think it was just overlooked.

Could you look into it solving this in general?

[ChrisRackauckas]

@MikeInnes

[ChrisRackauckas]

Handled by using BFGS.