callback functions see "old world" in 0.6

[stevengj]

It seems much harder to pass a proper closure as a callback function in Julia 0.6 than in previous versions due to #17057. In particular, the method described in the manual and in this blog post no longer works: passing the Function via a "thunk" void* that is called from a cfunction fails because the Function sees an "old world age".

See JuliaPy/PyCall.jl#343 and JuliaMath/Cubature.jl#22 and the mailing list, for example, but this will affect any package wrapping a C library that expects callback functions.

In PyCall, @yuyichao suggested using eval, but this is clunky and seems likely to impose a large performance penalty. Is there a better solution?

[yuyichao]

This describes a expected behavior so this is not a regression but simply a breaking change. The eval here shouldn't add that much overhead although a lighter weight one will be good.

[yuyichao]

The fix of this (adding doc or add a slightly cheaper way to call something in the latest world) won't be breaking.........

[stevengj]

We need documentation of this, and to update the blog post. Can we have eval(function, args...) or something?

Why does a call to a Function object need to use the caller's world age?

[yuyichao]

Inference cannot infer anything about a newer world so any change in world age has to be an optimization barrier. We also need to be consistent on what do we treat as optimization barrier and failure to do that is one part of #265. Since a function call can be optimized, it must never be an optimization barrier, i.e. the dispatch and the body of f() must see the same world as the caller.

[stevengj]

But a call to a Function object that is fetched from a pointer cannot be optimized, so it can be an optimization barrier, no?

[yuyichao]

The semantics shouldn't be inference dependent unless we have a clear rule of what's inferrable.

[stevengj]

Presumably unsafe_pointer_to_objref (or any ccall with a return type Any) should never result in an inferrable type?