Making peace with Images.jl

[timholy]

It's fairly well known that using DataArrays, Images produces a whole slew of ambiguity warnings. I'm contemplating trying to end those warnings by doing the tedious work of defining stub functions that throw errors for the ambiguous cases. For this to work well, we'll need to make it independent of the order in which Images and DataArrays are loaded, hence there may need to be some cooperation between the projects.

It's a little tricky making this work, but I may have a viable design. As a reminder, here are the constraints:

• The ambiguity-resolving methods (more specific than those defined in any of the packages) must come before the ambiguous methods. For this to work, all the relevant types (from both DataArrays and Images) have to be defined. Conversely, we don't want to add those definitions until all relevant types (i.e., both packages) are present.
• With precompilation, you can't just insert if isdefined(Main, :Images) somewhere in your code; precompilation will "lock in" the state at the time of compilation, and not be adjustable to the situation at the time of usage. All code that checks about the existence of another package must be in the __init__ function.
• __init__ runs as the final step of module definition/initialization. Consequently, if you add the ambiguity-resolving functions to Images' or DataArrays' __init__ function, it will be too late.

On the Images side, I plan to split out an ImagesCore module that contains the basic type definitions, which then gets used by Images. (It could be an internal module, but I wonder if @SimonDanisch would be happier if it were a standalone package so that packages like ImageMagick could use just the core and not bring in all the rest of the code.) I plan to place the check for DataArrays inside ImagesCore. In other words, it will look like this:

module Images
using ImagesCore
...
# code, including the stuff that causes ambiguity warnings with DataArrays

end



module ImagesCore

# define types

function __init__()
    if isdefined(Main, :DataArrays)
        eval(Expr(:import, :DataArraysImages))
    end
end

end

Since ImagesCore gets loaded at the beginning of Images, its __init__ function will execute in time to prevent the ambiguities. DataArrays could use a similar strategy, and also import DataArraysImages.

Interested?

[nalimilan]

Wasn't there a plan to print ambiguity warnings only when actually calling the offending methods? The level of cooperation you're suggesting isn't likely to be followed with each random combination of packages.

Of course, doing something waiting for a more general resolution isn't a bad idea.

[timholy]

That seems to be 0.5 material at the earliest. It depends on Jeff's new type system, and I'm not certain when that will be merged (but presumably in 0.5).

[tkelman]

Possibly dumb question: would NullableArrays change any of this ambiguity story?

[timholy]

The ambiguities arise because we have two container types (AbstractImage and DataArray) that define operations like

(+)(A::ContainerType, B::AbstractArray) = ...
(+)(A::AbstractArray, B::ContainerType) = ...

and julia gets worried that it doesn't know which to pick for operations like

(+)(A::AbstractImage, B::DataArray) = ...

So I don't think NullableArrays will change it.

[johnmyleswhite]

I like this plan a lot, but worry that it won't be worth the effort given that it's a short-term fix for what I would assume is a small demographic of users who want to work with both packages at the same time.

[timholy]

True. Now that I'm switching from Winston to Gadfly/Immerse, I'm one of them, which is why I am motivated to fix this. But it will work only if DataArrays does this; I'll be working away with using Images and no intention of using a DataArray, realize I need to plot something, say using Immerse and then presto lots of warnings.

But I fully understand the reluctance to uglify things. Maybe my better bet is just to define this internally. Honestly, the only thing I'm concerned about is folks missing important messages because people get used to ignoring a whole sea of junk messages.

[cstjean]

But I fully understand the reluctance to uglify things. Maybe my better bet is just to define this internally. Honestly, the only thing I'm concerned about is folks missing important messages because people get used to ignoring a whole sea of junk messages.

As a user who got used to ignoring the sea of junk, if you define it for yourself, please share!

[dlfivefifty]

+1

[ghost]

I can see a huge demographic of users interested in both packages, including me and so many others that have opened issues here and there about it. Julia should keep freedom in mind.

[timholy]

You have the freedom to fix this and share your fix with everyone else! A solution is sketched at the top; I haven't gotten to it yet, but that doesn't prevent anyone else from tackling it. This is the best kind of freedom of all 😄.

[ghost]

Hi @timholy, I was not complaining whatsoever, I was just not clear. I've been using the solution using package.function with no issues. I was just commenting on @johnmyleswhite's opinion about co-ocurrence of images.jl and dataframe.jl users.

[timholy]

Gotcha, thanks for clarifying.

[timholy]

Superseded by JuliaLang/julia#16125.