I'm getting TypeError("'<' not supported between instances of 'PyCall.jlwrap' and 'PyCall.jlwrap'")

[Lissanro]

I'm getting TypeError("'<' not supported between instances of 'PyCall.jlwrap' and 'PyCall.jlwrap'"). Full error output: https://pastebin.com/nD3LnkvM. I'm new to PyCall so I'm not sure if this is a bug or I'm doing something wrong.

I'm trying to use fastai2 Python library like this:

using PyCall
fai = pyimport("fastai2.vision.all")
path = fai.untar_data(fai.URLs.PETS)
files = fai.get_image_files(path/"images")

function is_cat(file)
  return uppercase(file[1])
endપ

dls = fai.ImageDataLoaders.from_name_func(path, files, is_cat, item_tfms=fai.Resize(224), bs=128)

Last line results in error above.

Files with cat images start with uppercase letter in URLs.PETS (for example: "Siamese_50.jpg"). ImageDataLoaders.from_name_func() is supposed to call helper function is_cat() to label each image file as either "true" or "false".

[stevengj]

is_cat gets translated into a callable Python object (of type PyCall.jlwrap), but possibly fai doesn't allow callable objects and only allows "real" Python functions. Try passing

PyCall.jlfun2pyfun(is_cat)

instead of is_cat in your argument list.

See also #268.

[Lissanro]

I have tried your suggestion and changed last line like this:

dls = fai.ImageDataLoaders.from_name_func(path, files, PyCall.jlfun2pyfun(is_cat), item_tfms=fai.Resize(224), bs=128)

But I'm still getting the same error (there are some small differences in Stacktrace in full output https://pastebin.com/F9z5h1wN):
TypeError("'<' not supported between instances of 'PyCall.jlwrap' and 'PyCall.jlwrap'")

However, if I do not use Julia function but Python function instead then everything works as expected. The following code runs successfully:

using PyCall
fai = pyimport("fastai2.vision.all")
path = fai.untar_data(fai.URLs.PETS)
files = fai.get_image_files(path/"images")

py"
def is_cat(file):
  return file[0].isupper()"

dls = fai.ImageDataLoaders.from_name_func(path, files, py"is_cat", item_tfms=fai.Resize(224), bs=128)
learn = fai.cnn_learner(dls, fai.resnet34, metrics=fai.error_rate)
learn.fine_tune(1)

But what if instead of uppercase() there was some very complicated preprocessing function and I really needed to use Julia to efficiently implement it? Let's pretend like I need to call Julia uppercase() here. I tried the following:

py"""
import julia
jl = julia.Julia()
def is_cat(file):
  return jl.eval("uppercase('" + file[0] + "')")"""

Wrapping Julia function call in Python function like that did not work. The result is the same TypeError as before (full output https://pastebin.com/V63NCYQ5):
TypeError("'<' not supported between instances of 'PyCall.jlwrap' and 'PyCall.jlwrap'")

But even if is_cat() is simple function like in this case, I would still prefer to define it in Julia language if possible. Is there something else I could try or defining pure Python function in cases like this is the only way?

[stevengj]

The return value of PyCall.jlfun2pyfun(is_cat) is a pure-Python function (a lambda), so I'm not sure where your jlwrap is even coming from.

Can you put a @show file as the first line of is_cat to see what file argument is being passed?

[Lissanro]

I was away for few days from my PC so couldn't test sooner... I have changed is_cat() like you suggested:

function is_cat(file)
  @show file
  return uppercase(file[1])
end

I have attached full terminal output: log.txt. Somewhere in the middle is the TypeError:

...
file = "wheaten_terrier_43.jpg"
file = "japanese_chin_69.jpg"
file = ERROR: LoadError: PyError ($(Expr(:escape, :(ccall(#= /home/lissanro/.julia/packages/PyCall/zqDXB/src/pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))) <class 'TypeError'>
TypeError("'<' not supported between instances of 'PyCall.jlwrap' and 'PyCall.jlwrap'")
...
in expression starting at /tmp/fastai.jl:23
"great_pyrenees_138.jpg"
file = "english_setter_46.jpg"
...

For some reason TypeError and Stacktrace happen between "fle = " and "great_pyrenees_138.jpg". I guess output of "@show file" and TypeError with Stacktrace come from different threads.

Why what supposed to be pure python function ends up as PyCall.jlwrap I do not know, but this is somehow related to calling uppercase(). As I have shown above, if I define py"is_cat()" without calling Julia functions, everything works as expected. But as soon as I add call to Julia's uppercase(), TypeError occurs because function becomes PyCall.jlwrap.

By the way, I noticed that in this case PyCall.jlfun2pyfun() have no effect at all. It is if I call Julia's function or not determines if I end up with PyCall.jlwrap or not. For example, the following works successfully without PyCall.jlfun2pyfun() because I commented out call to uppercase():

function is_cat(file)
  return true
#  return uppercase(file[1])
end

dls = fai.ImageDataLoaders.from_name_func(path, files, is_cat, item_tfms=fai.Resize(224), bs=128)
# dls = fai.ImageDataLoaders.from_name_func(path, files, PyCall.jlfun2pyfun(is_cat), item_tfms=fai.Resize(224), bs=128)

But calling uppercase() forces function to be PyCall.jlwrap no matter what, even if I use PyCall.jlfun2pyfun().

[stevengj]

What Is /tmp/fastai.jl:23?

[stevengj]

Maybe @show typeof(file)

[stevengj]

Oh, never mind, I see your error. It has nothing to do with function types. You are returning a Char from is_cat, but Python has no character type (only strings), so the Julia Char is getting wrapped.

You want to return a Boolean, not a char I think? I think you want isuppercase, not uppercase.

[Lissanro]

Thank you for explaining. /tmp/fastai.jl content is copy&pasted in first message but without comments so it has less lines. Ironically before opening this issue I tried isupper() but in current Julia there is no such function (it existed in the past and it is still mentioned in some places, but it was deprecated at first and then removed completely).

Instead of uppercase() I should have been calling isuppercase(). I'm not sure how I ended up using uppercase() and expecting that it returns a boolean. I guess I was paying too much attention trying to understand strange TypeError and I'm new to Julia. But thanks to your explanation I understand what the error means.

After I replaced uppercase() with isuppercase() everything works as expected. Thank you very much for your help.