Remote function cannot recursively call itself.

[robertnishihara]

Consider the following script.

import ray

ray.init()

@ray.remote
def f():
  return ray.get(f.remote())

f.remote()

When f.remote() is called, the following error is printed.

Traceback (most recent call last):
  File "<ipython-input-3-6f5c275d0ce3>", line 3, in f
NameError: name 'f' is not defined

This worked at some point but seems to have gotten broken. We weren't testing recursively defining remote functions because defining remote functions in this manner inside of the Python unit testing framework didn't work. However, we should add tests for this behavior and just not use the unit testing framework for that part.

[robertnishihara]

This could be partially addressed by #453. However, this doesn't fully support recursive remote functions. For example, the PR does not fix the following example.

def create_and_call_f():
  @ray.remote
  def f():
    return [f.remote()]

  f.remote()  

create_and_call_f()

Basically, you can't define a recursive remote function inside of another function or class. We'll have to look into how to do this properly.

[mehrdadn]

@robertnishihara

FYI, it turns out a lot of this work is something Numba has wrestled with.

The reason is that they are solving a very similar problem: instead of pickling a function, they compile it to LLVM IR, so they have to statically capture the closure of the function too, do full-on type-inference/type-checking, and and save the results in a binary format, etc. The main conceptual difference is their process is practically irreversible (you can't decompile LLVM IR) but aside from that, very much of what they do is surprisingly similar.

I don't know if it's possible to outright use Numba's code, but I would at least follow their footsteps as much as possible, whether terms of what to support vs. not support, or in terms of what the API should look like, or in terms of what solutions to apply to problems that come up with respect to pickling, or anything else. It'll be more idiomatic and lower the learning curve for people learning to use Ray, and Numba is already far more restrictive in terms of what code is allowed, Ray wouldn't be doing too badly haha.

[robertnishihara]

@mehrdadn that's a really great idea! Numba may be too strict in terms of what they allow, but they've probably solved a bunch of these problems!

[robertnishihara]

This issue is partially addressed by #453.