[Core][Streaming generator] Support num_returns.

[rkooo567]

Description

Sometimes, users want to provide all the object references ahead of time if they know num_returns.

ref1, ref2 = task.options(num_returns=2).remote()

Use case

- even though it’s for unary requests, we actually currently use a streaming call to yield exactly two messages, one ray serve system message and then the actual result of the method call (this is in order for replicas to reject requests at max capacity)
- we want to allow the user to fetch the object ref for the result, but with a streaming call the object ref for the second message won’t be ready until a result is actually yielded. This is the issue, we want to give user an object ref pointing to the second message when the task is scheduled, not when the call returns (which is not possible with streaming right now)

[rkooo567]

related: #46893

[kyle-v6x]

Any updates on this? It's tagged as enhancement but the streaming change made in 2.10.0 results in breaking the _to_object_ref() calls as per #46893. All our Ray deployments are stuck on pre-2.10 for this reason.

[lust4life]

bump, any updates?

[kyle-v6x]

Bumping again. It's assigned, but no work is being done. Should I start work on this myself? Is there any timeline? We would love to update ray past 2.10.

[jjyao]

Hi @lust4life @kyle-v6x, I'm working on it and I was interrupted by something else. I'll come back to this ASAP (next week).

[jjyao]

@rkooo567 we already have static generator if num_returns is specified, is that not enough?

[rkooo567]

But static generator is not streaming? So it depends on use cases, but imo static generator defeats the purpose of generator