proposal: Go 2: Make goroutine return channel with function result (promise)

[brainpicture]

If you starting a goroutine, for example:

go myFunc()

func myFunc() string {
  time.Sleep(time.Second)
  return "hello"
}

It's pretty hard to get the result of function execution. You need to create a channel

It could be nice if go will return channel with the first argument of a function.
Example:

myFuncResult := go myFunc()
fmt.Println(<- myFuncResult) // Prints hello

func myFunc() string {
  time.Sleep(time.Second)
  return "hello"
}

If a function returns several elements, several channels could be returned, (compiler should handle the optimization and create only one real channel)
Example:

myFuncResult1, myFuncResult2 := go myFunc()
fmt.Println(<- myFuncResult1, <- myFuncResult2) // Prints hello 3

func myFunc() (string, int) {
  time.Sleep(time.Second)
  return "hello", 3
}

So this proposal is no more than syntax sugar, the result of function should be automatically sent to channel by go compiler.

[ianlancetaylor]

Looks a lot like #25821.

[dpinela]

Also #26287.

[beoran]

This proposal is simpler than the two above though. However what is unclear in this proposal is how the goroutine started would write to the channel created.

[brainpicture]

This proposal is simpler than the two above though. However what is unclear in this proposal is how the goroutine started would write to the channel created.

the idea is, that compiler will automatically write the result of a function to the channel, so the proposal is just a syntax sugar

[beoran]

Ok, but you could only write one result like that, right? How could I write many results, perhaps, live, while my goroutine is running? Or is that not needed?

[bcmills]

For a function f with signature func([…]) (R₁, […], Rₙ), I can think of at least four different “intuitive” ways to encode the result as a “future” in Go:

1. As a pair (func() (R₁, […], Rₙ), <-struct{}). The channel is closed when the result is ready. The func() blocks until ready and then returns the results, and may be called arbitrarily many times.
2. As a func(<-chan struct{}) (R₁, […], Rₙ, bool). The passed-in channel is normally obtained from a call to context.Context.Done, and may be nil. The function blocks until either the channel is closed or the result is ready, and may be called arbitrarily many times; the final bool indicates whether the result was ready.
3. As a tuple of buffered channels (<-chan R₁, […], <-chan Rₙ). Each channel receives the corresponding result exactly once.
4. As a single buffered channel <-chan R₁ (if n is 1) or <-chan struct{R1 R₁, […], Rn Rₙ} (with suitably synthesized names, or in conjunction with a proposal for tuple types).

Of those options, this proposal chooses option 3, which raises some questions (closely related to the ones in my Rethinking Concurrency GopherCon talk):

• When, if ever, would the channel(s) created in this way be closed?
• Especially for types which have nil values (such as error): if the function returns nil, is the nil-value sent on the channel explicitly?

[bcmills]

I am somewhat concerned that this approach would be too error-prone. Particularly:

• If the caller attempts to read the result twice by receiving from the channel twice, they will either block or receive a zero-value from the second value, neither of which is likely the intended behavior.
• If the caller receives from the value immediately, the call appears to be concurrent but provides no concurrency.
• Returning a channel encourages the use of channels over (say) sync.WaitGroup or errgroup, which may bias programs toward the “asynchronous” style rather than the (in my experience more reliable) “structured concurrency” style.

[tema3210]

Looks interesting.
However:
1)Implementation of this is a kind of mcsp, or even scsp queue, meaning two atomics and no locks. Meaning special case of channel.
2)As noticed above, what if one already got the value from resulting channel, but another is still trying to recive it? The channel must be closed after recieving results, who will care about it? It can be covered in some struct with matching method, but result of go command will not be channel.
3) Go has async under the hood. I don't think that we need it in code.

[brainpicture]

Implementation of this is a kind of mcsp, or even scsp queue, meaning two atomics and no locks. Meaning special case of channel.
– Exactly, mb Go needed new type of channel, there are tons of cases where channels used just to read once

As noticed above, what if one already got the value from resulting channel, but another is still trying to recive it? The channel must be closed after recieving results, who will care about it? It can be covered in some struct with matching method, but result of go command will not be channel.
– I will think about this a little bit, maybe post another proposal later.

[brainpicture]

I am somewhat concerned that this approach would be too error-prone. Particularly:

• If the caller attempts to read the result twice by receiving from the channel twice, they will either block or receive a zero-value from the second value, neither of which is likely the intended behavior.
• If the caller receives from the value immediately, the call appears to be concurrent but provides no concurrency.
• Returning a channel encourages the use of channels over (say) sync.WaitGroup or errgroup, which may bias programs toward the “asynchronous” style rather than the (in my experience more reliable) “structured concurrency” style.

1. thats good point, but same point for any other function that returns channel (widely used pattern)
2. same – you going to face this problem using channels mannualy, syntax sugar would not make it any more complicated
3. agreed, but waitgroups has its own problems, topic for a long discussion

[bcmills]

same point for any other function that returns channel (widely used pattern)

Today, that pattern is an explicit, conscious choice from among several options. By baking it into the language, this proposal would promote channel-based asynchronicity to a default, so that the choice not to use this pattern would require more thought than the choice to use it. That's what I'm worried about.

[bcmills]

Here's a fifth alternative: for a function f with signature func([…]) (R₁, […], Rₙ), the statement go f() could instead return (*R₁, […], *Rₙ, <-chan struct{}).

When the goroutine exits, the pointers would be set to its results and then the final <-chan struct{} would be closed.

That would make your example read like:

r1, r2, done := go myFunc()
[…] // Actually do something concurrently here, because why else would you want the goroutine?
<-done
fmt.Println(*r1, *r2)

func myFunc() (string, int) {
  time.Sleep(time.Second)
  return "hello", 3
}

That handily resolves the questions about when the channels are closed or whether nil values are sent, because there is only one channel and closing it is the only reasonable thing that can happen. It also resolves the problem of multiple reads, since pointers can be read arbitrarily many times.

In exchange it introduces some potential pointer/value confusion, particularly if the results are passed as type interface{} or checked against nil.

(Note that the original proposal has exactly the same problem, since <-chan T is also nillable and can also be passed as type interface{}.)

[bcmills]

Personally, I think the (func() (R₁, […], Rₙ), <-struct{}) option is the least error-prone: it allows multiple-reads, and (at least in the case of functions with multiple return-values) makes it much less likely that the caller will confuse “the future destination of the result” with “the result itself” or attempt to read the result prematurely.

[ianlancetaylor]

If we had generics, we could write a version of this operation easily enough:

func GoAndReturn(type T)(f func() T) chan T {
    c := make(chan T)
    go func() {
        c <- f()
    }()
    return c
}

Adding this to the language would make it easier to support passing arguments to the function, and returning multiple results from the function. But those seem like relatively minor enhancements to the basic idea, and of course you could write a generic version for any specific combination of number of arguments and results.

[bcmills]

@ianlancetaylor, note that that particular generic implementation has the same usability problems described in #33046 (comment). (Namely, it is too easy to accidentally read the channel too many or too few times.)

That is: implementing this in user code may be straightforward, but the specific details needed to do it robustly are apparently not obvious.

[ianlancetaylor]

@bcmills Agreed, but I don't see how this proposal make those issues any better. The point is, as far as I can see, anything we can add to the language can also be done using a generic function.

[zigo101]

This proposal should also apply to deferred function calls.

[DeedleFake]

This proposal should also apply to deferred function calls.

I'm not at all sure how that would work.

func Example() {
  v := defer func() int {
    return 3
  }()

  // What is v? The function hasn't actually been run yet.
  fmt.Println(v)

  // Now the deferred function gets run. Did v have any kind of value before this?
}

[zigo101]

func Example() {
  var v chan int
  defer func() {
    <-v // use 1
  }
  v = defer func() int {
    return 3
  }()
  
  go func() {
  	<-v // use 2
  }()
}

[ianlancetaylor]

It seems that any approach we add to the language here can be done with a generic function, if we had generics. Therefore, this is a likely decline. It can be reopened if we are unable to add generics to the language.

Leaving open for four weeks for further comments.

[tema3210]

Alternatively, we can make the go statement return function which return the value, if present, and bool indicating whether value is present. Internally it should look like value and semaphore. Noting example with defer above, v is even not future, from proposal, we have to defer go somefunc() to actually make a future, and v in this case will be not completed future, i.e chan or my proposed function which will tell one that future is not ready

[ianlancetaylor]

@tema3210 I'm not sure quite what you are suggesting, but it sounds like a different proposal.

There were no further comments on this proposal, so closing.