proposal: Go 2: make short variable declaration re-declare already declared variables

[paskozdilar]

Author background

• Would you consider yourself a novice, intermediate, or experienced Go programmer?
  Intermediate.

• What other languages do you have experience with?
  C, C++, Python, Java, Scheme.

Related proposals

• Has this idea, or one like it, been proposed before?
  I was unable to find any similar proposal in the GitHub issue tracker.

• Does this affect error handling?
  Yes, it makes error handling more robust by removing a common loophole where you can ignore errors.

• Is this about generics?
  No.

Proposal

• What is the proposed change?
  Make the compiler treat each variable declared with the := operator as a new variable (i.e. force the user to use the re-declared variable) but re-use the variable in case it's been declared before.

• Who does this proposal help, and why?
  This proposal will help everyone because it will make error handling more robust.

• Please describe as precisely as possible the change to the language.
  When two or more variables are declared with the short declaration operator (:=), and one or more (but not all) of them have already been declared before, the current behavior is to treat the already-declared variable short declaration as assignment. This behavior allows the user to use the same variable in new contexts (e.g. returning an error from a new function) without having to use it (e.g. handle the error).

• What would change in the language spec?
  Unfortunately, I do not have enough expertise to define strict changes in the language spec.

• Please also describe the change informally, as in a class teaching Go.
  When you use a short variable declaration operator (:=) with multiple variables, you have to use all the variables, even if they were declared before.

• Is this change backward compatible?
  It doesn't seem so.

This code (courtesy of @DeedleFake) is valid in current Go:

func Example() (err error) {
  ok := true
  defer func() {
    if !ok {
      err = errors.New("failed")
    }
  }()

  v, ok := os.LookupEnv("SOMETHING")
  fmt.Println(v)
  return nil // With the proposed change, ok isn't treated as used. This is valid, if weird code, but it would suddenly break.
}

While it would be invalid with the proposed change.

• Orthogonality: how does this change interact or overlap with existing features?
  Since this is a minor change in the semantics of an operator, the interaction/overlap is identical to the current behavior.

• Is the goal of this change a performance improvement?
  No.

Costs

• Would this change make Go easier or harder to learn, and why?
  Not sure. It would create a small pothole in the semantics - all the variables declared by the := operator would be treated as new variables, yet already-declared variables would still take the same place in memory, as before the change.

• What is the cost of this proposal? (Every language change has a cost).
  Some (perhaps weird) existing code would break. In terms of language implementation, I can't see any obvious cost.

• How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?
  All tools that deal with language semantics would be affected - gopls and gofmt are the ones I can think of.

• What is the compile time cost?
  Most likely the same as current.

• What is the run time cost?
  Most likely the same as current.

• Can you describe a possible implementation?
  Unfortunately, I don't have the expertise to describe an implementation.

---

POSTED CHRONOLOGICALLY BEFORE THE ABOVE:

The current behavior of short variable declaration (:=) when one of the declared variables already exists is assignment. One of the consequences of this design is that errors can often be ignored by mistake when reusing the canonical err variable:

func() {
    x, err := FunctionThatMayFail()
    _ = x
    // Compiler error - err not used
}

func() int {
    x, err := FuncThatMayFail()
    if err != nil { return; }
    y, err := AnotherFuncThatMayFail()
    return y
    // No compiler error - invalid value of y when err != nil
}

My proposal is to strengthen the rules of short variable declaration operator (:=) so it (semantically) re-declares all variables, no matter if they're used or not. This will prevent the silent failure demonstrated in the above example. Compiler could detect when variable can be re-used.

This proposal would also make this valid Go, as it would be needed for complete robust error handling:

func() {
    x, err := FuncThatMayFail()
    // ...
    x, err := AnotherFuncThatMayFail()
    // ...
}

The consequences of this change seem small - when using the := operator in a scope deeper than the original variable's scope, Go re-declares the variable anyway:

func() {
    x := 0
    for i := 0; i < 10; i++ {
        x, y := i, i+1
        _, _ = x, y
    }
    fmt.Println(x) // prints zero
}

One of the edge cases I can think of is return values - in which case it would make sense to create a "special case" and return the last declaration of the re-declared variable:

func() (x int) {
    x, err := FuncThatMayFail()
    // return value x is updated
    x, err := AnotherFuncThatMayFail()
    // return value x is updated again
}

Another would be pointers:

func() {
    x, err := FuncThatMayFail()
    // ...
    xp1 := &x
    x, err := AnotherFuncThatMayFail()
    // ...
    xp2 := &x
    // do xp1 and xp2 point to the same variable?
}

In this case, it would make sense to make xp1 and xp2 equal, for the sake of backwards compatibility - as making them different could break code that relied on := doing assignment on same-named variables. So the "re-declaration" would be strictly a semantic property - a signal to the compiler that the value needs to be re-used after the re-declaration.

---

This change is (seems?) completely backwards-compatible - i.e. all valid Go programs would still be valid and have identical behavior. Some currently invalid Go programs would become valid after this change.

I have a feeling that I've missed quite a few details, since this proposal seems to fit the language well and has good consequences (robust error handling - lack of which is a common criticism of the language), but I can't think of any other edge cases right now.

I'm eagerly awaiting feedback on this. Cheers!

[seankhliao]

Please fill out https://github.com/golang/proposal/blob/master/go2-language-changes.md when proposing language changes

[DeedleFake]

What about closures, especially ones that are handed to defer or go? Unless I'm misunderstanding something, the following (contrived) example's behavior would change:

func Example() (err error) {
  ok := true
  defer func() {
    if !ok { // With the proposed change, this variable isn't updated by the := assignment below.
      err = errors.New("failed")
    }
  }()

  v, ok := os.LookupEnv("SOMETHING")
  if ok {
    fmt.Println(v)
  }
  return nil
}

[bheadmaster]

@seankhliao
Please fill out https://github.com/golang/proposal/blob/master/go2-language-changes.md when proposing language changes

Since this is a backwards compatible change, does it still qualify for Go2 language change? I was following the New external API or other notable changes format, which does not required all the Go2 language change paperwork.

This is an honest question, if I'm wrong with this line of thinking, then I'll fill it out.

@DeedleFake
What about closures, especially ones that are handed to defer or go?

Assuming the "re-declaration" becomes simply a matter of sematics (the underlying variable, i.e. memory address, stays the same - it's just that the compiler signals to the user that the variable needs to be re-used after the "re-declaration"), then the behavior in the closures would stay the same.

[seankhliao]

The "edge cases" are backwards incompatible, and this is a change to the language spec.

[DeedleFake]

Assuming the "re-declaration" becomes simply a matter of sematics (the underlying variable, i.e. memory address, stays the same - it's just that the compiler signals to the user that the variable needs to be re-used after the "re-declaration"), then the behavior in the closures would stay the same.

O.K., but just a slight change breaks that, then:

func Example() (err error) {
  ok := true
  defer func() {
    if !ok {
      err = errors.New("failed")
    }
  }()

  v, ok := os.LookupEnv("SOMETHING")
  fmt.Println(v)
  return nil // With the proposed change, ok isn't treated as used. This is valid, if weird code, but it would suddenly break.
}

[paskozdilar]

I see. Thank you both for the quick responses. I have updated the original post with the Go2 language proposal paperwork.

Sorry for posting with different accounts, I seem to have opened the wrong Firefox container.

[ianlancetaylor]

Perhaps we could have done this initially, but making this change today would not be backward compatible. It would cause code that currently works, and is correct, to stop compiling. I don't think we can make this change to the language.

It might be feasible to check for this case in vet or some other static analyzer. That would be worth exploring.

But as a language change this is a likely decline. Leaving open for four weeks for final comments.

[ianlancetaylor]

No further comments.