proposal: Go 2: Error handling that is compact and composeable

[gregwebs]

Author background

• Experience: 4 years experience writing Go code. Expert at writing if err != nil. I have forked and created Go error handling libraries.
• Other language experience: I like to say I have used most most of the common languages that are not JVM/CLR (Rust, Haskell, Python, Ruby, TypeScript, Nim, SQL, bash, terraform, Cue, etc)

Related proposals

• Has this been proposed before? Yes, the differences with existing proposal is extensively discussed in the proposal.
• Error Handling: Yes
• Generics: No

Proposal

Existing proposals to improve Go errors taught us that our solution must provide 2 things:

• the insertion of a return statement for errors
• compose error handler functions together before the error is returned

I believe existing solutions handle the first point well but have failed on the second. With a slight variation on existing error handling proposals, we can provide the second.

Additionally, adding a compiler-recognized way of returning errors creates the opportunity to add low-overhead error tracing and thus dramatically improve debugging.

Motivation: Go's biggest problem

Recently the Go Developer Survey 2022 Q2 results were released.
Now that Generics have been released, the biggest challenge for Go developers is listed as "Error handling / working with stack traces".

Exising Go code

A few years ago, Go developers described the issue with error handling and proposed a solution. As a short review, existing Go code looks like this:

f, err := os.Open(filename)
if err != nil {
        return …, err  // zero values for other results, if any
}

Existing solutions are a great starting point

New proposals allow the Go programmer to write the exact same code, but more tersely:

f := try os.Open(filename)

Where try is a new keyword or builtin that can be though of like a macro that generates the previous Go code with an early return of the error. The problem with this example is that it is trivial. Often we want to annotate the error with additional information, at least an additional string. Adding this code that modifies the error before it is returned is what I will refer to as adding an "error handler".

The original draft proposal solution used stacked error handlers, but this has difficulties around composition due to the automatic stacking and code readability since the error handler is invoked implicitly. A second proposal was put forth not long after which simply implemented try as above without any support for (stacked) error handlers. This proposal had extensive discussion that the author attempted to summarize. I believe the main problem with that second proposal is that it did not create any new affordances for error handling and instead suggested using defer blocks, which are clumsy to use with error handling.

Extending existing solutions with function-based error handling

Composing error handlers can be solved by adding a 2nd parameter to try. The second parameter is an errorhandler of type func(error) error or more precisely with generics: type ErrorHandler[E error, F error] func(E) F.

func(args...) (returnTypes..., error) {
        x1, x2, … xN = try f(), handler
        ...
}

turns into the following (in-lined) code:

func(args...) (rtype1, ... rtypeN, error) { // rtype1, ... rtypeN are the return types
        x1, … xN, errTry := f() // errTry is a new variable that is invisible or uniquely named
        if errTry != nil {
                return reflect.Zero(rtype1), ..., reflect.Zero(rtypeN), handler(errTry)
        }
        ...
}

Now we can cleanly write the following code given from the original problem statement:

func CopyFile(src, dst string) error {
        handler := func(err error) error {
                return fmt.Errorf("copy %s %s: %w", src, dst, err)
        }

        r := try os.Open(src), handler
        defer r.Close()

        w := try os.Create(dst), handler.ThenErr(func(err error) error {
                os.Remove(dst) // only if Create fails
                return fmt.Errorf("dir %s: %w", dst, err)
        })
        defer w.Close()

        try io.Copy(w, r), handler
        try w.Close(), handler
        return nil
}

ThenErr would be just a standard library function for chaining error handlers.
The new example dramatically reduces verbosity. Once the reader understands that try performs an early return of the error, it increases readability and reliability. The increased readability and reliability comes from defining the error handler code in one place to avoid de-duping it in your mind at each usage site.

The error handler parameter is optional. If no error handler is given, the error is returned unaltered, or alternative mental model is that a default error handler is used which is the identity function type ErrorId[E error] func(err E) E { return err }

Supporting quick string annotation

We can support an error handler in the form of a format string.

x = try os.Create(), "os.Create annotation %w"
// The format string is translated to:
func(err error) error { return fmt.Errorf("os.Create annotation %w", err) }

The format string would be required to have %w or %v that the error will be applied to. It would also be required to be a string literal: if string re-use is desired, an error handling function can be used.
Since this proposal just uses functions, there may be a way to try to support this use case with normal functions, something like:

func wrapErr(format string, v ...interface{}) func(error) error {
    return func(e error) error {
        v = append(v, e)
        return fmt.Errorf(format + ": %w", ...v)
    }
}

An issue with the function helper approach is the potential for extra overhead: try would need to be able to lazily invoke such an error handler generator.

Try an error directly

Sometimes the right-hand side is large. It may be preferable in some cases to assign the error value and then try it with an annotation.

x, err := valueAndError()
try err, fmt.ErrorHandler("valueAndError %w") // where ErrorHandler returns an error handler function
// or with direct support for a format string
try err, "valueAndError %w"

Similar to today, this style will require the usage of a linter to ensure that the error is checked, and it pollutes the existing scope with a variable that otherwise isn't needed.

Supporting prefix and suffix handlers

It is possible to support error handlers in the prefix position as well. Although this is sometimes convenient, a Go programmer knows that it isn't always better to have multiple ways of doing the same thing- this might be a bad idea.

x = try "valueAndError annotation %w", valueAndError()
x = try handler, valueAndError()

Conclusion

This proposal allows for:

• the insertion of a return statement for errors
• composition of error handler functions together before the error is returned

Please keep discussions on this Github issue focused on this proposal rather than hashing out alternative ideas.

Appendix: Citations

• https://go.dev/blog/survey2022-q2-results
• https://go.googlesource.com/proposal/+/master/design/go2draft-error-handling-overview.md
• https://go.googlesource.com/proposal/+/master/design/go2draft-error-handling.md
• Proposal: A built-in Go error check function, "try" #32437 - the original try error proposal
• https://github.com/pingcap/errors - a library for adding stack traces to errors
• https://github.com/pingcap/errcode - a library for error codes via interfaces

Appendix: keyword or builtin

I personally have a preference for making try a keyword, but I would be happy as well it were a builtin. The difference in new code is just a pair of parenetheses. The builtin approach does have the advantage of avoiding breaking existing code that uses try as a variable name, which does occur in the Go compiler code base. However, go fix should be able to perform automatic renames to allow for smoother upgrades.

Appendix: prior art

The err2 library implements try as a library function! Error handlers however, are created with defer: this disconnects the error handlers from their actual error. So this library requires code review to ensure that errors are actually being handled correctly or special linter rules that understand the library.

I forked err2 to make it more similar to this proposal. I am calling the result err3.

There have been proposals for dispatching on or after an error value assignment. These are quite similar to this proposal but suffer from requiring assignment and not being usable as an expression. This proposal is still open and may be the clearest one. Other closed proposals:

• proposal: Go 2: errors: allow function on left hand side of assignment #52416
• proposal: Go 2: allow to mark assigments of errors to return immediately when non-nil or to call a handler function #42318
• proposal: Go2: introduce check keyword for errors on LHS of assignment #46655
• Proposal: Go2: Drying up Error Handling in Go #36284
• Proposal: Go2: try: Change try() proposal to use a special character in the position where an error is returned #32884

There has been a proposal for a postfix catch, but it also includes an error assignment on the LHS.

• proposal: Go 2: "onerr return" #32848

There was a proposal for a postfix handle that used a prefix try and an implicit `err variable. And other proposals for a postfix error handler:

• proposal: Go 2: add "or err: statement" after function calls for error handling #33029

There has been a proposal similar to this one with what I would consider the right idea. Unfortunately the proposal was very light on detail and otherwise rushed with mistakes in the examples given: #49091. It had a different syntax: instead of a comma, it used a with keyword. I would be fine with introducing a second keyword, but it does cause more backwards compatibility issues.

I made a similar proposal with a different syntax before there was a process for submitting proposals for changes to the language. The github issue was closed suggesting that I create a blog entry instead. That proposal confused many initial readers by showing problems with existing proposals in code instead of just creating a new proposal (it was later edited). Since then, the additional concept of format strings and error traces has been added (in addition to using try instead of ?).

Appendix: generic enumerations

Now that Go has Generics, we might hope for this to get extended to enumerations and have a Result type like Rust has. I believe that when that day comes we can adapt try to work on that Result type as well just as Rust has its ? operator.

Appendix: Learning from other languages

The try proposed here is very similar to Zig and Swift.

In addition to Swift using try similar to this proposal, Swift also adds try! (panic) and try? (return nil) variants which Go could consider doing adding in the future as well.

Zig has try and also a second keyword catch that is the same as this proposal. try is used with no error handler and catch is for adding an error handler. I avoid this approach in favor of a single try keyword with a 2nd parameter to avoid introducing an additional keyword. Additionally, the catch keyword in Zig comes with a special syntax for anonymous functions. I believe it is best to use existing Golang syntax for anonymous functions (Zig has no such construct). Having a terser syntax for anonymous functions would be great, but I believe it should be universally applicable. I do think it is possible to allow error handlers to not require type annotations.

It could make sense to use a catch postfix like Zig instead of a try prefix, just as Rust uses a ? as a postfix operator. Personally I would be fine with this option (either catch or ?). I think though that when a line is very long it is useful to see the try at the beginning instead of hunting for it at the end.

Zig is very Go-like in its philosophy and could be a great source of ideas of language evolution. It has an error tracing feature that would solve a major pain point in Go error handling: tracing errors. Also an interesting feature called Error Sets that Go could evaluate. Outside of error handling, but related to a common source of crashes in Go, Zig's implementation of Optionals is something that Go should evaluate as well.

Appendix: returning a type other than error

In Zig, catch allows the return of a type other than an error, for returning a default value in case of an error.

const number = parseU64(str, 10) catch 13;

My feeling is that this use case is not common enough to justify support for it. Most of the time I would want to at least log something before discarding an error.

Appendix: code coverage

There are concerns about code coverage. It may be a significant burden for line-oriented code coverage tools to figure out how to tell users if the error paths are getting exercised. I would hate for a helpful tool to hold back back language progress: it is worth it for the community to undertake the effort to have code coverage tools that can determine whether try is getting exercised.

Appendix: in-lining try

This issue is addressed in a comment reply.

Appendix: examples

import (
        "fmt"
)

func existingGo (int, error) {
        // The existing way of returning errors in Go:
        // 3 additional lines to return an error.
        // If we need to do more than just return the error, this is a pretty good system.
        // However, in the simple case of just returning the error, it makes for a lot of ceremony.
        x, err := valAndError()
        if err != nil {
                return 0, nil
        }
        return x + 1
}

func valAndError() (int, error) {
        return 1, fmt.Errorf("make error")
}

func newGo() (int, error) {
        x := try valAndError()

        // Add a handler inline, by itself not much better than the existing way
        x = try valAndError(), func(err error) error {
                return fmt.Errorf("valAndError %w", err)
        }

        // declare and compose handlers separately
        handler := func(err error) error {
                return fmt.Errorf("valAndError %w", err)
        }

        // handler function annotation
        x = try valAndError(), handler

        handler2 := func(err error) error {
                return fmt.Errorf("part 2 %w", err)
        }

        // compose handler function annotations
        // requires a ThenErr library function
        x = try valAndError(), handler.ThenErr(handler2)

        // We can support putting a handler on either side.
        x = try handler, valAndError()

        // The common case wraps an error with a string annotation.
        // So we can support giving a format string as the handler.
        // Wrap an error with a format string. %w or %v is required.
        x = try valAndError(), "valAndError %w"

        // We can support putting it on either side.
        x = try "valAndError %w", valAndError()

        // Try can also be used directly on an error.
        x, err = valAndError()
        try errors.Wrap(err, "valueAndError)"
        // However, that has overhead that using a builtin format string argument could eliminate
        try err, "valueAndError %w"

       // Format string annotations with additional values should use a handler
        i := 2
        x = try valAndError(), func(err error) error {
                return fmt.Sprintf("custom Error %d %w", i, err)
        }

        // Using a custom error type
        // For convenience the error type can define a method for wrapping the error.
        x = try valAndError(), theError{ num: i }.Wrap
}

type theError struct{
        num int
        err error
}

func (theError) Error() String {
        return fmt.Sprintf("theError %d %v", theError.num, theError.err)
}

func (theError) Wrap(err error) theError {
        theError.err = err
        return theError
}

func (theError) Unwrap() theError {
        return theError.err
}

Costs

• Would this change make Go easier or harder to learn, and why?

Harder to learn the language spec because users must learn a new keyword. However, when verbose error handling code is removed, beginners will be able to read and evaluate Go code more quickly and learn Go faster.

• **What is the cost of this proposal? **

The costs are discussed in detail elsewhere

• understanding a new keyword
• requiring go fix for upgrading if we stick with using a keyword instead of a builtin
• code coverage tool upgrading

• How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?

All, but those that re-use Go libraries may just need to upgrade their library usage?
I think these tools would then have less source code to analyze and thus run more quickly.
It will mostly remove the need for linters that check for proper error handling.

• What is the compile time cost?

Without handlers I would think it could be reduced because the compiler can generate code that previously would have been hand-written- the generated code does can be optimized in some ways, for example by not needing to be typechecked. It will reduce the error linting time (see above) for those of us that run linters right after compilation since checking for proper error handling will be easier. Supporting format strings and validating them to contain %w or %v at compile time would have some cost, but we could forego that validation. I am not familiar with the compile time costs of using functions for error handling.

• What is the run time cost?

None without handlers. The potential cost of supporting format strings is discussed in the proposal. I would think that the overhead of the error handlers can be removed by in-lining the functions? Normally once a function is returning an error, runtime performance is not a concern, so the most important thing is reducing the runtime cost of the happy path. It seems like the happy path should be undisturbed if handlers associated with the error path can be evaluated lazily.

• Can you describe a possible implementation?

I started a branch that gives some idea of some of the changes required, but keep in mind that it is incomplete and already making implementation mistakes.

• Do you have a prototype? (This is not required.)

This can be mostly implemented as a library, this is done here.
However, it has several limitations that can only be solved with compiler modifications.

• It uses panic/recover which is slow and requires a defer at the top of the function and using a pointer to a named return variable for the error
• The Check api is okay, but the Try api is awkward.

Try(f(), handler) with no return is fine but with a return things get awkward due to how Go's multiple return values are not programmable and generic varags are not available. The result is : x := Try1(f())(handler).

[DeedleFake]

First of all, I don't think that go fix is a valid way to avoid the Go 1 Compatibility Guarantee. The guarantee is that existing working code will simply continue to work. Users shouldn't need to run go fix, and, worse expect all of their dependencies to run go fix, just to keep things working, especially not when there are other solutions that don't have that issue. The problem would be mitigated to a fair extent by the go directive in go.mod, but if a solution can avoid the problem in the first place, I think that's better.

From what I've seen, most issues with error handling seem to revolve around an inability to apply DRY to error handling mostly itself stemming from an inability to return from the calling function from an inner function easily. A lot of languages avoid this by using an exception system that, much like panic(), causes the entire stack to return continuously until a catch, or a recover() in the case of a panic(), is encountered. This has massive issues, but it does solve that particular problem.

You mention that try is similar to a macro, and I wonder if the problem would simply disappear on its own if Go had its own hygenic macro system? A macro can return from the calling function, since it's not really 'called', and regular usage of it to remove the error return from a function would result in effectively the same thing as an exception system but without all of the hidden magic. There could be a series of standard error-handling macros such as try!() (Yields all but its last argument unless the final argument was a non-nil error in which case it would return from the caller with that error. Other returns would need to be possible via the macro system.) and explain!() (Same thing as try!() but automatically wraps non-nil errors using fmt.Errorf().) and maybe even catch!() (Does what your Zig example from above does.), and then if someone needed repeated custom error handling, they could just write their own. Complicated cases could just keep using manual ifs the way existing code does. Introducing macros and using them for error handling would also avoid the backwards compatibility problem of a keyword.

[ianlancetaylor]

The biggest concern about the earlier try proposal was the unexpected flow of control change in the middle of an expression. That is try f1() + try f2() can return in the middle, a potential return signaled only by the presence of try. This proposal seems to have the same issue.

[DeedleFake]

What if try was only possible at the outermost part of an expression? In other words, it would only be available as the top of a lone expression or the first thing on the right-hand side of an assignment or channel send? However, if a statement contained a try expression, the expression would also be able to use some indicator inside of the expression to pass inner parts back up. That way the presence of try would always be prominent, making it easy to see that an expression might return.

I'm not a huge fan of this particular syntax, but here's a few possible examples of legal and illegal statements:

x := try { f1() } // Legal.
x := f1() + try { f2() } // Illegal.
x := try { f1() } + try { f2() } // Illegal.
x := try { f1() + f2() } // Legal.
x := try { check f1() + check f2() } // Legal.

This might just be too confusing, though.

[gregwebs]

The biggest concern about the earlier try proposal was the unexpected flow of control change in the middle of an expression. That is try f1() + try f2() can return in the middle, a potential return signaled only by the presence of try. This proposal seems to have the same issue.

@ianlancetaylor I tried to follow the previous discussions and address all relevant points. This actually caught me off-guard, so thanks for bringing it up. At least, I read the summaries and this wasn't explicitly mentioned as one of the main concerns. I do see now that one of the summaries linked to a discussion that contained this as part of the text of the #2 point.

I think this concern is a red herring, let me explain why.

Discomfort with inlining try is quite similar to discomfort with reading lines with nested function calls. What happened is that someone was quite comfortable with inlining try (and probably nesting function calls) and created examples of transforming existing Go code and maximalized inlining of try. For example the following was given:

        mimeHeader, err := tp.ReadMIMEHeader()
        if err != nil {
                return nil, err
        }
        req.Header = Header(mimeHeader)

transformed to:

        req.Header = Header(try(tp.ReadMIMEHeader())

However, this is just gratuitous inling- a direct translation is:

        mimeHeader := try tp.ReadMIMEHeader()
        req.Header = Header(mimeHeader)

That is, try does not create any new function nesting issues: users that like function nesting create them. It is true that with try there will be more opportunities for nesting function calls for those that want to. In many cases the biggest blocker of direct function chaining now is having to handle errors in between function calls. With a smooth error handling option in place there are more opportunities for chaining function calls. This is actually a good sign: it indicates that Go code is easier to compose. The community will have to figure out how to best deal with easier function composition, but making function composition readable is not actually a new problem for Go code.

Today the problem of nesting function calls is an issue that is handled by not nesting function calls when it impairs readability. When it improves readability, function calls are placed on separate lines and an intermediate variable is declared. try won't force anyone to stop doing this.

In summary, this is just an issue for a style guide to mention if it does not already. I would be happy to create some linter rules to warn users about gratuitous function chaining, particularly in the context of using try, if existing rules don't work well enough.

[gregwebs]

I have made some minor edits to the proposal text just for making the text a little clearer and adding a few more links, including a link to my prior comment from the proposal. There is no change in substance, but if you are reading it for the first time from an email it would be preferable to come to Github to read it instead.

Thanks to all that have taken time to read the proposal so far! Please try to keep discussions focused on this specific proposal. @DeedleFake I would love a macro system that I could write try with, but that's a whole different proposal! Perhaps you could start putting together a proposal to at least provide compelling reasons for macros (note that go generate is effectively a very limited macro system) starting with a wiki page, or mail list posting, etc and tag me on it.

[atdiar]

Unfortunately, I don't find the examples very compelling.
Currently, the consistency of the if err! = nil makes it easily recognizable at a glance.

I am more inclined to keeping things simple and consistent if a bit boilerplate-y.
Correctness over stylistic considerations.

[apparentlymart]

While reading this proposal I developed a (probably incomplete) mental model of what you were describing which I wanted to repeat here in case I misunderstood (please let me know!) and in case it's helpful to others who prefer to think in "rules" rather than in examples.

What I understood:

• try is a keyword that can be followed by any expression whose type is error, or specifically by a function call expression where the final return value of the callee has type error. I'll call this expression "V" (for "value") below.

• The required expression may optionally be followed by a comma and another expression which produces a value of type func (error) error. If the comma and second expression is emitted, the compiler behaves as if the expression were func (err error) error { return err } (the identity function). I'll call this expression "T" (for "transform") below.

• If V is an expression producing a single error then the compiler generates code that behaves as if the following:

  hiddenTemp := E
  if hiddenTemp != nil {
      return T(hiddenTemp)
  }

  The try operator presumably has no result in this case, as if it were a call to a function with zero return values.

• If V is a call to a function F with N return values where N >= 2, whose final return value is of type error then it seems a little harder to write out some idiomatic Go code that it translates to in these general terms (rather than specific examples), so I'll use prose instead.

  try in this case is an expression producing one fewer return value than F, exactly matching the first N-1 return values of F.

  It calls F with the given arguments and then tests if the final return value EV is nil. If so, it produces the remaining return values as its result and control flow continues as normal. If non-nil, it behaves as return T(EV) and therefore produces no result because control returns to the current function's caller.

• As a convenient shorthand (and possibly a performance optimization for this common case), the compiler might allow T to instead be an expression that produces string. If so, the compiler behaves as if T were func (err error) error { return fmt.Errorf(T, err) }.

  It's not clear to me from the proposal if the intent was to require V to be specifically a string literal in this case or whether any expression which produces a string would be allowed. For example, would it be valid to declare const msg = "foo: %w" and then use msg for T? Would it be valid if msg were instead a string variable?

I think the above covers the main proposal text at the time I'm writing this. There's subsequent discussion about the possibility of making try instead behave as if it were a part of the assignment and channel send operators, or in other words that this keyword would be permitted only after :=, =, and <-. As far as I can tell, that modification doesn't affect the rules for try I described above, and instead just restricts where the keyword try would be accepted at all.

---

With my understanding above in mind, I have some thoughts and follow-up questions (which are likely to be invalid if my understanding above was incorrect):

• If I have a weird function func foo() (error, error), am I allowed to write try try foo() to shorthand handling both of those errors in a single statement?

  • If so, and if I want to provide an explicit transform function called wrapErr, would that look like try try foo(), wrapErr, wrapErr? (Would this be a reason to prefer using a function-call-like syntax to make the nesting clearer? try(try(foo(), wrapErr), wrapErr))

• This feels familiar to me by analogy to Rust's postfix ? operator, so I feel more comfortable with it than some others might be if they are coming from other languages which favor expression-based control flow.

  However, it does still feel somewhat unprecedented to allow an implicit return statement inside an expression.

  Perhaps an existing counter-example is that function call expressions can already potentially panic, which is a funny kind of return to caller. With that said: if this ends up being syntactically similar to a panicking function aside from the additional keyword try, might that suggest that the distinction between exceptional return and normal return is not as strong as initially intended? Or to say this another way: does this terser syntax make it harder to teach the difference between returning an error and panicking, for people who are familiar with exception-based control flow in other languages?

• One potential hazard with using the word try is that it might appear as a false cognate to someone who is familiar with languages which employ structured exception handling. Might it be better to select a different keyword or symbol which doesn't have such baggage? Or do you think there's sufficient precedent in other languages to use the word try in this different way?

  (The proposal does mention that both Zig and Swift use the word try in a way more like this proposal. I am not familiar with either of those languages, so I'm not meaning this question to dismiss those analogies; those are probably good examples of other uses of try that could argue against this point.)

• A part of the proposal I didn't touch on in my description above was the "error tracing" section. If I understood correctly it seemed like that was proposing that try would annotate the errors it returns with some sort of callsite location information.

  Using the naming scheme I employed for my description above, would you think of that as being an extra wrapper around the function given in T which implicitly adds another level of wrapping to capture the trace information? Does that then mask the true error type returned by T?

  Alternatively, perhaps your intent was that the function given in T could optionally have a second argument which somehow represents the trace information, in which case the compiler would generate some extra code to gather that information and pass it in to the translation function, with it then being up to the translation function to decide how (and whether) to incorporate that information into the resulting error.

I'm sorry for asking a bunch of unrelated questions in one comment; I know that GitHub's comment UI doesn't really work very well for this sort of multi-threaded discussion but I also don't want to spam the issue with lots of separate comments. 😬

[gregwebs]

@apparentlymart good questions. See my next comments for a formalization of the transformation.

It's not clear to me from the proposal if the intent was to require V to be specifically a string literal

Good question. If someone wants to re-use a string variable then it is probably becoming worth the effort of using a function, so we may not need to support a string variable. I think it would simplify the first implementation to only allow a literal, and then we could consider relaxing that in the future.

There's subsequent discussion about the possibility of making try instead behave as if it were a part of the assignment and channel send operator

This is good for the simple case, and I would gladly take it over the status quo, but adding error handling to this is difficult.

If I have a weird function func foo() (error, error), am I allowed to write try try foo() to shorthand handling both of those errors in a single statement?

That should be allowed according to how try operates. I haven't ever seen a Go function that returns two errors though. A better example might be to try to pair the try keyword with a different Go keyword. I am glad you are trying to poke holes in this- I agree that if we think through all possible use cases we might consider it necessary to use parentheses, in which case we can implement it as a builtin.

With that said: if this ends up being syntactically similar to a panicking function

A Go panic is equivalent to an exception. An exception is very different because it unwinds the stack to find a handler up the call chain. This try returns directly to the call site.

Might it be better to select a different keyword

I personally don't care what word we use. We could tentatively accept a proposal with the provision that there will be a further decision as to the exact word that is used and whether it is a builtin or a keyword.

[gregwebs]

... proposing that try would annotate the errors it returns with some sort of callsite location information. Using the naming scheme I employed for my description above, would you think of that as being an extra wrapper around the function given in T which implicitly adds another level of wrapping to capture the trace information? Does that then mask the true error type returned by T?

That is what some error packages do now, and it works fine since we have standardized on wrapping now. However, it isn't necessary in this case because we are implementing this in the compiler itself. You can follow the link to read how Zig implements it: a function that returns an error will have the caller pass in a hidden stack trace variable that try will write to before returning.

Thanks again for the good questions. I will try to roll up some of these Q&A into the proposal.

[gregwebs]

Sorry, I wasn't thinking straight when I wrote down the translation. Also, I don't think we should avoid assignment to non-error variables when there is an error. So the translation would be.

func(args...) (returnTypes..., error) {
        x1, x2, … xN = try f(), handler
        ...
}

turns into the following in-lined code:

func(args...) (rtype1, ... rtypeN, error) { // rtype1, ... rtypeN are the return types
        x1, … xN, errTry := f() // errTry should be invisible or have a unique name
        if errTry != nil {
                return reflect.Zero(rtype1), ..., reflect.Zero(rtypeN), handler(errTry)
        }
        ...
}

[apparentlymart]

Thanks for the nudge to read the Zig documentation! I'm sorry I didn't follow that link on my first read; I was taking a broader look then and so it wasn't clear to me quite how important that link was to the subsequent discussion.

I don't yet have any experience with Zig so I did a bit of homework to understand how Zig thinks about errors and how that is similar or different from Go. Here are my notes in case they are useful to others reading this proposal who are also not Zig-experienced:

• An error in Zig is really just an integer value from a global enumeration generated internally by the compiler. Different parts of a program will declare their own error sets which add additional members to that hidden enum. Therefore the individual error sets are essentially erased by the compiler, because each distinct error type has its own hidden integer value unique across the whole program. When returning an error the result is just that integer, and so there's no facility for carrying other data along with the error.

  This is different from Go where error is an interface type and so "error values" are interface values containing types that implement error. An error value can carry arbitrary data with it, and we typically expect error values to be self-contained and sufficient to fully describe the context of the problem.

• Fallible functions in Zig return "error union types", which are written as E!T (type T or an error from error set E). It's often possible to write just !T because the compiler can inspect the error return statements in the function and automatically infer which error set they all belong to.

  I understand this as being a special-cased sum type, like Result<T, E> in Rust. This is different from Go where we conventionally represent fallible functions by returning multiple separate return values where the last one is of type error.

• The "Error Return Traces" design which is linked in the proposal seems to rely on the fact that "error union types" are a first-class concept in Zig. When calling a function which returns such a type (a "fallible function"), the compiler generates different code than it would generate for an infallible function. It also generates different code depending on whether the caller of a fallible function is also fallible. What I understood is:

  • If an infallible function calls a fallible function, the compiler secretly allocates a "stack trace" data structure on the stack and passes a pointer to it as a hidden argument to the callee.
  • If a fallible function calls another fallible function, the caller must already have a pointer to a "stack trace" data structure due to the previous rule, and it propagates that same pointer into the callee as a hidden argument.
  • If the callee is an infallible function, no additional hidden argument is passed, regardless of whether the caller is fallible or not.

  The error trace UI shown in the Zig docs is, I assume, the way the runtime handles the main function itself being fallible. The docs don't get into this level of detail, but presumably the startup code that runs before main knows that main is fallible and allocates the stack trace data structure before the user program even begins running, and then checks that data structure after main returns.

  Any functions which calls an infallible function can call a function errorReturnTrace to obtain the stack trace pointer to access any accumulated trace. I guess this means that the compiler treats this as a builtin which returns the current function's stack trace pointer, which either points to somewhere in the current function's stack frame (if it's infallible) or was passed in as a hidden argument (if it's fallible). I also infer from this design that each function has zero or one stack trace pointers: if a single function calls multiple fallible functions then they all receive the same stack trace pointer.

  In today's Go, the distinction between fallible and infallible functions is largely just a convention: if the type of the last return value is error then the function is fallible. All of the context about the error (including stack traces or other traces as needed) is inside the object that the error value refers to, so there isn't any implicit second return path for tracing error return metadata.

My intention in pointing out these differences is not to argue that Zig's design is not applicable to Go, but instead just to identify some areas where the design might need to be altered to fit into Go's type system and conventions.

The need to generate different code for fallible vs. infallible does seem to conflict with the fact that "fallible" exists only as a convention in Go. It does seem possible in principle for the compiler to treat functions where the last return value is of type error as special, but subjectively it feels to me like that goes against the grain of Go's language design so far. Perhaps it's reasonable to argue that any sort of first-class error handling requires treating error-returning functions as special, rather than just a convention. What do you think?

[DeedleFake]

It does seem possible in principle for the compiler to treat functions where the last return value is of type error as special,

Would that mean that something like errors.New() or fmt.Errorf() would be fallible?

[gregwebs]

The need to generate different code for fallible vs. infallible does seem to conflict with the fact that "fallible" exists only as a convention in Go. It does seem possible in principle for the compiler to treat functions where the last return value is of type error as special ...

Yes,try requires this convention. So doing this with error tracing as well isn't an issue. This convention is extremely universal in Go code, to the point that I would expect future proposals that attempt to formalize this convention. But as mentioned in this proposal, I think the right time for this is when generics support enumerations and we can have a Result type like Rust.

Would that mean that something like errors.New() or fmt.Errorf() would be fallible?

I think the practical implication of this question is: will the error trace go down all the way into a function such as errors.New(). The answer is yes for a simple implementation. However, I think the implementation can be developed to eliminate that from the trace. A function that returns just an error (and can never return a nil) can probably be automatically removed from the trace.

[gregwebs]

There are 2 approaches for instrumenting error tracing. The one I originally had in mind was to only use try to instrument tracing, in which case errors.New would certainly not be instrumented. But the other approach we just discussed would be to automatically instrument functions even without the usage of try. It does seem that error tracing could be made automatic based on the established Go convention of returning error in the last position, in which case there would be no dependency on try and it could be removed from this proposal and made into an entirely separate proposal.

[apparentlymart]

@gregwebs I was trying to focus on the Zig-like approach in my last comment because I was responding to your callout that I hadn't properly read it the first time. 😀

But indeed there do seem to be at least two approaches here, and possibly three if you count the two variants I mentioned in my original long comment here. Here's a summary of those three in the hope of making it clearer for other readers:

1. Zig-style: the compiler generates additional code to capture error traces out of band of the actual error values, using hidden arguments to any fallible functions.

   Although not explicitly stated yet, I'm assuming this would include allowing func main() to be a fallible function and making the runtime print out the error trace if that function does indeed return a non-nil error.

   If user code needs to interact with the error trace then it must do so via some intrinsic-function-based API that returns the current function's out-of-band trace object, because that information would not be directly accessible through the error value itself.

2. try keyword automatic error wrapping: as a hidden extra step after evaluating the error transformation function, try would then wrap that result in a special built-in error type which captures some sort of trace information.

   That special trace error type would implement the existing error unwrapping interfaces so that calling code can peel off the wrapper in the conventional way to obtain the original error value inside it.

   Presumably if a particular error is returned through multiple levels of try then either each try adds another level of wrapping or the wrapping code notices that the error is already of the special error type and shallow-replaces the wrapper with a new wrapper that aggregates the two levels of tracing together. Either way, the error that emerges at the other end allows enumerating all of the try operators that the error returned through.

3. try keyword with explicit error wrapping: the error transformation function can take an optional extra argument which represents trace information generated by the try expression.

   A transformation function which takes that extra argument can therefore incorporate the information from it into the error value it returns in whatever way makes sense for that error type. The result can therefore be of any dynamic type, without any implicitly-added wrapper that callers must peel off to recover the original error.

I suppose there is also an implied additional option of not trying to support "error tracing" as a first-class feature at all, and instead leaving that as something to be handled by library code using existing runtime mechanisms. I've not thought too hard about what the implications of that would be, but at first glance it does seem like an automatic tracing mechanism is not really a crucial part of the proposal, and that having the shorter syntax without the explicit control flow constructs is the main goal.

(I imagine this discussion should aim to reduce this to a single specific proposal -- whether one of the three above or another approach -- but it'll probably take a little more discussion to figure out which of these has the best combination of pros and cons.)

[gregwebs]

I favor the Zig out-of-band approach because

• it works with existing code
• wrapping has greater overhead and changes the type

I removed error tracing from this proposal. Automatic error tracing would be an amazing new feature for Go. Let's not discuss it anymore on this proposal though other than to facilitate creating a new proposal where it can be discussed. Link your new proposal here or contact myself or us on this issue if you want help making a new proposal.

[gregwebs]

I forked the err2 package to implement this as a library that I am calling err3.
However, it has several limitations that can only be solved with compiler modifications.

• It uses panic/recover to accomplish a non-local return which is slow and requires a defer at the top of the function and using a pointer to a named return variable for the error
• The lack of generic support for variadic args or support for handling multiple return values as a tuple requires defining a function for each possible number of return values (Try, Try1, Try2).
• Multiple return values will explode into function call arguments, but no additional arguments can be supplied

As a result, the CheckX api is okay, but the TryX api cannot take a handler as an argument. This is solved by returning a function that handlers that are applied to, which is not an ideal API. Try(f(), handler) with no return is fine but with a return the API is: x := Try1(f())(handler).

[bitfield]

Now that Generics have been released, the biggest challenge for Go developers is listed as "Error handling / working with stack traces".

“Biggest” here might need a little qualification. 11% of those responding to the ”biggest challenge” question mentioned error handling, but only 665 people responded to that question. So in terms of actual numbers, roughly 73 people out of the 5,752 surveyed find error handling (in some way) their biggest challenge in using Go. That is, again roughly, 1% of respondents.

So another, I accept less persuasive, way to interpret this data would be to say that around 99% of survey respondents do not find error handling a significant challenge in using Go.

This has no bearing on the merits (or otherwise) of this specific proposal, of course, but I suggest it has some bearing on whether it's worth anyone's while to make any proposals about changing error handling. To say ”it's the biggest challenge” is to run the risk of being misinterpreted as saying “it's a challenge for a large proportion of Go users“, which the survey data suggests very strongly is not the case.

[steeling]

+1 to removing the auto tracing from this proposal. I'd also recommend dropping ThenErr, as it doesn't seem critical to the core concepts here, and can be added at a future date if desired. My fear is that it detracts from the core of the proposal if people take issue to that specific component. Maybe framing it as a "potential future direction" could help.

[steeling]

In similar proposals the concern was raised that by "sandwiching" the function call with a keyword (try), and a handler function, it detracts from the readability of the program, ie:

w := try os.Create(dst), func(err error) error {
        os.Remove(dst) // only if Create fails
        return fmt.Errorf("dir %s: %w", dst, err)
}

makes the os.Create call a bit harder to see. One thought to improve this would be to make the try keyword implicit if a handler is provided such that the above becomes

w := os.Create(dst), func(err error) error {
        os.Remove(dst) // only if Create fails
        return fmt.Errorf("dir %s: %w", dst, err)
}

However that would have some syntax collision with multiple assignments, which would necessitate the need for a syntax change. A lot of symbols/keywords would work, but I'll use the @ symbol for the sake of the argument, so we now have:

w := os.Create(dst) @ func(err error) error {
        os.Remove(dst) // only if Create fails
        return fmt.Errorf("dir %s: %w", dst, err)
}

[ianlancetaylor]

Thanks. This proposal doesn't address one of the biggest concerns about the try proposal (#32437) which is that the flow of control can be buried inside a complex expression. It does address other concerns about try, but not that one. Therefore, this is a likely decline. Leaving open for four weeks for final comments.

[steeling]

"Buried flow control" feels sort of subjective IMO. At its core, it's a commentary on readability that claims having explicit flow control is more readable than cluttered, albeit explicit if err != nil {...} statements. I'd argue that it's up to the user to write readable code, and certainly more readable code can be produced by introducing try as a tool. If a user decides to write difficult to read code, they can already do so without the use of try.

Plus as commented in that proposal, panic affects flow control already.

Lastly, go-vet, or go-lint, or even the widely used golangci-lint can add checks to recommend/restrict usage of buried try calls for users that want to protect against it.

[ianlancetaylor]

Understood. Still, that was one of the primary reasons for rejecting the try proposal, and this proposal has the same concern. We aren't going to revisit that decision unless there is new information.

[apparentlymart]

The subsequent discussion included the possibility of making try be allowed only in some specific contexts: normal assignment, channel writes, and as the first token in a statement.

try EXPR
foo := try EXPR
ch <- try EXPR

Would that be a sufficient constraint to ensure that the control flow implied by this feature is not buried inside complex expressions?

(I don't really have strong feelings either way about this variant of the proposal, so I'm raising it only in the hope of moving discussion forward towards something that might qualify as "new information", and not as a vote for or against it.)

[gregwebs]

Thanks. This proposal doesn't address one of the biggest concerns about the try proposal (#32437) which is that the flow of control can be buried inside a complex expression. It does address other concerns about try, but not that one. Therefore, this is a likely decline. Leaving open for four weeks for final comments.

@ianlancetaylor I believe you already raised this concern and I responded to it here. And another user made the same point.

But I haven't seen any reply to this point (solve it with a style guide) it in this discussion or elsewhere. I may have missed it- there are hundreds of comments. Rather than not responding to this point and closing the proposal, I was hoping for a thoughtful response as to why this cannot be handled by a style guide, gofmt, and linters.

Let me re-phrase the point again here: It is up to programmers (with the aid of style guides, formatters, and linters) to write their code in a readable way. try does not cause things to be buried in expressions, programmers do.

You can write code the exact same as now but just turn the if expression into 1 line instead of 3.

x, err := f()
try err, "f failed"

Sometimes this is the best because the code is already indented and f() is a larger expression that takes up a lot of space.

I agree with @apparentlymart that the usage of try can be limited to good style, but I think as a base case that could be done with gofmt or golangci-lint rather than putting restrictions into the compiler itself.

BTW, I updated my library implementation of this to be named try.
It is quite a productivity booster around error handling once you get used to it. Unfortunately because it is a library that cannot implement try properly there are some awkward things to get used to.

I am beginning to use this code in the real world. I have < 100 usages and so far, but I haven't found a single opportunity to do anything other than have it be the first expression on the RHS of an assignment or the leading statement of the line.

I need to make a tool to transform error handling in a code base and am hoping to leverage the go fix tooling. I think this logic around recognizing error handling could be ported to gofmt, but even if not, re-formatting to use an intermediate variable seems like it should be possible?

[ianlancetaylor]

@apparentlymart Restricting try to only certain uses makes the language more complicated and is non-orthogonal. Go aims to be simple. Adding extra rules that everybody most learn is not simple.

@gregwebs Sure, people could use a style guide. But not everybody will. Somebody reading unfamiliar Go code must always be aware of the possibility that a try appears embedded in a complex expression.

I want to emphasize again that we have to be able to make decisions and then move on. If we must reconsider these decisions over and over, we can't make progress. We've made a decision here.

[apparentlymart]

The decision I understood as having been made, and was aiming to respond to, is that it's not appropriate introduce any language constructs which have an effect resembling return in an expression position, because in Go we are accustomed to only statements affecting control flow. (With the notable exception of the panic function, which is exceptional for various reasons already discussed at length in #32437 and elsewhere.)

Is it true that the there has been a more general decision that values of type error should never receive special treatment in the language, and should instead remain purely a matter of convention and idiom?

I can understand the (assumed) objection that any construct which is used only to deal with values of type error (or is otherwise focused only on error handling) is inherently non-orthogonal in a language where errors are just values and all of the patterns around them are a matter of convention. I just want to confirm whether that is indeed a specific objection.

My goal is to help those who wish to make future proposals in this area understand what bar they need to meet to be considered new information rather than a rehash of previous discussion, or to be clear that such proposals are simply inconsistent with the language design goals and so not worth making in the first place. I find it upsetting to see so many people spending time proposing, counter-proposing, and discussing in a direction which is (as far as I can tell) not a desired language design direction; I would like to help folks making these proposals to "move on" if this is a settled matter already.

I acknowledge that language design is a hard problem requiring consideration of many different conflicting goals. I'm not meaning to imply that it's possible to rule definitively against all solutions that might hypothetically be proposed, but I hope that "avoid any language features that treat error-typed values as special" is narrow enough to be answerable as a potential design principle that could help divert effort to proposals which take a different approach, along with considering the already-understood "no new control flow inside expressions" principle.

Thanks!

[ianlancetaylor]

The decision I understood as having been made, and was aiming to respond to, is that it's not appropriate introduce any language constructs which have an effect resembling return in an expression position, because in Go we are accustomed to only statements affecting control flow.

Yes.

I would add that it's not going to be OK to introduce a language construct that can change flow control in an expression, and then say "people should only use it in these ways, as a style guideline." And it's not going to be OK to introduce a language construct that is parsed as an expression, and then say "but this thing that looks like an expression is not permitted where we otherwise permit any expression."

Thanks.

[apparentlymart]

Thanks for the additional clarification about the objection, @ianlancetaylor.

My conclusion from this is that it might still be acceptable to propose new statement-level syntax for error-specific control flow. It is specifically anything "expression-like" that is ruled out by previous decisions. I think that's a helpful constraint for future proposers to consider, though I do now wonder whether there's a better place to record that information than in this specific issue's discussion thread! Is there anything like a central "decision log" for proposals team decisions related to particular common subjects that have accumulated many separate proposals?

I suspect that part of why these seemingly-redundant proposals keep coming up is that it's often hard to understand exactly what has been decided before, because the decisions are a relatively small detail in otherwise-long discussion threads for specific proposals.

[ianlancetaylor]

I agree that new statement level syntax is a possibility.

There is some general discussion in #40432.

[gregwebs]

I suspect that part of why these seemingly-redundant proposals keep coming up is that it's often hard to understand exactly what has been decided before, because the decisions are a relatively small detail in otherwise-long discussion threads for specific proposals.

This is exactly what happened to me. I read through hundreds of comments and many other proposals. There was one thread (among many others) with this complaint, and one summary that mentioned the thread. The summaries indicated that it was a collection of issues as a whole that lead to the proposal being rejected rather than there being any single issue that had been decided on.

If these decisions, or any other hard constraints were listed somewhere, then we might be able to have a well-functioning community proposal process. This is the second time I have tried to navigate the process. The first time My proposal was closed- I found out there was no process and that you had to be a member of the Go team to make a proposal. This time there is a process- but it still seems that in at least some cases you have to be a member of the Go team to understand what things have been decided on.

Every project needs help with staying organized and writing things down. I think the community could help. I would gladly submit documentation of this decision if I knew where. Maybe here? https://github.com/golang/proposal/blob/master/design/go2draft-error-handling-overview.md

[gregwebs]

Okay, I see the issue now that @ianlancetaylor linked to about error handling mentions this decision as "major". Unfortunately, I missed that issue. Perhaps we can edit the other issues/proposals and link to it at the top of the description?

[ianlancetaylor]

Error handling in particular has unfortunately become an enormous mess, as can be seen by looking at https://github.com/golang/go/issues?q=label%3Aerror-handling+ . It's an area where there are a vast number of variants of approaches, and nobody can keep up with all the details of all the earlier discussions. I wrote #40432 as an attempt to put a bit more order on the topic, but I don't know that it has helped much in practice.

I have no objection to adding a mention of #40432 on existing issues, but it sounds like a tedious process. Perhaps I will try to remember to mention the issue when adding the error-handling label to issues.

[gregwebs]

So it is normally poor form to discuss alternatives on a proposal. However, this proposal was apparently dead on arrival so I will attempt to explain the minimal adaptations to adapt it to the constraints that we now understand. If a new Phoenix arises from the ashes of this discussion, a new proposal will be needed.

An intermediate err variable

The easy way to satisfy the constraint of not being an expression is to remove the ability for try to return anything (that is what an expression does). So what you have left is this proposal, but only the cases where try is only given an error. So this proposal proposed:

x := try f(), "f failed"

but instead we could only allow for:

x, err := f()
try err, "f failed"

After using my library a bit, I actually prefer this approach the most. And to automatically translate existing code, it is often necessary. A lot of the reason why it was nice to hide the err variable in the original proposal just comes down to the Golang's issues around variable reuse during assignment. However, these issues are already present and can be solved with linters.

Error handler functions

Above I show the special string to error handler version mentioned in the proposal. But let's discuss error handler functions.
We need to choose between having the second argument be a lazy expression of type error or be a handler function of type func(error) error.
lazy:

// concretely
x, err := f()
try err, fmt.Errorf("f failed: %w", err)
// generally
x, err := f()
try err, handler(err)

It's not quite DRY to use this same variable multiple times. With go's scoping issues, I could see this ending up creating a defect where the variable used in the handler is different that the one given to try. If we support an error handler function func(error) error, then we have just:

x, err := f()
try err, handler

The lazy form can be used this way by currying the function so that it needs error to be applied.

It is also possible that the keyword can support both a lazy error expression and a handler function func(error) error.

Assignment

Even if we support the intermediate variable form, it isn't necessary for a function that returns only an error to assign it to an intermediate variable.

try f(), "f failed"

It would be weird to not support this. But supporting this means that when f() is changed to returns a result in addition to an error, we will have to create a second line. This type of workflow already happens in Golang with respect to errors, but if we are going to be consistent we would need to require assignment. The original proposal suggests this form:

x := try f(), "f failed"

This makes try look and behave like an expression, which has been decided against. Is there a way instead to make the concept of a "try assignment" statement?

try x := f(), "f failed"

But we still must support the intermediate error variable form as well, because that is best for readability in many cases, so we will end up with this keyword accepting 2 very different things as its first argument.

Assignment and Lazy handlers

There is a way to force users to use try in only one way. That is to make the error handler lazy (as mentioned above) and require an error handler argument. This way err must be assigned. To fully commit to this approach, we would need to not have the special string error handler and require an error handler. So the simplest form of just returning the error would be:

err := f()
try err, err

This looks strange, although it is the same strangeness of today with if err != nil { return err } . But it's kind of strange that a new, better language construct would look like this.

A way forward

I think we have to allow for the single line try f(), ... when the function only returns an error. Adding a new return variable to f is painful, but it is more rare that a function returns only an error and then adds a non-error result. Normally a function has 1 non-error result and then adds more.

// just error can avoid assignment
try f(), "f failed"

// if there is a non-error result, all the function returns must be assigned to variables
x, err := f()
try err, "f failed"

For the error handler, I would personally prefer not supporting the lazy form and having curried functions. The equivalent of the special string form can be written as a curried function:

func Fmtw(msg string) func(error) error {
  return func(err error) error { return fmt.Errorf(msg + ": %w", err }
}

try f(), Fmtw("f failed")

Error handlers then support both the single line without assignment and the 2 line with assignment forms.

[gregwebs]

Okay, thank you all for your feedback on this. I made a new proposal with try as a statement. I think this is the solution that addresses all the community feedback on error handling in a Go-like way and that if you look at the proposal is what the community really already converged on, but it just hasn't been written down coherently with all the pieces together yet.