proposal: Go 2: deriving code ala Haskell

[fkuehnel]

It's already discussed in 19412 that sum (discriminated union) types are nothing for GO 1.xx. However, unrelated to this is the fact that I find myself frequently writing boilerplate code for the interface type replacement of a discriminated union:

type Node interface {
   Show() string
   Read(string) bool
}

// here are some Node implementations
type IntNode struct {
   value int64
}

func (n IntNode) Show() string {
   fmt.Sprintf("%d", n.value)
}

func (n IntNode) Read(s string) bool {
   if val, err := strconv.ParseInt(s, 10, 64); err != nil {
      return false
   } else {
     n.value = val
     return true
  }
}

// The boilerplate code becomes worse with const types
type MathConstant uint16
const (
   Pi MathConstant = iota
   Euler
   Catalan
)

type MathNode struct {
  value MathConstant
}

func (n MathNode) Show() string {
  switch n.value {
  case Pi:
     return "Pi"
  case Euler:
     return "Euler"
  case Catalan:
    return "Catalan"
  }

  return "Must never happen"
}

func (n MathNode) Read(s string) bool {
  switch s {
  case "Pi":
    n.value = Pi
  ...
  default:
    return false
  }
  return true
}

And this is just a small example of how it could go.

Now to the Haskell deriving inspired proposal:

The idea here is like in Haskell to have a well-defined way to generate common implementations, i.e. Show, Read for algebraic types. This makes code reviews much easier and helps avoid making errors. Obviously, writing and reading less code is beneficial. Here, one could eliminate most of the boilerplate code with the 'deriving' syntax,
but one could still choose to implement custom versions if needed. This means it would be backward compatible to GO 1.xx versions:

type Node interface deriving (Show, Read)
// This will autogenerate functions with the following signature
// Show() string
// Read(string) bool

type IntNode struct {
  value int64
} deriving (Show, Read)

type MathConstant uint16
const (
   Pi MathConstant = iota
   Euler
   Catalan
)

type MathNode struct {
  value MathConstant
} deriving (Show, Read)

[atdiar]

You can probably achieve that without a language change.

Just have a type parametered Node and a type switch in the definition of Show and Read for the different implementations?

[fkuehnel]

You can probably achieve that without a language change.

Just have a type parametered Node and a type switch in the definition of Show and Read for the different implementations?

I don't think this is really eliminating the need to write boilerplate code. If I want to use the above code example to write:

type MyToy struct {
  n Node
  ...
}

node := &MyToy{n: &MathNode{Pi}}
fmt.Println(node.n.Show())

[atdiar]

@fkuehnel

You can use embedding if you want MyToy to expose the same interface as Node (method promotion)

type MyToy struct {
  Node 
  ...
}

[ianlancetaylor]

type Node interface deriving (Show, Read)

I don't understand what that means.

[fkuehnel]

type Node interface deriving (Show, Read)

I don't understand what that means.

This is a language suggestion that stands for

type Node interface {
  Show() string
  Read(string) bool
}

[fkuehnel]

@fkuehnel

You can use embedding if you want MyToy to expose the same interface as Node (method promotion)

type MyToy struct {
  Node 
  ...
}

Sure, but this proposal is to have the compiler autogenerate code for really common-use functions, i.e. Show, Read, or even more (see Haskell deriving capabilities) for algebraic types.

[DeedleFake]

stands for

Where are the full function signatures for Show() and Read() defined?

[fkuehnel]

stands for

Where are the full function signatures for Show() and Read() defined?

I added the signatures to the example above:

type Node interface deriving (Show, Read)

Is equivalent to

type Node interface {
  Show() string
  Read(string) bool
}

[fsouza]

I added the signatures to the example above:

But where/how do you define Show and Read? How do you define what methods will be injected in the type and what their signature will be? And the implementation too, as the proposal indicates that deriving may also be used in structs?

[fkuehnel]

Well, this is not really magic since Haskell shows that it is possible to autogenerate canonical functions, Show, Read and the Go compiler also has the information about simple and algebraic types (structs) at compile time. This can be used to facilitate autogeneration functions Show, Read... at compile time.

[rittneje]

@fsouza Going by my vague memory of Haskell, I believe the intent is that Show and Read are "interfaces" (actually typeclasses, but Go doesn't have those).

type Node interface deriving (Show, Read)

is roughly like

type Node interface {
    Show
    Read
}

type Show interface {
    Show() string
}

type Read interface {
    Read(string) bool
}

but also Show and Read would be "magic" interfaces built into the language itself such that the Go compiler knows how to generate a reasonable default implementation for those methods for any arbitrary type. Consequently, you can only use deriving with a small set of pre-canned interfaces.

At least for Show, I'm not sure why fmt.Sprint and friends are not a sufficient alternative in practice.

[fkuehnel]

Stricly, there is not a 1:1 analogy to Haskell, but in Haskell 'deriving' is used in conjunction with defining data types. Haskell doesn't have interfaces but type classes, that's not too important here. Indeed, there would be still some magic here such that the GO compiler would be able to generate a good enough default implementation for all simple and struct data types.
I didn't state it'll be super simple to implement this proposal. An application for this is parsing and building syntax trees...

[fsouza]

In general, I don't think the feature is a good fit for Go. The fact that it'd require assist from the compiler makes it worse IMO. Like, I think that it wouldn't be as bad if this proposal also included information about this could be implemented by libraries, but at that point you'll probably be proposing macros, which have been proposed before.

I don't think the use case for parsing/building syntax trees is common enough in the Go community to justify baking such "magic" into the compiler.

[rittneje]

Personally I think go generate meets this kind of use case in a better way. See for example https://github.com/abice/go-enum

[sirkon]

Needed derivation a while ago.

My usecase:

• I had a custom structured logging library similar to zerolog without its quirks (at the cost of occasional allocations here and there) at my last job.

  log.Err().Int("invalid-value", -1).Msg("the value must not be negative")

• Logging should be done carefuly, it is an anti-pattern in services to log errors as soon as you get it out of some function - just enrich it with context, add annotation and return upwards - it will be logged with a logging middleware further.
• With structured logging we can add structured context, we would miss it with std errors. We did our custom errors library where our errors hold structured context as well:

  errors.New("the value must not be negative").Int("invalid-value", -1)

  Logger is aware of it and will show invalid-value.

Besides generic Int, Str, Any, etc, we have predefined keys:

return errors.New("the action is not allowed for the user").UserID(userID)

The problem we should implement these methods for both Logger and Error. It would be nicer to have something like

type Contexter interface {
    Int(key string, value int)
    Int8(key string, value int8)
    ...
    Bool(key string, value bool)
    Any(key string, value any)
}

// UserID extension method for Contexter.
func (c Contexter) UserID(userID string) Contexter {
    return c.Str("user-id", userID)
}

Both logger and error instance implements Contexter, then, when used with something like this

type Logger(Contexter) struct {
    ...
}

would add UserID extension method for an instance of Logger as soon as it implements Contexter. Same for custom Error.

[atdiar]

The custom logger can't embed the default logger?
Looks like the perfect job for composition but I don't know about your full requirements.

[fkuehnel]

Personally I think go generate meets this kind of use case in a better way. See for example https://github.com/abice/go-enum

Indeed, go generate could do code generation. I'm not certain if this can also address code testability and correctness.

[sirkon]

The custom logger can't embed the default logger? Looks like the perfect job for composition but I don't know about your full requirements.

Reply to my post post? Well, not exactly, the following is possible:

return errors.New("error message").Str("key", "value").UserID(userID)
...
log.Err().Str("key", "value").UserID(userID).Msg("error message")

It needs and supports chaining, which is exactly the same for logger and custom errors with the only difference: UserID of logger returns logger, UserID of error returns error. Embedding can't do this.

[atdiar]

Ok I see. I am not sure that the problem can even be solved by the current proposal since the method signatures are different.

[rittneje]

@sirkon I believe the crux of your problem is the desire for method call chaining. If not for that you could just define a normal function like so, yes?

func UserID(c Contexter, userID string) Contexter {
    return c.Str("user-id", userID)
}

To that end, just wondering if something like this would meet your needs.

type ContexterOption func(Contexter) Contexter

type Contexter interface {
    ...
    With(ContexterOption) Contexter
}

func UserID(userID string) ContexterOption {
    return func(c Contexter) Contexter { return c.Str("user-id", userID) }
}

Then instead of c.UserID("abc") it would be c.With(UserID("abc")).

[sirkon]

@rittneje it will of course, and I considered this way at one point. Not with closures, options were a struct like

type Option struct {
    Name string
    Type OptionType

    // Only one of these value fields below are to be used depending on the Type
    Str  string
    Int  int64
    Flt  float64
    Any  interface{}
}

and a set of constructors.

But, it is too verbose to my taste and I used a code generator instead in the end, that builds a code in logging and error repos using .go file like below as a reference:

var (
    // UserID ...
    UserID string
    // RegionID ...
    RegionID string
    // Time ...
    Time time.Time
)

[sammy-hughes]

@fkuehnel, code generation, as with "go generate" or home-rolled tools, is the "correct" way to accomplish these tasks, in Go. Another example would be Google's grpc generator tool, github.com/golang/protobuf/protoc-gen-go, which is capable of generating simple CRUD grpc services, based on the proto specs alone.

As stated above, this is the documented approach for such use-cases. It is satisfactory for both testability and correctness, if not ergonomics.

There are many things about Go which are not to anyone's taste in particular. It's unfortunate, but given that I'm one of the few Gophers in a Python shop that's been "transitioning" for longer than I've been there, I have a healthy appreciation for how deliberate the Golang dev team is.

[fkuehnel]

@fkuehnel, code generation, as with "go generate" or home-rolled tools, is the "correct" way to accomplish these tasks, in Go. Another example would be Google's grpc generator tool, github.com/golang/protobuf/protoc-gen-go, which is capable of generating simple CRUD grpc services, based on the proto specs alone.

As stated above, this is the documented approach for such use-cases. It is satisfactory for both testability and correctness, if not ergonomics.

There are many things about Go which are not to anyone's taste in particular. It's unfortunate, but given that I'm one of the few Gophers in a Python shop that's been "transitioning" for longer than I've been there, I have a healthy appreciation for how deliberate the Golang dev team is.

I really like this idea, keep the core language lean but 'go generate' can do the heavy lifting!

[ianlancetaylor]

Based on the discussion above, and the emoji voting, this is a likely decline.

Leaving open for four weeks for final comments.

[ianlancetaylor]

No further comments.