proposal: spec: tuples as sugar for structs

[jimmyfrasche]

Updates:

• only allow unpack when all fields are exported
• unpack skips unexported fields of structs from different packages
• include a vet rule
• explicitly include a change to Go 1 compat agreement so that unpack is treated the same as unkeyed struct literals

This proposal adds basic tuples to Go with one piece of sugar and two builtins.

The sugar is for struct(T0, T1, …, Tn) to be shorthand for a struct type with n fields where the ith field has type Ti and is named fmt.Sprintf("F%d", i). For example, struct(int, string) is a more compact way to write struct { F0 int; F1 string }.

This gives us a tuple type notation for and answers all questions about how they behave (as the desugared struct they are equivalent to). By naming the fields F0 and so on, this both provides accessors for the individual elements and states that all tuple fields are exported.

The variadic pack builtin returns an anonymous tuple, with the appropriate types and values from its arguments. In particular, by ordinary function call rules, this allows conversion of a function with multiple returns into a tuple. So pack(1, "x") is equivalent to struct(int, string){1, "x"} and, given func f() (int, error), the statement t := pack(f()) produces the same value for t as the below:

n, err := f()
t := struct(int, error){n, err}

The unpack builtin takes any struct value and returns all of fields in the order of their definition, skipping _ fields and unexported fields from a different package. (This has to be somewhat more generally defined as tuples aren't a separate concept in the language under this proposal.) This is always the inverse of pack. Example:

// in goroutine 1
c <- pack(cmd_repeat, n)

// in goroutine 2
cmd, payload := unpack(<-c)

The struct() sugar let's us write pairs, triples, and so on for values of mixed types without having to worry about names. The pack and unpack builtins make it easier to produce and consume these values.

No changes are needed to the public API of reflect or go/types to handle tuples as they're just structs, though helpers to determine if a given struct is "tuple-y" may be useful. go/ast would need a flag in StructType noting when a struct used the tuple syntax but as long as the implicit field names are explicitly added by the parser. The only place this would be needed is for converting an AST back to a string.

The only potential danger here is unpack. If it's used on a non-tuple struct type from a different package it would be a breaking change for that package to add an additional exported field. Go 1 compat should be updated to say that this is a acceptable just as it says that adding a field that breaks an unkeyed struct literal is acceptable. Additionally, a go vet check should be added that limits unpack to structs with exclusively "F0", "F1", …, "Fn" field names. This can be relaxed at a later time.

This is a polished version of an earlier comment of mine: #33080 (comment) In the years since I've written many one-off types that could have just been tuples and experimented with generics+code generation to fill in the gap. There have been multiple calls for tuples in random threads here and there and a few proposals:

• proposal: Go 2: add tuple type #61920
• Proposal: Go2: Tuple type #32941
• proposal: spec: allow conversion between return types and structs #33080 (not tuples per se but related)

[earthboundkid]

I'm not sure the struct() sugar is necessary, but the pack and unpack builtins are brilliant.

What if instead of struct(), we said type MyRecord record{ string, error } was sugar for type MyRecord struct{ F1 string; F2 error }? ch := make(chan record{ string, error }) reads better to me than ch := make(chan struct( string, error )).

[jimmyfrasche]

I'd be fine with that but honestly I prefer struct because

1. emphasizes that it is still just a struct
2. no new keywords

[jimmyfrasche]

If there is a new keyword it should probably be tuple as record generally means things more like structs

[DeedleFake]

The only danger here is unpack. If it's used on an exported non-tuple struct type from a different package it would be backwards incompatible for that package to add an additional exported field. I think the only way around that is to say don't do that and not consider that an incompatible change. It is safe to unpack a value of type that is meant to be a tuple regardless of its source as the length of a tuple is necessarily part of its API.

I think this makes sense. It's similar to the existing situation with struct literals initialized without field names, i.e. v := Example{2, "something"}. That's technically legal, but it's not safe in terms of backwards compatibility and should be used with care.

One question I have though is whether it would be valid to directly use pack() with the results of a function call. For example, something like

func F1() int { return 1 }
func F2() (int, string) { return 2, "something" }

func main() {
  v := pack(F1()) // Allowed?
  v = pack(F1()) // Still allowed? Same type?

  v2 := pack(F2()) // Allowed?
}

[jimmyfrasche]

Yes, unpack is basically the mirror image of unkeyed struct initialization.

@DeedleFake I copied your example and put the examples in inline:

func F1() int { return 1 }
func F2() (int, string) { return 2, "something" }

func main() {
  v := pack(F1()) // v is the same as if the call were pack(1) so: struct(int){1}
  v = pack(F1()) // this is still allowed and is the same type

  v2 := pack(F2()) // This is allowed by func call rules and is the same as pack(2, "something")
}

[earthboundkid]

One question I have though is whether it would be valid to directly use pack() with the results of a function call.

I think it should work like the current system where you can do f(a()) if the arguments of f match the return types of a. So you could do:

func Foo() (a string, b int, c error)
func Bar(a string, b int, c error)

packed := pack(Foo())
// ...
Bar(unpack(packed()))

In Python, you need *arg, **kwargs to "unpack" arguments, but since Go is strongly typed, that shouldn't be necessary, and it should just always do the right thing.

One question is if this should work (I think it should):

func Foo() []string
func Bar(...string)

packed := pack(Foo())
// ...
Bar(unpack(packed()))

A vararg is just a final slice, so unpack should be able to just transparently unpack into it.

I am less sure if this should work:

func Foo() (a, b, c string)
func Bar(...string)

packed := pack(Foo())
// ...
Bar(unpack(packed()))

I think probably not, but it's a harder call.

[jimmyfrasche]

@carlmjohnson You are correct about the return type matching (though you have an extra () in your example code).

Neither of those last two would work, though: changes to varargs would need to be a separate proposal.

[earthboundkid]

This works now, so I guess the unpack version of it should too:

func Foo() (a, b, c string)
func Bar(...string)

Bar(Foo())

Varadics are different issue and I guess can be handled separately.

[jimmyfrasche]

Interesting. I could have sworn that was an error! Yes that would work.

And I also misread your first example:

func Foo() []string
func Bar(...string)

Bar(unpack(pack(Foo())))

This will not work. It's the same as Bar(Foo()) which needs to be Bar(Foo()...) so you'd need Bar(unpack(pack(Foo()))...) which works. It's a little confusing because the tuple is type struct([]string) so unpack only has a single return which is a slice

[jaloren]

In some languages with tuples, a tuple is a type that can be passed to a function and thus part of a function signature. In this design, that wouldn't be supported is that right?

I think one common use will be dealing with configuration functions that take more than 4 arguments. Currently, my rule of thumb is that any function that takes more than 4 arguments should take a config struct instead. With tuples, I could instead see APIs designed and used like this:

// package widget
func Setup(argOne int, argTwo string, argThree bool, argFour float64, argFive bool) error {
 // impl here
}
---
// main package

import(
  "widget"
)
func main(){
input := pack(argOne,argTwo,argThree,argFour,argFive)
if err := widget.Setup(unpack(input)); err != nil {
 panic(err)
}
}

I am unsure if that's a good idea or not but I think the temptation will be high because its the path of least resistance. Defining a struct is going to be much heavier by comparison and who wants to do that just to handle one function. If we do think that's a good idea, then being apple to pass the tuple as a type would be nice.

[thediveo]

How does struct(T0, T1) work across packages when there is no exported type Foo struct(T0, T1) that gets used? Is this out of scope on purpose? Pardon me if I overlooked the corresponding passage in the draft above.

[earthboundkid]

How does struct(T0, T1) work across packages when there is no exported type Foo struct(T0, T1) that gets used? Is this out of scope on purpose? Pardon me if I overlooked the corresponding passage in the draft above.

IIUC, it would follow the usual rules about type T struct{ /**/ } vs. type T = struct{ /**/ }, where you can pass unnamed struct types between packages without needing a central definition. I take it back, that won't work because each anonymous struct type is considered different.

[jimmyfrasche]

@thediveo & @carlmjohnson

https://go.dev/ref/spec#Type_identity

Two struct types are identical if they have the same sequence of fields, and if corresponding fields have the same names, and identical types, and identical tags. Non-exported field names from different packages are always different.

By construction there are no tags and the field names are automated so as long as they have the same number of types in the same order they're the same

proof!: https://go.dev/play/p/Ek8vIDKndw-

this also works if one (but not both) of the packages define the type: https://go.dev/play/p/pUGiqhUxzQB

[earthboundkid]

Okay. I thought it would work, but then I confused myself. This works because all the fields are public, but this does not because the fields are private.

[jimmyfrasche]

@jaloren

In some languages with tuples, a tuple is a type that can be passed to a function and thus part of a function signature. In this design, that wouldn't be supported is that right?

You could include a tuple in a function signature like:

func F(n int, tup struct(Point, Color))

It's an ordinary type (specifically a struct!) so you can use it however or whenever you would any other type.

I think one common use will be dealing with configuration functions that take more than 4 arguments.

I do not think that will be common at all. As your own example shows it buys nothing over not doing it other than having to include an extra pack and unpack.

Using a defined struct is immensely superior for the use case specifically because you can name the fields and easily omit irrelevant ones by using a keyed literal.

The major use case for tuples is having some types that you need to bundle together but there's no real need for anything other than that. If you've ever written a type with no methods like

type locationWeightPair {
  L location
  W weight
}

just so you could use it as a map key or throw it down a channel, you could just use struct(location, weight) under this proposal.

[DeedleFake]

I really like this proposal. The more I think about it the nicer it seems. I'm trying to figure out if it could help with two-value iterator problem over in #61405, and I think it probably could, but only partially. You could do something like

// t is a struct(string, error)
for t := range tupleSeq {
  v, err := unpack(t) // Needs an extra line, but nicer than needing to manually unpack a whole struct.
  // ...
}

The ability of unpack() to work with non-tuple struct types as well should make the above more straightforward for some cases, too, but that backwards compatibility complication might cause a few problems there.

[jimmyfrasche]

It would only help with iterators if range auto-expanded tuples, which is what, for example, Python does. Since tuples aren't a separate kind of type in this proposal and there doesn't seem to be any interest in raising the number of loop variables past two so I don't see that happening under this or any other proposal, realistically.

You could write generic functions to convert a 2-iter into a pair and vice versa. That may be useful. The xiter funcs that are supersets of what is commonly called zip and zipLongest define types that are basically tuples.

The ability of unpack() to work with non-tuple struct types as well should make the above more straightforward for some cases, too, but that backwards compatibility complication might cause a few problems there.

That's more necessary than it is useful. I think it would make sense and be fine for some types like image.Point which are essentially just a pair with benefits. For others, to reiterate, it's just the same problem with using an unkeyed literal but in reverse. (If this were on paper I'd triple underline that bit.)

[jimmyfrasche]

Go already has tuples in the special case where the type happens to be the same for all the items: arrays. For example, [2]int and struct(int, int) are both capable of containing the same amount of information as the other.

Given that, I think it would make sense to expand unpack to work for arrays as well. It's not fundamental to the proposal and it could be added later, so I'm not going to include it for now, but something to consider.

[leaxoy]

I think there is no need add new keyword or functions, just use () to pack tuple in the right hand side and unpack in the left hand side.

For example:

a := 100
b := "string"
c := []int{1,2,3}

// pack
t := (a, b, c)

// and unpack

(x, y, z) := t
// or () can be omitted to
x, y, z := t // same as current syntax, no need introduce extra complexity
// so x is 100, y is "string" and z is []int{1,2,3}

[jimmyfrasche]

@leaxoy There are no new keywords in this proposal: just use of one keyword in a new context and two new predeclared identifiers. I don't think just using () can be made to work either syntactically or philosophically. Perhaps I am wrong but I'm not especially interested in that possibility myself as I like the explicit pack and unpack and think they fit with the language better.

[septemhill]

Do we allow tuple as a field in a struct ?

type TupleInside struct{
   FieldOne string
   FieldTwo int
   FieldThree struct(int, string, float64)
}

If we do, how do we marshal/unmarshal the tuple case ?

[earthboundkid]

Do we allow tuple as a field in a struct ?

I think yes, because it’s just a struct with some sugar for the declaration.

If we do, how do we marshal/unmarshal the tuple case ?

When you unpack a struct containing a tuple, the target variable would be a struct of the appropriate type. Just like if you have a, b, c := f() and c is a struct.

[DeedleFake]

@septemhill

The code in your comment is 100% equivalent to

type TupleInside struct {
  FieldOne String
  FieldTwo int
  FieldThree struct {
    F0 int
    F1 string
    F2 float64
  }
}

It's just syntax sugar.

[septemhill]

@DeedleFake

Sure, I understand it's just syntax sugar.

For example, if we want to marshal the TupleInside to json

type TupleInside struct {
   FieldOne string                         `json:"field_one"`
   FieldTwo int                            `json:"field_two"
   FieldThree struct(int, string, float64) `json:"field_three"`
}

After de-sugar and marshalling, we would get json as following:

{
  "field_one": "field_one",
  "field_two": 123,
  "field_three": {
    "F0": 234,
    "F1": "f1",
    "F2": 345.345
  }
}

So, that means we cannot customize the tag name for each field in struct(int, string, float64)?
It would always be F0, F1 and F2.

Please correct me if I got something wrong, thanks.

[jimmyfrasche]

That is correct.

[apparentlymart]

Yes, it is true that struct tags are not a part of this proposal, and so I expect most folks will want to avoid using tuple-like structs in types intended for reflection-based marshalling.

I don't see that as a significant problem, though. Not all types are useful in JSON serialisation, and that's okay. If you are doing JSON serialisation then you will choose your types with that in mind.

[DeedleFake]

Random thought: Make the field names something visible, but awful and non-idiomatic, like Tuple_Field_0, etc., and then add a go vet warning with a comment about pack() and unpack() for direct accesses of fields with that name pattern.

[jimmyfrasche]

Arguably Fn already satisfies that.

I don't think it would be unreasonable to access the fields, for example, to sort a slice of tuples so I think effectively outlawing it overshoots somewhat.

[DeedleFake]

Hmmm... What if structs allowed numeric field names, without syntactic support for it? You wouldn't be able to define a numeric field name in a normal struct, but when defining a tuple the resulting struct type definition would get numeric fields instead of named ones. Then, allow structs to be indexed into, i.e. v[1], but require the index to be a constant integer. Internally it would still be a struct. reflect would also see it as a struct, and reflect could be allowed to access struct fields with numeric names normally like any other struct field. pack() and unpack() could then be made to only work with these special numeric fields, removing the question of how to use unpack() with unexported fields from other packages.

In other words,

type Tuple struct(int, string)

would effectively be equivalent to

type Tuple struct {
  0 int
  1 string
}

except that the second would be syntactically illegal because you can't start an identifier with a number. pack() would work as proposed originally, but unpack() would, conceptually, start unpack all of the fields starting with the one named 0 and incrementing until it runs out of fields, ignoring all others. It shouldn't be possible for there to be others, but it would give it a clean rule that way. reflect deals with struct field names as strings anyways, so it could just be allowed to deal with them manually through reflect like they were any other exported field.

v[N] would be syntactic sugar for v.N. N would have to be a constant, but unlike a normal field access it would be allowed to be a named constant. Other than that it would behave exactly the same in all respects as v.N, complete with compile-time checks for field existence.

[jimmyfrasche]

I could imagine tools and libraries doing the wrong thing by failing to special case numeric field names but not well enough to come up with a specific example.

In that vein I idly wonder if it would be possible to decree that the fields follow an unexported scheme like f%d BUT that anything all uses of the struct() syntax are treated as if they're declared in the same package so that their identity isn't bound to their origin. I don't think that's a great idea as the simpler solution of the F%d scheme being much simpler than alternatives.

[dsnet]

I could imagine tools and libraries doing the wrong thing by failing to special case numeric field names but not well enough to come up with a specific example.

I suspect some reflection libraries may only look at exported fields by checking whether the first rune of a name is unicode.IsUpper since the Go spec calls out uppercase identifiers as exported.
There is now a reflect.StructField.IsExported method, but that was added more recently (Go 1.17).
(I'm assuming fields of a tuple should be exported, but that's debatable.)

[jimmyfrasche]

One argument for numeric fields is that they could be unexported but made exported later. Though yes, I'd prefer they be exported for the rare case where you do want to just take a quick peek in the crate without having to take the whole thing apart.

The argument against doesn't just include reflect but go/ast and co. There is now an assumption that the fields are all legal identifiers so violating that could cause some fun problems.

[griesemer]

A numeric field would still be stored as a string. Only go/parser would have to be adjusted if one were to allow actually writing such fields. If t is a tuple, the 1st field could be accessed as t.1. Again, only the parser has to be adjusted.

[jimmyfrasche]

Only the parser has to be adjusted for the standard library to work. My concern, perhaps unfounded, is tools compiled against the new version whose logic hasn't been updated and doing something wrong because it expects the field name to be a valid Go identifier or as @dsnet points out not realizing that "2" is exported because it has an ad hoc check instead of using one of the various IsExported predicates. If that's the only change that needs to be made to such code it's probably not that bad.

[jimmyfrasche]

Since the standard selector syntax code generation accessing fields would work if the name is left unaltered. There could be issues if it assumes that it can write out the field name in a new struct definition or create a variable using the field name as a prefix. Other than that the only bugs I can think of it causing are incorrectly categorizing it as exported.

[jimmyfrasche]

Assuming numeric field names, presumably given

type tup struct(int, int)
var s struct {
  tup
 a, b int
}

both s.1 and s.a are legal

[dsnet]

Would s.0777, s.0b10101, s.0xbad_face, or s.5_543 be legal? All of them are integer literals per the spec, but I'd be horrified to ever see that in source code. If no, it unfortunately means we need to declare a subset of the integer literal grammar for this.

[jimmyfrasche]

it's a natural grammar

[griesemer]

@dsnet I'd just allow decimal integers. A leading zero is only permitted if the field name is 0. So s.0, s.1, s.42 but certainly not s.0o8 etc. These are names, not numbers, and we'd get to decide the syntax. The point would be to distinguish them from regular identifiers.

[jimmyfrasche]

There are two proposals for field access in generics:

1. (implicit) any field shared by the type set is allowed
2. (explicit) require a new kind of F T constraint meaning types may only be in the type set if they have a field F of type T

The F%d scheme works with either. The t.1 syntax works with the implicit but not the explicit proposal unless you are allowed to declare numeric fields in an interface but not a struct.

[jimmyfrasche]

Assuming numeric field names, presumably given

type tup struct(int, int)
var s struct {
  tup
 a, b int
}

both s.1 and s.a are legal

Would unpack(s) be legal and return the same thing as unpack(s.tup)?

[jimmyfrasche]

Library changes for numeric struct fields:

add a predicate to go/token to check if a string matches ^(0|[1-9][0-9]*)$

change token.IsExported to check if the first rune is uppercase or the above predicate is satisfied (ast.IsExported just calls token.IsExported)

in go/ast, note that Ident may contain a numeric string when used in a Field in a FieldList used in a StructType parsed from a tuple or in a SelectorExpr.Sel; possibly add an IsTuple method to StructType

Make a note in go/types.Tuple that there are tuples now but these are unrelated and change the unexported isExported predicate to match the token version; possibly add an IsTuple method to Struct

go/parser,printer,format do not need any visible changes

The reflect IsExported predicates do not need to be changed and would work as-is. Possibly add an IsTuple predicate and Pack/Unpack helpers.

Pros:

No "accidental tuples" so encoding/json could always output a tuple struct as a list without having to worry about backwards compat.

Safe to define unpack to be tuple-only without any contortions.

Undeniably stylish.

Cons:

Possible to cause issues in tools that use go/ast and libraries that use reflect that do their own checks for exportedness or require that field names are identifiers.

More complicated.

Conclusion:

I am starting to lean toward this. It's costlier than the simple F%d scheme but it confers benefit in kind. It may require changes throughout the ecosystem but they'd be extremely niche and most likely very simple to change and could be changed in a way that continues to work with older versions of Go.

Good idea, @DeedleFake!

[dsnet]

^[1-9][0-9]*$ implies that we are 1-indexed instead of 0-indexed, right?
Personally, it feels like tuple fields should be 0-indexed to be consistent with how slices and arrays work.

[jimmyfrasche]

oops, yes: corrected to: ^(0|[1-9][0-9]*)$

[dsnet]

possibly add an IsTuple method to Struct

If tuple T is embedded in struct S, what does reflect.TypeFor[S]().IsTuple report?
I assume the answer is false? It does feel a bit odd that fields are forwarded to the parent, but not the tuple-ness of the child. Something about this just feels off.

[jimmyfrasche]

Why? Embedding isn't inheritance so there's no IS-A relationship. Metonymy aside, I wouldn't say that my car is an engine though it contains one last I checked.

[jimmyfrasche]

A more interesting embedding case is two tuples of different length:

type T1 struct(int)
type T2 struct(int, int)
var s struct { T1; T2; f int }

There's an s.1 but no s.0 because that selector is ambiguous.

If unpack is defined to return only numeric fields unpack(s) would only return s.1.

[warning: this section is not something I'm recommending, just saying this as part of an argument] There's nothing technically blocking allowing users to add numeric fields to struct. A major reason to use them over an identifier scheme is that it avoids incorrectly opting in existing structs. If user defined numeric fields are added at the same time as or later than tuples, there is no such concern. That would allow

var n = struct {
  4 int
  2 int
}{-1, 1}

If unpack is defined to return only numeric fields unpack(n) could be defined to return 1, -1 or -1, 1.

Given those two things it seems important to

• not allow user defined numeric fields
• define unpack to only work directly on a tuple

[jimmyfrasche]

#64613 proposes s... for what is essentially unpack(s) in this proposal. It also extends composite literal rules to allow S{f()} which covers pack(f()) though it requires having a struct type to pack into. This generalizes some of the mechanism here but does not cover all of them. Notably it doesn't allow working with unspecified fields. It is simple to adapt this proposal:

1. just use ... instead of unpack
2. keep struct(T0, T1, ..., Tn) for tuple types
3. either keep pack without unpack or use struct(v0, v1, ..., vN) for constructing tuple values

[DeedleFake]

I generally agree, but I think that I prefer unpack() over .... It reads much nicer in a few places and removes all question of operator precedence with, for example, something like <-c....

[jimmyfrasche]

#66651 proposes variadic generics that would allow this proposal to be written as ordinary code: (along with generic type aliases which are coming soon, see: #46477)

package tuple

type Of[T... any] = struct { F T }

func Pack[T... any](v T) Of[T] {
  return Of[T]{v}
}

func Unpack[T... any](t Of[T]) T {
  return t.F
}