proposal: Go 2: explicitly nillable / non-nillable types

[themeeman]

Background

In Go, we are locked into some types having a nil representation and some not. The general rule is the "pointery" types have nil values, but even this is somewhat self-contradictory at first glance with the string type; this could be seen as a kind of pointer, yet cannot be nil.

This proposal is to give the developer more control over what types are nillable and what aren't, without adding much complexity to the simple type system.

Motivation

Explicitly nillable

A common use case of nilable types is databases. For example, we may have a string type in our DB that could be null, and want to express that in Go. There are two common solutions for this:

1) Use a struct

type NullString struct {
  	Value string
  	Valid bool
}

This allows us to distinguish between "nothing" and "empty" through the boolean, although falls down as it has limited static checking; anyone could set the Valid field to false, even when the Value field is non-empty. It also creates clutter in our codebase, as it needs to be rewritten for each type of Null field in our DB.

2) Use a pointer

The pointer type gives us a nil value like we want, and also comes with the static checking we want. Seems good so far. However, the pointer type also comes with all of the pointer semantics. This means we don't show our intent clearly; we could be showing a string that is passed around and modified, which we don't necessarily want. It feels like a hack.

Explicitly non-nillable

Sometimes, we never want certain nillable types to be nil. The most common, in my opinion, is maps. Sometimes we want it to always just contain data or be empty, we don't want it to actually "be" nothing.

The Proposal

I am proposing adding two new type qualifiers: # for non-nillable, and ? for nillable. These will be prefix qualifiers, similar to pointers.
The following can be ? qualified and made nillable

• int types
• float types
• complex types
• string
• struct types
• array types

Any type that cannot be ? qualified can be # qualified. These are:

• pointer types
• map types
• slice types
• channel types
• function types
• interface types

They can be nested in compound types, so they don't have to be only at the top level. However, multiple qualifiers of this kind can't be on the same level, as this would create strange corner cases, so is not worth allowing.

Examples:

var a ?int // nillable int type
var b ?float32 // nillable float32 type
var c ?[3]int // nillable array of 3 ints
var d ?struct { 
	name ?string
	age int
  	days #map[string]?time.Time // non-nillable map of strings to nillable timestamps
} // nillable struct
var e ???int // not allowed more than one qualifier
var f ?#?#?#?#?#float64 // no way
var g * #* #* #* #*?int64 // OK, pointer to non-nillable pointer to non-nillable pointer to non-nillable pointer to non-nillable pointer to nillable int64

Semantics

Nillable

Nillable types won't be far off what we know today - they can be thought of as changing the zero value of whatever it may be to nil.

Attempting to perform numeric operations on a numeric type that is nil will result in a runtime panic.
Example:

var h ?int = 5
var i ?int = 3
...
fmt.Println(a + b) // Danger! a or b could be nil

if a != nil && b != nil {
	fmt.Println(a + b) // Safe
}

This checking may seem tedious, however that is more to do with the language itself and out of the scope of this proposal.

This propagates to any operation on a nil type, except method calling. In keeping with Go's traditions, calling a method on a nil type is not an error.

Non-Nillable

The idea of making types non-nillable is more nuanced than nillable. We are doing something that is new to Go entirely; we are removing the zero value.

As a result of this, # qualified types can't be initialised implicitly, and must be explicitly. This also means that a struct containing a # qualified type can't be initialised implicitly either.

Expressed formally, a composite type cannot be implicitly initialised if any of its component types have no zero value.

This is a hefty restriction on use of non-nillable types. However, I believe it is justified, because the guarantees we get for it are worth the cost (imo).
Examples:

var j #*int // Compile time error
var k #*int = #new(int) // OK, see Conversions below
var l #map[string]string // Compile time error
var m #map[string]string = {} // OK
type Person struct {
	Name string
	Pets #[]string
}
var n Person // Compile time error, non nillable Pets field implcitily initialised
var o Person = {} // Compile time error
var p Person = {Pets: []string{"Millie", "Clara"}} // OK
var q #struct{} = {} // OK

Conversions

In keeping with Go, there will not be any implicit conversions between types.

#T -> T and T -> ?T

These conversions are lossless, so they will always succeed (as they are adding new information). Thus, they can be done at compile time using the normal conversion operator.

T -> #T and ?T -> T

These are lossy, so there needs to be runtime check that fails if the type is nil. For this, I propose two new operators: ? and #.
? Casts away the nilness of a value, and panics if the value is nil.
# Casts a value into non-nillable, and panics if the value is nil

Both of these will have "safe" variants using the , ok idiom.

Examples:

r := (?int)(5) // r is type ?int, with value 5
s := (?int)(nil) // s is type ?int, with value nil
t := #make([]int, 5) // t is a non nillable []int. Uses the "unsafe" version, but will ever panic
u := ?s // Panics because s is nil
u, ok := ?s // u is type int, contains 0. ok is false.
r, ok := #make(chan int) // redundant ok, will always be true

Alternatives

There are some alternative stuff to consider that I have left to the end for clarity.

Choice of prefix, not postfix

We could use postfix qualifiers instead of prefix, like most other languages. (e.g. int? instead of ?int)
However, I chose not to opt for the prefix because

1. Its more in keeping with the Go style and
2. More importantly, postfix creates a parsing ambiguity (is [3]int? a nillable array of int or an array of nillable int)

Use of # instead of !

One might expect the use of ! like other languages. However, it creates an ambiguity with the logical not operator, which is unneccessary.

Choice to remove zero values

An alternative approach to non-nillable types would be to create a new zero values to replace nil. Slice would have the empty slice and map would have the empty map.
However, for pointer values this would not work without allowing dangling pointers.

Therefore, for this approach to work, we would have to disallow non nillable pointers, which are a major use case for this proposal.

Conversion operators

The conversion operators are an interesting idea that I am not 100% confident in, although it seems to me to be the simplest and cleanest approach to it. An alternative would be some built in function.

Conclusion

I believe this proposal will improve the usability and expressivity of the language, outweighing the potential complexity increase.

[ianlancetaylor]

See also #28133 and #22729.

[networkimprov]

Have you considered an interface, which enables arbitrary implementations?

type Nilish interface {
   isNil() bool
}

Re never-zero types, I plan to propose default initializers like this: #28366 (comment)

[odiferousmint]

I don't like the idea of adding more symbols to the syntax. It is one of the problems I have with Rust. I don't want to look at Go code and not know what it is intended to do because of all the symbols. :/

[ianlancetaylor]

The fact that nil is overloaded already leads people into confusion (https://golang.org/doc/faq#nil_error). This proposal would seem to make that problem worse.

Although I read your motivation section, it's not clear to me that this solves a real problem. We can already express these ideas in the language in a more verbose fashion.

While there may be something to be done in this area, this particular proposal, which would sprinkle ? and # throughout Go programs, doesn't feel like the right approach for Go. It seems too invasive for the current look of the language.

It's also worth noting that ?int will be more expensive than int, and that may catch people by surprise.

We may be able to approach some of these ideas through vet checks, with some way for a function to declare that some argument must not be nil.