Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Future-proofing enums/structs with #[non_exhaustive] attribute #2008

Merged
merged 11 commits into from
Aug 26, 2017

Conversation

clarfonthey
Copy link
Contributor

@clarfonthey clarfonthey commented May 25, 2017

This is a post-1.0 version of #757, changed to use an attribute instead of a dedicated syntax. This would allow crates to more easily mark enums as "non-exhaustive," requiring downstream crates to add a wildcard arm to matches, so that adding new variants is not a breaking change.

Additionally, this syntax is extended to structs as well.

Rendered

Edit: Updated rendered link to rfcs repo version.

@scottmcm
Copy link
Member

scottmcm commented May 25, 2017

Another thing in favour of #[non_exhaustive] on structs: the non-pub field prevents construction, whereas the attribute could allow it through FRU. That would allow extensible builders, with struct Builder { a: b, c: d, ..Default::default() }. And it has the same ergonomic advantages as with enum, allowing in-crate matching on the exact set of fields without the bonus non_exhaustive: (), and not needing it on in-crate construction either.

@clarfonthey
Copy link
Contributor Author

clarfonthey commented May 25, 2017

Because you mentioned it: if there's enough support for it, I'd be more than happy to promote the struct/trait parts into the main RFC. I've mostly kept what's there small so that we can at least get the enum part in.

Part of the reason why I did that is also ensuring that we choose a name that would work for structs/traits if/when they get added too.

@aturon aturon added the T-lang Relevant to the language team, which will review and decide on the RFC. label May 25, 2017
@KodrAus
Copy link
Contributor

KodrAus commented May 26, 2017

I like it 👍

This RFC cements the current state of the world with respect to exhaustive matching: where a producer's enum/struct declares itself unexhaustively matchable, and a consumer must respect that with a catch-all pattern. That sounds reasonable to me, because it's the only way for a producer to be sure additions are non-breaking.

If I understand the RFC correctly, the #[non_exhaustive] attribute isn't adding hidden fields or anything. ie it's not making observable changes beyond the match semantics in the consumer, so I think an attribute is a fine way to do this.

@sfackler
Copy link
Member

I feel a bit uneasy about using an attribute to control this. I'd prefer .. I think. Attributes don't normally change the semantics of a type that significantly.

@dlight
Copy link

dlight commented May 26, 2017

I feel a bit uneasy about using an attribute to control this. I'd prefer .. I think. Attributes don't normally change the semantics of a type that significantly.

A tangential comment: #[async] is exactly an attribute that will change the semantics (of a function) significantly. So I think the naming scheme of attributes #[non_exhaustive] and #[async] (and the possibility of other attributes like this) should be discussed together, for consistency.

@seanmonstar
Copy link
Contributor

A tangential comment: #[async] is exactly an attribute that will change the semantics (of a function) significantly.

I'd kind of expect to eventually have keywords async and await, I must assume they are currently an attribute and macro is because that's the easiest to implement and explore. But probably not the place to have this discussion.

Copy link

@kevincox kevincox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent writeup. This sounds like a great change to me.

Should there be a way to warn downstream crate users when their match is
non-exuhaustive? What if a user wants to add a warning to their matches using
non-exhaustive enums, to indicate that more code should be added to handle a new
variant?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would argue that if you expect users to handle every case then this isn't for you. You should not use this attribute and bump your major version. A warning on not handling a particular case is also incredibly likely to be low-value noise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair! I mostly carried this over from #757 because it was discussed a lot in the RFC there.

## Extensions to structs

It makes sense to eventually allow this attribute on structs as well. Unlike
enums, it's very easy to make a struct non-exhaustive:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a good idea to me. However in this case non-exhaustive seems a less relevant keyword word because it is usually used to describe matching. incomplete, extensible or will_extend now start to sound better and work for both.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also note here that the term "extensible" was used on the previous RFC.


Tangentially, it also makes sense to have non-exhaustive traits as well, even
though they'd be non-exhaustive in a different way. Take this example from
[`byteorder`]:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This concept seems sufficiently different to deserve a different attribute.

Copy link
Member

@whitequark whitequark May 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say #[extensible] applies here just as well--as in extending the set of supertraits. In case we go for .. for enum and struct (which I think is a good idea), we could go with:

trait A: B + C + .. {}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. extensible does sound good. Not only for extending traits but also extending the provided methods. And when I do look at it that way it does seem fairly related.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that they seem sufficiently different, but I see them all as solving the same problem: instead of preventing a user from implementing/using something by creating a hidden field, this simply states that not all of the dependencies can be listed by the user.

@clarfonthey
Copy link
Contributor Author

clarfonthey commented May 26, 2017

I feel a bit uneasy about using an attribute to control this. I'd prefer .. I think. Attributes don't normally change the semantics of a type that significantly.

That is fair! I mostly changed it to an attribute because at the time of the original RFC (#757), it was decided that adding extra syntax was not worth the benefits, and someone suggested that an attribute might be better.

That said, a lot has happened since the initial 1.0 release, and I think that people are more willing to accept a syntax addition for this now than when the original RFC was written.

@kennytm
Copy link
Member

kennytm commented May 26, 2017

👍 extending this to structs, 👎 extending this to traits. The semantic change applied to traits is different enough that should take a different attribute or syntax (e.g. #[sealed]); I was expecting #[non_exhaustive] applied to traits means "this trait may gain additional methods in the future".

@kennytm
Copy link
Member

kennytm commented May 26, 2017

If this RFC is accepted and stabilized, there should be a (clippy?) lint that suggests any C-like enum returned from extern function (-> E or &mut E) be annotated #[non_exhaustive], to prevent rust-lang/rust#36927 ("Match on repr(C) enum returned from C library with unknown value leads to UB").

@clarfonthey clarfonthey changed the title Future-proofing enums with #[non_exhaustive] attribute Future-proofing enums/structs with #[non_exhaustive] attribute May 26, 2017
@leoyvens
Copy link

leoyvens commented May 28, 2017

Another thing in favour of #[non_exhaustive] on structs: the non-pub field prevents construction, whereas the attribute could allow it through FRU.

@scottmcm I don't see how the attribute would allow FRU, to the contrary it should forbid FRU outside the module that defines the struct. This should be in the RFC.

@strega-nil
Copy link

@kennytm that doesn't really make much sense. A match on a non-exhaustive enum outside the bounds would still be UB, and it's not uncommon for C enums to be treated like Rust enums - i.e., only within the bounds is okay.

@kennytm
Copy link
Member

kennytm commented May 28, 2017

@ubsan a match on a non-exhaustive C-like enum, not arbitrary enum. This particular case should be definable.

@clarfonthey
Copy link
Contributor Author

I'm pretty sure that casting an arbitrary integer to an enum, C-like or not, is undefined behaviour unless it corresponds to a valid variant. I don't think that annotating the enum differently is going to change this problem.

@scottmcm
Copy link
Member

@leodasvacas Oops, you're absolutely correct. Allowing FRU would need it to use the alternative desugar. ("jFransham's Super-FRUs", I remember it being called...)

@le-jzr
Copy link

le-jzr commented May 29, 2017

@clarcharr What makes it UB for C-like enums is that it fails the assumption that an exhaustive match will always execute one of its branches. If exhaustive match is impossible, missing labels pose no problem.

@le-jzr
Copy link

le-jzr commented May 29, 2017

Or to put it simply, putting #[non_exhaustive] on C-like enum could be a statement that every value of the underlying integer type is valid, even if those values don't have labels. Insofar as that statement applies to all code, even within the same crate, which of course is not what's being proposed in this RFC.

@clarfonthey
Copy link
Contributor Author

Note that the attribute does not say "all bit patterns of this enum are valid," only "more variants may be added in the future."

I don't think that this is a good solution to the problem you're suggesting.

@le-jzr
Copy link

le-jzr commented May 29, 2017

The attribute says what the RFC says it says, and RFC is not merged yet. ;)

But yes, it's somewhat different, even if intimately related. The solution to C enum problem requires #[non_exhaustive], but it's a somewhat stronger guarantee. In particular, match code generation needs to ensure that all unnamed values fall into the wildcard branch, which is not necessarily true right now (as far as I know).

@Ericson2314
Copy link
Contributor

I do want some "real" solution for this. I want to be able to machine-check semver comparability, and the "doc trick" will be ugly to implement for any tool which does that.

@Virtlink
Copy link

This solution would be welcome, as it's a non-breaking change for Rust 1.x. But I think it's the wrong way around.

Non-exhaustive should be the default. This allows a crate author to add new enum variants and add new public and private fields to a struct, without breaking user's code. Of course, this forces users to handle the case where enums are extended, prevents users from using the default constructor to construct a struct, and also prevent users from destructuring the struct.

If the author really wants to enable the user to construct and destruct structs and exhaustively match enums, they have to promise that they won't add members to the struct or enum by finalizing it through some keyword or attribute (or otherwise risk breaking user's code).

Maybe for Rust 2.0?

This pattern could again be solved by using `#[non_exhaustive]`:

```rust
#[non_exhaustive]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thinkno_extern_impl is a better name, because exhaustivenes feels weird for traits.

## Functional record updates

Functional record updates will operate exactly the same regardless of whether
structs are marked as non-exhaustive or not. For example, given this struct:
Copy link
Contributor

@petrochenkov petrochenkov Aug 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, the decision was the opposite - FRU is not permitted with non_exhaustive structs (in other crates).
FRU (with current rules) is a promise to not add private fields to a struct, and we want to be able to add private fields to non_exhaustive structs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that FRU still worked when structs have private fields?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No :(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm misunderstanding, I tried this out on the playground and it seems to work: https://play.rust-lang.org/?gist=eabe36303f9da0ced44c237ca3b29d45&version=stable

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

…right, I'll update the RFC.

@@ -417,6 +417,10 @@ Then we the only valid way of matching will be:
let Config { 0: width, 1: height, .. } = config;
```

We can think of this as lowering the visibility of the constructor to
`pub(crate)` if it is marked as `pub`, then applying the standard structure
rules.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to apply to unit structs as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do; one sec.


```
#[non_exhaustive]
pub struct Unit {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this seems to be equivalent to 3d56889#r135395315, but I'd still prefer to use the same wording for tuple and unit structs.

@petrochenkov
Copy link
Contributor

LGTM now 👍

@scottmcm
Copy link
Member

Huzzah! The @rust-lang/lang team has decided to accept this RFC.

To track further discussion, subscribe to the tracking issue here: rust-lang/rust#44109

Thanks for the RFC, @clarcharr!

@scottmcm scottmcm merged commit edbb821 into rust-lang:master Aug 26, 2017
@clarfonthey clarfonthey deleted the non_exhaustive branch August 27, 2017 03:13
bors added a commit to rust-lang/rust that referenced this pull request Nov 4, 2017
RFC 2008: Future-proofing enums/structs with #[non_exhaustive] attribute

This work-in-progress pull request contains my changes to implement [RFC 2008](rust-lang/rfcs#2008). The related tracking issue is #44109.

As of writing, enum-related functionality is not included and there are some issues related to tuple/unit structs. Enum related tests are currently ignored.

WIP PR requested by @nikomatsakis [in Gitter](https://gitter.im/rust-impl-period/WG-compiler-middle?at=59e90e6297cedeb0482ade3e).
@Centril Centril added A-data-types RFCs about data-types A-typesystem Type system related proposals & ideas A-privacy Privacy related proposals & ideas labels Nov 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-data-types RFCs about data-types A-privacy Privacy related proposals & ideas A-typesystem Type system related proposals & ideas final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. T-lang Relevant to the language team, which will review and decide on the RFC.
Projects
None yet
Development

Successfully merging this pull request may close these issues.