repr(ordered_fields)

[RustyYato]

Add repr(ordered_fields) and provide a migration path to switch users from repr(C) to repr(ordered_fields), then change the meaning of repr(C) in the next edition.

This RFC is meant to be an MVP, and any extensions (for example, adding more reprs) are not in scope. This is done to make it as easy as possible to accept this RFC and make progress on the issue of repr(C) serving two opposing roles.

Rendered

To avoid endless bikeshedding, I'll make a poll if this RFC is accepted with all the potential names for the new repr. If you have a new name, I'll add it to the list of names in the unresolved questions section, and will include it in the poll.

[clarfonthey]

Not to add too many extra colours to the list, but repr(consistent) feels like a good name for this, since the purpose is to provide a consistent layout that does not depend on generics, compiler version, or target. The important thing is just that it's consistent, not that it matches what C does.

(Note: those three things should cover every case I've seen that uses repr(C) that should use repr(ordered_fields), but please feel free to correct me if I missed anything.)

Whereas repr(C) is explicitly, match what C does.

Also, while it may be more technical than most users need to understand, it would be helpful if the RFC reiterated the current issues with repr(C) that we want to fix, and potential future differences between repr(C) and repr(ordered_fields) that could pop up. I've read some of them but am not 100% sure of the details, and it would be nice to keep as part of the RFC.

[Lokathor]

Just as a small point of style the Guide Level Explanation is usually "what would be written in the rust tutorial book", and the Reference Level Explanation is "what could be written into the Rust Reference". This isn't a strict requirement, but personally I'd like to see the Reference Level part written out. Using the present tense, as if the RFC was accepted and implemented.

[programmerjake]

Suggested change

	`repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in memory layout.
	`repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in-memory layout.

"cross-platform" -- the layout will differ when there are different layouts for struct members' types, in particular primitive types can have different alignments which changes the amount of padding.

e.g., #[repr(ordered_fields)] struct S(u8, f64); doesn't have the same layout on x86_64 and i686

[RustyYato]

Good point, this will need to be documented as a hazard in the ordered_fields docs. However, the repr itself will be cross-platform. For example, #[repr(ordered_fields)] struct Cross([u8; 3], SomeEnum); will be truly cross-platform (given that SomeEnum is!).

[moonheart08]

Not to add too many extra colours to the list, but repr(consistent) feels like a good name for this, since the purpose is to provide a consistent layout that does not depend on generics, compiler version, or target. The important thing is just that it's consistent, not that it matches what C does.

(Note: those three things should cover every case I've seen that uses repr(C) that should use repr(ordered_fields), but please feel free to correct me if I missed anything.)

Whereas repr(C) is explicitly, match what C does.

Also, while it may be more technical than most users need to understand, it would be helpful if the RFC reiterated the current issues with repr(C) that we want to fix, and potential future differences between repr(C) and repr(ordered_fields) that could pop up. I've read some of them but am not 100% sure of the details, and it would be nice to keep as part of the RFC.

Just voicing support for repr(consistent) as naming.
Aside from the above, it more clearly hones in on the primary promises of the RFC, which is not just ordering but also exact type representation for things like enums. Field ordering is not the only thing it promises.

[joshtriplett]

Nominating this so that we can do a preliminary vibe-check on it in a lang triage meeting.

[scottmcm]

Big fan of doing this split, especially for structs. (It's less obvious what choices to make for other things, IMHO, but at least for structs this is something I've wanted for ages, so that for example Layout::extend can talk about it instead of C.)

Pondering the bikeshed: declaration_order or something could also be used to directly say what you're getting.

(This could be contrasted with other potential reprs that I wouldn't expect this RFC to add, but could consider as future work, like a deterministic_by_size_and_alignment where some restricted set of optimizations are allowed but you can be sure that usize and NonNull<String> can be mixed between different types while still getting the "same" field offsets, for example.)

[RustyYato]

I think this is also useful for unions, so we don't need to rely on repr(C) to ensure that all fields of a union are at offset 0.

This could be contrasted with other potential reprs that I wouldn't expect this RFC to add...

This also works as an argument against names like repr(consistent), since there are multiple consistent and useful repr, making it not descriptive enough.

[scottmcm]

I do think that declaration_order or ordered_fields is a bit weird on a union, because of course they're not really in any "order".

It makes me ponder whether we should just have repr(offset_zero) for unions to be explicit about it, or something.

(Which makes me think of other things like addressing rust-lang/unsafe-code-guidelines#494 by having a different constructs for "bag of maybeuninit stuff that overlap" vs "distinct options with an active-variant rule for enum-but-without-stored-discriminant". But those are definitely not this RFC.)

[RustyYato]

I don't mind spelling it as repr(offset_zero) for unions if that helps get this RFC accepted 😄. However, I have a sneaking suspicion that this isn't the contentious part of this RFC.
I know the name isn't optimal (intentionally). This can be hashed out after the RFC is accepted (or even give a different name for all of struct, union, and enum).
The most important bit for me is just that we do the split (for all of struct, union, and enum, to be consistent).

[RustyYato]

I've updated how enums's tags are specified, now they just defer to whatever repr(C)'s tag type is. This is done to reduce the friction of switching from repr(C) to repr(ordered_fields). To ensure that all uses of repr(ordered_fields) can be cross-platform, I've adding a lint to ensure that the user also adds an explicit repr for repr(uN)/repr(iN).

[Mark-Simulacrum]

I think this is too noisy. Most code out there using repr(C) is probably fine - IIUC, if you're not targeting Windows or AIX, maybe definitely fine? - and having a bunch of allow(...) across a bunch of projects seems unfortunate.

Maybe we can either (a) only enable the lint for migration, i.e., the next edition's cargo fix would add allows for you or (b) we find some new name... C2 for the existing repr(C) usage to avoid allows. But (b) also seems too noisy to me.

[clarfonthey]

Maybe it could just be an optional edition compatibility lint, so if someone enables e.g. rust_20xx_compatibility it shows up but otherwise not.

[RustyYato]

(a) only enable the lint for migration

That was the intention, hence the name edition_2024_repr_c. I'll make this more clear, that this is intended to be a migration lint.

Rustfix would update to #[repr(ordered_fields)] to preserve the current behavior. For the FFI crates, #![allow(edition_2024_repr_c)] at the top of lib.rs would suffice. If you have a mix of FFI and non-FFI uses of repr(C), then you'll have to do the work to figure out which is which, no matter what option is chosen to update repr(C) - even adding repr(C2), since then the FFI use case would need to update all their reprs to repr(C2).

Overall, I think this scheme only significantly burdens those who have a mix of FFI and non-FFI uses of repr(C). But they were going to be burdened no matter what option was chosen.

[RustyYato]

Is the new wording/lints too noisy still?

[joshtriplett]

I think the new wording is still too noisy. We shouldn't assume that most people using repr(C) are using it for ordering rather than FFI.

[RustyYato]

That wasn't my intention, but I don't see another way to do all of the following in the next edition:

• make repr(C) mean - same layout/ABI as what the standard C compiler does
• make repr(ordered_fields) - the same algorithm that's listed for repr(C) in the Rust reference
• ensure that everyone who upgrades to the next edition gets the layout they need (as long as they read the warnings and follow the given advice)
• make it as painless as possible for people who don't mix FFI and stable ordering cases (which I suspect is the vast majority of people). In other words, each crate currently uses repr(C) either exclusively for FFI or exclusively for some stable layout.
• for people who do mix FFI and stable ordering cases in one crate, at least the warning should give them all the places they need to double-check, and they can silence the warning on a case-by-case basis.

I'm open to suggestions on how to handle the diagnostics. Within these constraints, I think my solution is the only real option we have. If there are some objections to these constraints, I would like to hear those too, maybe I missed the mark with these constraints, and missed a potential solution because of it.

[RalfJung]

Suggested change

	Of course, making `SomeFFI` size 8 doesn't work for anyone using `repr(C)` for case 1. They want it to be size 0 (as it currently is).
	Of course, making `SomeFFI` size 8 doesn't work for anyone using `repr(C)` for case 1. They want it to be size 0 (as it currently is).
	Fixing the layout of this struct will also most likely let us remove a long-standing hack from our implementation of the MSVC ABI:
	unlike all other ABIs, we do *not* entirely skip ZST with that ABI but instead mark them to be passed by-ptr.
	This is needed to correctly pass types like `SomeFFI`.
	If we fix our layout computation to match that of MSVC, we no longer need this special case in the ABI logic.

[RalfJung]

Turns out this was not entirely right, see rust-lang/unsafe-code-guidelines#552 (comment). So probably best to remove this paragraph again.

[Jules-Bertholet]

That issue should presumably be addressed in this RFC, no? repr(C) 1-ZSTs need to be passed by pointer, but does that apply to repr(ordered_fields) as well?

[RalfJung]

The main problem is that we have already specified that all 1-ZST have the same ABI. We need to figure out how to clean that up but I don't think this RFC needs to do that.

[Jules-Bertholet]

This RFC currently specifies that new repr(C) matches C. That requires breaking the “all 1-ZST have the same ABI” rule. So the RFC either needs to specify that it’s actually “repr(C) matches C except for 1-ZSTs”, or it needs to specify that we are breaking the rule. Can’t have it both ways

[RustyYato]

I did see that you claimed this before in rust-lang/unsafe-code-guidelines#552. I'm not sure what this is referencing.

If this is referencing the current implementation where 1-ZSTs are explicitly skipped in the ABI. I'm not seeing any documentation guaranteeing this behavior, so it should be fine to change this.

If this is referencing this: https://internals.rust-lang.org/t/creating-1-zsts-guaranteed-to-have-same-extern-c-abi-as/19399/18
Where you said:

To make repr(transparent) work, we need all 1-ZST to have the same ABI, no matter their repr.

This seems rather subtle, and it should be explicitly documented somewhere (perhaps the Rust reference on type layouts/abi).

And if this is indeed your reasoning, then we could (in future editions) ensure that #[repr(C)] 1-ZSTs are only allowed in a #[repr(transparent)] type Foo if all other types in Foo are normal non-#[repr(C)] 1-ZSTs. (i.e. treat #[repr(C)] types as if they are not 1-ZSTs for #[repr(transparent)]).

[RalfJung]

Yes it is a subtle indirect consequence, as explained at the top of rust-lang/unsafe-code-guidelines#552:

Note that the reason for this rule of any two 1-ZST being ABI compatible is that for types like #[repr(transparent)] ZST([u8; 0]; ()), we promise that the type is ABI-compatible with the field that is being transparently wrapped, and that could be either field. So the only way to always deliver on our repr(transparent) ABI-compatibility promise is to make all 1-ZST ABI-compatible.

If ABI(ZST) == ABI( () ) and ABI(ZST) == ABI([u8; 0]) it follows that ABI( () ) == ABI([u8; 0]).

then we could (in future editions) ensure that #[repr(C)] 1-ZSTs are only allowed in a #[repr(transparent)] type Foo if all other types in Foo are normal non-#[repr(C)] 1-ZSTs. (i.e. treat #[repr(C)] types as if they are not 1-ZSTs for #[repr(transparent)]).

Yes we could. Though ABI is a cross-edition concern so it'd only help marginally to do this on new editions only.

But we could try to crater whether just doing a transition for this is realistic...

[Jules-Bertholet]

Though ABI is a cross-edition concern so it'd only help marginally to do this on new editions only.

I assume it would be based on the edition where the repr(C) 1-ZST is defined? I.e., same as everything else in this RFC. Note that, if you have:

// on new edition with new `repr(C)`
#[repr(C)]
struct NoFields {}

#[repr(transparent)]
struct Foo(u32, NoFields);

NoFields won’t be a ZST at all on windows-msvc, so it’s not really much more problematic to say Foo doesn’t compile on windows-gnu either.

[RalfJung]

I assume it would be based on the edition where the repr(C) 1-ZST is defined?

Usually it'd be based on the edition of the crate that the repr(transparent) type is defined in.
Otherwise, bumping the edition of the crate that has the empty C struct could lead to downstream build failures.

NoFields won’t be a ZST at all on windows-msvc

Yeah, also see rust-lang/unsafe-code-guidelines#552 (comment).

[Jules-Bertholet]

Bumping the edition of the crate that has the empty C struct could lead to downstream build failures.

Well, yes, the whole point of this RFC is that changing the meaning of repr(C) is a breaking change that must be done over an edition.

[Jules-Bertholet]

Is this something you intend to do as part of this RFC, or is it a future possibility? If the former, you should specify the exact behavior in the RFC. If the latter, it should be mentioned in the future possibilities section.

Also, what is the motivation for “(continuing to) disallow repr(ordered_fields, packed(N)) from containing aligned fields”? Should we not simply have the packed take precedence?

[RalfJung]

Yeah I think there's a pretty obvious semantics for repr(ordered_fields, packed(N)) with repr(aligned) fields -- treat those fields like all other fields, i.e., cap their alignment to N. Not sure if that's worth punting to a future possibility.

[RustyYato]

Is this something you intend to do as part of this RFC, or is it a future possibility? If the former, you should specify the exact behavior in the RFC. If the latter, it should be mentioned in the future possibilities.

This is a reference to another RFC, I'll make this more clear.

Also, what is the motivation for “(continuing to) disallow repr(ordered_fields, packed(N)) from containing aligned fields”? Should we not simply have the packed take precedence?

This is the status quo, so no motivation needed. You would need some motivation to switch to packed taking precedence.

[RustyYato]

Yeah I think there's a pretty obvious semantics for repr(ordered_fields, packed(N)) with repr(aligned) fields -- treat those fields like all other fields, i.e., cap their alignment to N. Not sure if that's worth punting to a future possibility.

I would rather not scope creep this RFC to one that's too large and controversial to accept. Given that there is already a whole other RFC for exactly this case, I don't think this is a minor addition.

So, I'm trying to keep this RFC as small as possible. So, since it is currently a hard error, I'm keeping it like that. But as a compromise, I will add it as a unresolved question, so we don't lose track of this. If it is not controversial and won't hamper accepting this RFC, I'm happy to add it in.

[RalfJung]

Let's see what the lang team says; they often like fixing some answer for those questions to make it more concrete what the RFC could turn into (and this one seems rather uncontroversial to me).

Note that we anyway have to define what the layout is for a repr(ordered_fields, packed) with a field of generic type, even if that generic type ends up instantiated with a repr(align) type. There's no way we can skip that question (except by forbidding generic types in repr(ordered_fields, packed), or by adding entirely new type system concepts to prevent this instantiation from happening -- neither of which is or should be suggested by the RFC). The current hard error is kind of a red herring, it doesn't actually reliably rule out repr(align) nested inside repr(packed).

[RalfJung]

What should repr(C) do when the corresponding C code is rejected by the default target C compiler?

This affects, for instance, fieldless structs, which are rejected by MSVC (but accepted by GCC). By extension it then also affects structs where all fields are PhantomData, as those should arguably be removed when translating a Rust type to a C type.

[Jules-Bertholet]

I think we should specify that repr(Rust) or repr(ordered_fields) 1-ZSTs (incl. tuples) are always removed, but otherwise if there is no direct C equivalent (e.g. repr(Rust) or repr(ordered_fields) fields, or fieldless struct if not supported by the C compiler), the layout should be consistent within the same compiler version but otherwise unspecified.

[RustyYato]

I've added this as an unresolved question, so we don't lose track of this.

I agree that we should remove any repr(Rust)/repr(ordered_fields) 1-ZSTs, but if we have a type that otherwise doesn't have an equivalent supported C type, maybe we should either error or try working with the C community to close this gap (i.e. fieldless types). Either by ensuring that they never make it compile in the future and giving us free rein to give semantics, or by making the code compile with some layout/ABI and then matching it ourselves (this would be preferred).

If the type has no equivalent (i.e. some repr(Rust)/repr(ordered_fields)), then either the layout is unspecified or we could treat it some opaque blob with the same size and align, and lay it out how a C compile would lay out an struct containing an array of the same size and align. (i.e. __attribute__((aligned(ALIGN))) struct Blob { char x[SIZE] };). This seems like it would be the most consistent.

We've been bitten before by making choices about what the C code ought to do and had to do some painful roll-backs (like this RFC, and env::set_var). So I think being more conservative around C code is probably a good thing.

[RalfJung]

or we could treat it some opaque blob with the same size and align, and lay it out how a C compile would lay out an struct containing an array of the same size and align. (i.e. attribute((aligned(ALIGN))) struct Blob { char x[SIZE] };)

I don't think we should promise any particular layout -- by leaving it unspecified, we at least de jure can change what we do to match what C does if it ever becomes legal in C.

[RalfJung]

The RFC does a good job at establishing that there are two potentially distinct use for today's repr(C). What the RFC doesn't do (and I think is important) is evaluate why it's actually a problem to conflate these two. Specifically curious what use cases break if we just "fix" reprC).

This post and its neighbors discuss some of that.