Distinguish repr(C) ZSTs from others in ABI compatibility rules

[Jules-Bertholet]

View all comments

FCP: #157973 (comment)

(Split out from compiler implementation in #156112)

Some C ABIs pass and return ZSTs by pointer. But () should never be returned by pointer, as it must match void. To account for this, we have to weaken the present guarantee of "any two types with size 0 and alignment 1 are ABI-compatible" to exclude repr(C).

t-lang nomination summary comment

Fixes rust-lang/unsafe-code-guidelines#552; see also #78586, #155299.
Also related to #155984.

@rustbot label T-lang A-ABI needs-fcp

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: libs
• libs expanded to 11 candidates
• Random selection from 6 candidates

[Jules-Bertholet]

We exempt arrays for 2 reasons:

• A future version of C, or some other language we would like to do FFI with, might allow passing arrays to functions directly by value, in a way that would conflict with these guarantees
• It would be nice to also use "trivial ABI" in the specification of repr(C), and we need this clause for that. See discussion at repr(ordered_fields) rfcs#3845 (comment)

We could also simplify this clause by saying merely:

Suggested change

	/// - Not an array whose element type fails to satisfy these conditions
	/// - Not an array

The downside would be a larger breaking change.

Note that repr(transparent) will need to be adjusted to account for this change (by rejecting non-trivial arrays as "additional" fields).

View changes since the review

[RalfJung]

I'm fine with saying that "all arrays with trivial-ABI element type have trivial ABI".

However, even that less-breaking version breaks ghost again, right? Similar to dtolnay/ghost#41. ghost uses a zero-length array of *const T, which definitely does not have trivial ABI. I don't know why they do that...

EDIT: Ah, ghost is saved by having a repr(Rust) type around the array, under the rules discussed here.

[Mark-Simulacrum]

Probably mostly an opsem question in my eyes, at least to figure out the right shape. I see there's already been some discussion but I think nailing the language is probably better delegated too.

r? opsem

View changes since this review

[Mark-Simulacrum]

Some C ABIs pass and return ZSTs by pointer. But () should never be returned by pointer, as it must match void. To account for this, we have to weaken the present guarantee of "any two types with size 0 and alignment 1 are ABI-compatible" to exclude repr(C).

I think the implication here is that () is repr(C)? Am I reading that right? Where do we make that guarantee? Or is the thinking that the language here would make () and #[repr(C)] struct Foo; not ABI compatible?

One callout is that ZSTs aren't (I think?) standardized -- C and C++ without extensions both require types to be non-ZST if I remember right (e.g., see https://stackoverflow.com/a/2632075). Maybe that has changed since then though?

It seems like at minimum, it would be nice to avoid weakening this guarantee for Rust ABI even if we do so for C ABIs as a result of the weird platforms.

[Jules-Bertholet]

Or is the thinking that the language here would make () and #[repr(C)] struct Foo; not ABI compatible?

Yes, this.

One callout is that ZSTs aren't (I think?) standardized

Correct.

[RalfJung]

@rust-lang/opsem @rust-lang/lang what do you think? I think we do have to take back a bit of what we promised here, since we did promise too much. The current docs say:

• Any two types with size 0 and alignment 1 are ABI-compatible.

This is plain wrong for extern "C" on Windows where the following types do not have the same ABI when used in return position:

// This is returned by-ptr.
#[repr(C)]
struct T1 {}

// This matches a C function returning `void`.
type T2 = ();

T1 is passed and returned by-ptr by MSVC so we have to do the same. But () means void so it cannot be returned by-ptr.
In argument position they do have the same ABI, but the way we achieve this is silly: we pass a pointer for () arguments.

So the proposal is to restrict the above rule as follows:

• Any two types fulfilling all the following conditions are ABI-compatible; such types are said to have "trivial ABI":
  • It has size 0.
  • It has alignment 1.
  • One of the following apply:
    • It is a repr(Rust) (implicitly or explicitly, possibly with additional flags such as packed) struct, enum, union (regardless of its fields).
    • It is a tuple (regardless of its fields, and including ().
    • It is a repr(transparent) struct, enum, or union, and all fields have trivial ABI.
    • It is an array, and its element type has trivial ABI. (This requirement applies even to arrays of length 0.)
    • It is the never type !.
    • It is a function item type or closure type.

Under the old rules, we effectively said "All 1-ZST have trivial ABI". Now we have some further restrictions. I think the only types that used to have trivial ABI but don't any more are repr(C) 1-ZST and zero-length arrays of non-zero-sized types such as [u8; 0], as well as repr(transparent) wrappers around such types.

Note that we also have to adjust the logic for repr(transparent) to ensure that all the types it ignores have "trivial ABI" according to these rules. So there's two breaking changes here:

1. Code that relied on the ABI compatibility we are taking away now has UB.
2. Code that relied on repr(transparent) ignoring a 1-ZST that we no longer consider to be "trivial" will stop compiling.

I cratered the 2nd part and found 0 regressions.

I don't know how to measure the fallout from the 1st breakage. We should implement this in Miri, but even that will just give us a very incomplete partial picture.

That said, given that the current docs are wrong, we have to do something. The only actually open question (in my eyes) is how bespoke we want the rules to be with the aim of breaking less code.

• We could have different ABI compatibility rules for the "Rust" ABI and the "C" ABI, and say that all 1-ZST have "trivial Rust ABI". This would theoretically reduce the amount of case 1 breakage (but we don't know anything about that breakage so it's unclear if there even exists code that would be helped by this). Currently our ABI compatibility notion is independent of the ABI used by the function, so this adds non-trivial extra complexity on the docs side. I don't think we should do this.
• The rules in this PR say that all repr(Rust) 1-ZST have trivial ABI, without recursing further. This means that adding repr(transparent) to a type can make it lose its "trivial ABI" property! The way to intuitively understand this is that by adding repr(transparent) you are forcing the ABI to be compatible with at least one field, and if that field has an ABI that's determined by the C platform then we can't say that it has trivial ABI. The reason we do this is two-fold: this means more types have "trivial ABI", so we reduce the amount of type 1 breakage. And it means we can keep a pattern used by the ghost crate working (specifically, this allows us to consider a ghost-declared phantom type as being "trivial" for repr(transparent)).

[traviscross]

We talked about this briefly in the lang call last week. We had wanted to see more discussion, if possible, particularly from those on opsem, to inform the decision.

Looking at it now, it occurs to me that this would also benefit from a Reference PR, and that such a PR would help facilitate the lang discussion.

[RalfJung]

Does the Reference have any discussion of ABI compatibility currently? Hard to make a PR with a diff to something that's not even in the Reference yet.

[traviscross]

What do you think needs to be done in the Reference at this point?

I had in mind defining trivial ABI and documenting this new rule in the Reference.

Where the Reference doesn't fully cover something, we've in the past then not added new or changed language rules in that area to the Reference (because we don't expect people to fully document an area in order to document one change). I now think that's less good than simply accepting (and documenting) that the Reference is incomplete in that area and still adding the rule for the thing we're changing in the language. So that's what we've been asking for on other changes: documenting the one rule being added.

But, here, I'll take care of it. I'm working on adding all of the ABI rules to the Reference. I'll roll this up in that.

[Jules-Bertholet]

Note that once this merges, I'll propose a follow-up to change what we accept in repr(transparent). That will need a Reference PR, and I think it makes sense to do the move at that point.

[RalfJung]

Finally -- thanks @Jules-Bertholet and everyone else who was involved. :)
@bors r+ rollup

[rust-bors]

📌 Commit fa05b78 has been tentatively approved by RalfJung

It will be put into the queue for this repository once PR CI succeeds.