Distinguish repr(C) ZSTs from others in ABI compatibility rules

[Jules-Bertholet]

View all comments

(Split out from compiler implementation in #156112)

Some C ABIs pass and return ZSTs by pointer. But () should never be returned by pointer, as it must match void. To account for this, we have to weaken the present guarantee of "any two types with size 0 and alignment 1 are ABI-compatible" to exclude repr(C).

Fixes rust-lang/unsafe-code-guidelines#552; see also #78586, #155299.
Also related to #155984.

@rustbot label T-lang A-ABI needs-fcp

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: libs
• libs expanded to 11 candidates
• Random selection from 6 candidates

[Jules-Bertholet]

We exempt arrays for 2 reasons:

• A future version of C, or some other language we would like to do FFI with, might allow passing arrays to functions directly by value, in a way that would conflict with these guarantees
• It would be nice to also use "trivial ABI" in the specification of repr(C), and we need this clause for that. See discussion at repr(ordered_fields) rfcs#3845 (comment)

We could also simplify this clause by saying merely:

Suggested change

	/// - Not an array whose element type fails to satisfy these conditions
	/// - Not an array

The downside would be a larger breaking change.

Note that repr(transparent) will need to be adjusted to account for this change (by rejecting non-trivial arrays as "additional" fields).

View changes since the review

[RalfJung]

I'm fine with saying that "all arrays with trivial-ABI element type have trivial ABI".

However, even that less-breaking version breaks ghost again, right? Similar to dtolnay/ghost#41. ghost uses a zero-length array of *const T, which definitely does not have trivial ABI. I don't know why they do that...

EDIT: Ah, ghost is saved by having a repr(Rust) type around the array, under the rules discussed here.

[Mark-Simulacrum]

Probably mostly an opsem question in my eyes, at least to figure out the right shape. I see there's already been some discussion but I think nailing the language is probably better delegated too.

r? opsem

View changes since this review

[Mark-Simulacrum]

Some C ABIs pass and return ZSTs by pointer. But () should never be returned by pointer, as it must match void. To account for this, we have to weaken the present guarantee of "any two types with size 0 and alignment 1 are ABI-compatible" to exclude repr(C).

I think the implication here is that () is repr(C)? Am I reading that right? Where do we make that guarantee? Or is the thinking that the language here would make () and #[repr(C)] struct Foo; not ABI compatible?

One callout is that ZSTs aren't (I think?) standardized -- C and C++ without extensions both require types to be non-ZST if I remember right (e.g., see https://stackoverflow.com/a/2632075). Maybe that has changed since then though?

It seems like at minimum, it would be nice to avoid weakening this guarantee for Rust ABI even if we do so for C ABIs as a result of the weird platforms.

[Jules-Bertholet]

Or is the thinking that the language here would make () and #[repr(C)] struct Foo; not ABI compatible?

Yes, this.

One callout is that ZSTs aren't (I think?) standardized

Correct.

[RalfJung]

Suggested change

	/// - Any two types fulfilling all the following conditions are ABI-compatible;
	/// such types are said to have "trivial ABI":
	/// - Any two types with "trivial ABI" are ABI-compatible.
	/// A type has trivial ABI if is satisfies all of the following:

View changes since the review

[RalfJung]

I think I do agree with this modulo wording nits, though maybe I misread and my suggestion above (to require "all fields must have trivial ABI" rather than the oddly negated repr(transparent) condition) isn't just a wording change but a substantial change. In the following I am referring to my proposed ruleset. EDIT: Updated ruleset, or this equivalent formulation.

@rust-lang/opsem @rust-lang/lang what do you think? I think we do have to take back a bit of what we promised here, since we did promise too much. The current docs are plain wrong for extern "C" on Windows where the following types do not have the same ABI when used in return position:

// This is returned by-ptr.
#[repr(C)]
struct T1 {}

// This matches a C function returning `void`.
type T2 = ();

In argument position they do have the same ABI, but the way we achieve this is silly: we pass a pointer for () arguments.

Note that we also have to adjust the logic for repr(transparent) to ensure that all the types it ignores have "trivial ABI" according to these rules. So there's two breaking changes here:

1. Code that relied on the ABI compatibility we are taking away now has UB.
2. Code that relied on repr(transparent) ignoring a 1-ZST that we no longer consider to be "trivial" will stop compiling.

I cratered the 2nd part and found 0 regressions.

I don't know how to measure the fallout from the 1st breakage. We should implement this in Miri, but even that will just give us a very incomplete partial picture.

That said, given that the current docs are wrong, we have to do something. The only actually open question (in my eyes) is how bespoke we want the rules to be with the aim of breaking less code.

• We could have different ABI compatibility rules for the "Rust" ABI and the "C" ABI, and say that all 1-ZST have "trivial Rust ABI". This would theoretically reduce the amount of case 1 breakage (but we don't know anything about that breakage so it's unclear if there even exists code that would be helped by this). Currently our ABI compatibility notion is independent of the ABI used by the function, so this adds non-trivial extra complexity on the docs side. I don't think we should do this.
• The current rules say that all repr(Rust) 1-ZST have trivial ABI, without recursing further. This means that adding repr(transparent) to a type can make it lose its "trivial ABI" property! The way to intuitively understand this is that by adding repr(transparent) you are forcing the ABI to be compatible with at least one field, and if that field has an ABI that's determined by the C platform then we can't say that it has trivial ABI. The reason we do this is two-fold: this means more types have "trivial ABI", so we reduce the amount of type 1 breakage. And it means we can keep a pattern used by the ghost crate working (specifically, this allows us to consider a ghost-declared phantom type as being "trivial" for repr(transparent)).