Tracking issue for refactoring the way we represent function call ABIs

[RalfJung]

This is the issue tracking implementation of rust-lang/compiler-team#672. Note that we do not have a final design yet; the best way to represent call ABI, and to disentangle it from the "storage kind" of a type (which is what the Abi type currently largely represents) is yet to be determined.

Note that for the purposes of this, by "call ABI" I mean "the target-independent information that is necessary and sufficient to compute how arguments and returns values are passed between caller and callee". The perfect end state (that we may or may not ever reach) for this would be to say that two types are ABI compatible if and only if the computed call ABI information for them is the same -- that would be very nice for the spec and for MiniRust, anyway.

I do not mean "the target-specific information saying which arguments are passed in which register / on the stack, which are copied and which are passed indirectly". This already exists to some extend as a concept in rustc, called PassMode. It may need some reforming, but that would be a separate discussion.

• The core need is that a Rust type must have its ABI computed in a manner that is not reimplemented for every target and every codegen backend. e.g. the traversal over types should skip repr(transparent) in essentially every case. Then every target architecture must be ported to it:
  • aarch64
  • arm
  • csky
  • loongarch
  • loongarch64
  • mips
  • mips64
  • powerpc
  • powerpc64
  • riscv32
  • riscv64
  • sparc
  • sparc64
  • x86
  • x86_64
• layout.abi is not about actual ABI, but only about IR codegen: handling the types as SSA values versus handling them as memory locations.
  • Rename it Rename rustc_abi::Abi to BackendRepr #132246
  • and decouple it from actual ABI handling

See in particular this comment.

Implementation history

• @workingjubilee: the first few PRs are just going to be moving stuff around so that we can actually co-locate as much of the ABI code as possible:
  • compiler: Stop reexporting enum-globs from rustc_target::abi #131424
  • compiler: {TyAnd,}Layout comes home #131473
  • Rename rustc_abi::Abi to BackendRepr #132246
  • compiler: Move rustc_target::spec::abi::Abi to rustc_abi::ExternAbi #132385

[bjorn3]

Should this be C-cleanup or C-tracking-issue? The description for the latter says that it is about RFC's and unstable features, which this is not. But on the other hand it is tracking implementation of an MCP.

[RalfJung]

I'd say both make sense. 🤷

Cc @eddyb

[RalfJung]

Looks like fixing #117480 may be blocked on this.