Adds stable layout types to pass to the runtime

[brooksprumo]

Problem

Please refer to #29852.

Summary of Changes

Add shadow structs with stable memory layouts to pass to the runtime.

Fixes #29852

Testing

In addition to CI, I built solana-validator with Rust nightly (cargo 1.69.0-nightly (e84a7928d 2023-01-31)) and ran it against mainnet-beta. It has caught up and has been running & making roots for 21+ hours.

Additionally, I've run cargo +nightly test against the following dirs:

• sdk/
• sdk/cargo-build-sbf/
• sdk/cargo-build-bpf/
• sdk/cargo-test-sbf/
• sdk/cargo-test-bpf/
• sdk/program/
• programs/bpf_loader/
• programs/bpf-loader_tests/
• program-runtime/
• program-test/

[alessandrod]

Solid PR so far! Just a couple of minor comments but otherwise looks great

[brooksprumo]

Here's all the unique types that are used with translate_type:

$ rg translate_type | sed -n 's/.*translate_type.*::<\(.*\)>.*/\1/p' | sort | uniq
&[u8]
AccountInfo
ProcessedSiblingInstruction
Pubkey
SolInstruction
StableInstruction
T
edwards::PodEdwardsPoint
i32
ristretto::PodRistrettoPoint
scalar::PodScalar
u64
u8

And removing the built-ins:

AccountInfo
ProcessedSiblingInstruction
Pubkey
SolInstruction
StableInstruction
edwards::PodEdwardsPoint
ristretto::PodRistrettoPoint
scalar::PodScalar

Their details:

• AccountInfo: safe, is repr(C)
• ProcessedSiblingInstruction: safe, is repr(C)
• Pubkey: safe, is repr(transparent)
• SolInstruction: safe, is repr(C)
• StableInstruction: safe, is repr(C)
• edwards::PodEdwardsPoint: safe, is repr(transparent)
• ristretto::PodRistrettoPoint: safe, is repr(transparent)
• scalar::PodScalar: safe, is repr(transparent)

Are there other types that I've missed?

[brooksprumo]

force-push to rebase onto the tip of master.

[alessandrod]

$ rg translate_type | sed -n 's/.translate_type.::<(.)>./\1/p' | sort | uniq
[snip]
T
[snip]

😱

Looks like that scary T is get_sysvar, so all sysvars need to be checked (I checked a couple and they're repr(C) so hopefully they all are!

[brooksprumo]

$ rg translate_type | sed -n 's/.translate_type.::<(.)>./\1/p' | sort | uniq
[snip]
T
[snip]

😱

Looks like that scary T is get_sysvar, so all sysvars need to be checked (I checked a couple and they're repr(C) so hopefully they all are!

Looking through the sysvars, I found:

• Clock: repr(C)
• EpochSchedule: repr(C)
• Fees: repr(C)
• Instructions: not repr(C), but it is an empty type. Is it a zero-sized type? Are those allowed in C? Is this type actually used anywhere? IOW, is this one OK?
• RecentBlockhashes: repr(C)
• Rent: repr(C)
• Rewards: repr(C)
• SlotHashes: repr(C)
• SlotHistory: repr(C)
• StakeHistory: repr(C)

I think we're OK, since bpf_loader only cares about Clock, EpochSchedule, Fees, and Rent. Those are the only sysvars that I saw that had Syscalls associated with them. which I think means those are the only ones that use translate_type()?

[alessandrod]

This looks good to me now, thanks for much for doing this work!

Up to you if you want to try to hack downstream-projects in CI to run with nightly, but if a validator running this makes roots, I think we're probably good to land!

[alessandrod]

I think we're OK, since bpf_loader only cares about Clock, EpochSchedule, Fees, and Rent. Those are the only sysvars that I saw that had Syscalls associated with them. which I think means those are the only ones that use translate_type()?

Yep the other sysvars use bincode (eew), so we're good

[joncinque]

The lack of this in 1.14 is causing issues for people who build their on-chain program with the compiler from 1.16 and the 1.14 crates. You can see my writeup at #31960 (comment)

Although we don't typically recommend that people do that, the other difficulties with upgrading to the 1.16 crates has meant that people first upgrade their toolchain, only to find that their current program doesn't work because of the Vec ABI changes in the new Rust compiler that's bundled with 1.16.

Would it be possible to backport this PR to 1.14? I know it's very late, but the changes don't seem too invasive, and it'll unblock a lot of developers.