v1.14: Adds stable layout types

[joncinque]

Problem

Some on-chain program devs use the new toolchain because they need to use a newer Rust compiler for their dependencies, but can't upgrade to the 1.16 crates due to breaking changes.

When they try to test their program however, they typically get Invoked an instruction with data that is too large because of the ABI changes for Vec in Rust 1.67.

There's a more complete writeup at #31960 (comment)

Summary of Changes

Backport #30124 and #30192 to add the stable types. The issue only manifests when performing a CPI, so I tested this by running the transfer-tokens test https://github.com/solana-labs/solana-program-library/tree/master/examples/rust/transfer-tokens with patched crates and the Solana 1.16.3 toolchain.

I know it's late in 1.14's lifespan for backports, but this would unblock a lot of downstream developers. Please let me know if you have any specific concerns, or perhaps ways to make this change less dangerous.

Since it's just guaranteeing the layout already present in older Rust versions, I'm thinking that it should be pretty safe.

Fixes #31960

[brooksprumo]

Seems scary 😬

Some on-chain program devs use the new toolchain because they need to use a newer Rust compiler for their dependencies, but can't upgrade to the 1.16 crates due to breaking changes.

Is is possible to use older dependencies that use v1.14?

[brooksprumo]

Or alternatively, can they upgrade their whole toolchain/dependencies to v1.16?

[joncinque]

Is is possible to use older dependencies that use v1.14?

Yes, but they'll have to find all the right versions that support Rust 1.60, which might not always be possible.

Or alternatively, can they upgrade their whole toolchain/dependencies to v1.16?

That's the ideal solution, but because of a chain of breakages, a lot of people can't upgrade to the 1.16 crates right away.

[brooksprumo]

Is is possible to use older dependencies that use v1.14?

Yes, but they'll have to find all the right versions that support Rust 1.60, which might not always be possible.

😢

Or alternatively, can they upgrade their whole toolchain/dependencies to v1.16?

That's the ideal solution, but because of a chain of breakages, a lot of people can't upgrade to the 1.16 crates right away.

😢

[joncinque]

Let me try a test that reverts the changes in the runtime, but keeps all the sdk / program-test changes. Would that eliminate the fear factor?

[brooksprumo]

Let me try a test that reverts the changes in the runtime, but keeps all the sdk / program-test changes. Would that eliminate the fear factor?

I think for me I feel unqualified to assess the risk—with or without the runtime changes. I think @alessandrod will have a better/complete view of the situation.

[joncinque]

I think for me I feel unqualified to assess the risk—with or without the runtime changes. I think @alessandrod will have a better/complete view of the situation.

Ok no problem at all, thanks for responding.

It does still work with reverted runtime changes, but it's a slightly weird situation: you use the newer Rust compiler to build your program, but you need to use an older version of Rust to build / run the tests, since the change is only included in the program 🙃

[joncinque]

See 15cb917 for the commit that removes runtime changes

[codecov]

Codecov Report

Merging #32797 (e8177d5) into v1.14 (605da4a) will increase coverage by 0.0%.
The diff coverage is 92.3%.

@@           Coverage Diff            @@
##            v1.14   #32797    +/-   ##
========================================
  Coverage    82.1%    82.2%            
========================================
  Files         665      670     +5     
  Lines      185541   185665   +124     
========================================
+ Hits       152464   152618   +154     
+ Misses      33077    33047    -30     

[joncinque]

After a discussion with the team offline, we've decided that this change is too risky to backport. Releases are coordinated with a certain version of the Rust compiler, and it is dangerous to encourage mixing versions.

You'll see all sorts of comments like #29354 (comment) which repeatedly state that you must use the version of Rust that corresponds to the crates.