Invoked an instruction with data that is too large error
#31960
Comments
|
Do you have some details on which specific instructions are throwing it? And is it always the same instructions on each program that errors? |
|
For auction-house example, did you reproduce the issue with the client https://github.com/armaniferrante/auction-house/blob/armani/pda/tests/auction-house.ts ? I wonder if I can get it just using this repo as starting point |
Yes. Switching back to I've run into this issue in Anchor CI and the easiest way to reproduce it would be to clone Anchor and run the mentioned tests(they are submodules). Test workflow is useful to check the commands to run. |
|
Will try to check this out this week |
|
@acheroncrypto here is a simple program to reproduce it: use anchor_lang::prelude::*;
use anchor_spl::token::{self, Mint, TokenAccount};
declare_id!("xxxx");
const METADATA_ACCOUNT_PREFIX: &str = "xxxx";
#[program]
pub mod real_program {
use super::*;
pub fn create_metadata_account(ctx: Context<CreateMetadataAccount>, ) -> Result<()> {
Ok(())
}
}
#[derive(Accounts)]
pub struct CreateMetadataAccount<'info> {
/// CHECK
#[account(mut, signer)]
pub owner: AccountInfo<'info>,
pub mint: Account<'info, Mint>,
#[account(
init,
payer = owner,
seeds = [METADATA_ACCOUNT_PREFIX.as_bytes(), mint.key().as_ref()],
token::mint = mint,
token::authority = owner,
bump,
)]
pub real_metadata_account: Box<Account<'info, TokenAccount>>,
/// CHECK
#[account(address = anchor_spl::token::ID)]
pub token_program: AccountInfo<'info>,
/// CHECK
pub system_program: AccountInfo<'info>,
pub rent: Sysvar<'info, Rent>,
} |
|
@acheroncrypto could you provide details about reproducing this run wtih auction-house? I've tried But it in the validator log program is not mentioned. |
|
@KirillLykov you can see this run. It shows exactly what commands to run to reproduce it. From the same CI logs: This error message is not very helpful but if you get the transaction logs, you will see Note that the first test is a setup step that doesn't even use the |
|
@effeaucarre could you provide also a client to execute this program? |
|
I was getting the above error so when i downgraded to 1.14.18, i am getting the following error. ( anchor version: 0.27.0 ) I cannot upgrade to anchor version 0.28.0 since it is breaking for switchboard-v2 crate. |
|
@KirillLykov here is a script to reproduce the issue: #!/usr/bin/env bash
set -e
# Set solana version
solana-install init 1.16.0
# Create a temp directory
mkdir data-too-large-error
pushd data-too-large-error
# Clone and build token-metadata program
git clone https://github.com/metaplex-foundation/metaplex.git
pushd metaplex
git checkout cbb590d5
pushd rust/token-metadata/program
cargo build-bpf
popd
popd
# Create an Anchor project(only for testing)
anchor init reproduce
cd reproduce
# Add the program to genesis
echo '[[test.genesis]]
address = "metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s"
program = "../metaplex/rust/target/deploy/metaplex_token_metadata.so"' >> Anchor.toml
# Setup tests
yarn add @metaplex/js@=4.4.1
echo 'import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { programs } from "@metaplex/js";
import { Token, TOKEN_PROGRAM_ID } from "@solana/spl-token";
import { PublicKey } from "@solana/web3.js";
import { Reproduce } from "../target/types/reproduce";
const TOKEN_METADATA_PROGRAM_ID = new PublicKey(
"metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s"
);
const { CreateMetadata, MetadataDataData } = programs.metadata;
describe("reproduce", () => {
anchor.setProvider(anchor.AnchorProvider.env());
const program = anchor.workspace.Reproduce as Program<Reproduce>;
const payer = program.provider.publicKey;
it("reproduces", async () => {
const nftMintClient = await Token.createMint(
program.provider.connection,
// @ts-ignore
program.provider.wallet.payer,
payer,
null,
6,
TOKEN_PROGRAM_ID
);
// Create the metadata
const [metadata] = await PublicKey.findProgramAddress(
[
Buffer.from("metadata"),
TOKEN_METADATA_PROGRAM_ID.toBuffer(),
nftMintClient.publicKey.toBuffer(),
],
TOKEN_METADATA_PROGRAM_ID
);
const tx = new CreateMetadata(
{ feePayer: payer },
{
metadata,
metadataData: new MetadataDataData({
name: "test-nft",
symbol: "TEST",
uri: "https://nothing.com",
sellerFeeBasisPoints: 1,
creators: null,
}),
updateAuthority: payer,
mint: nftMintClient.publicKey,
mintAuthority: payer,
}
);
try {
await program.provider.sendAndConfirm(tx);
} catch (e) {
console.log(e);
return;
}
throw new Error("Did not reproduce");
});
});
' > tests/reproduce.ts
# Run tests
anchor test
# Return back to the initial directory
popdThe script uses Anchor only to test from TypeScript and the program that returns this error is not an Anchor program. Compiling the program with
|
|
Reproduced the problem with script above, note that works only with |
|
Based on the following runs, looks like compiler toolchain (not sdk) is the source of problem: # OK
RUST_LOG=debug /home/klykov/.local/share/solana/install/releases/1.14.18/solana-release/bin/cargo-build-bpf --sbf-sdk "/home/klykov/.local/share/solana/install/releases/1.14.18/solana-release/bin/sdk/bpf"
# OK
RUST_LOG=debug /home/klykov/.local/share/solana/install/releases/1.14.18/solana-release/bin/cargo-build-bpf --sbf-sdk "/home/klykov/.local/share/solana/install/releases/1.16.0/solana-release/bin/sdk/sbf"
# Fails
RUST_LOG=debug /home/klykov/.local/share/solana/install/releases/1.16.0/solana-release/bin/cargo-build-bpf --sbf-sdk "/home/klykov/.local/share/solana/install/releases/1.16.0/solana-release/bin/sdk/sbf"
# Fails
RUST_LOG=debug /home/klykov/.local/share/solana/install/releases/1.16.0/solana-release/bin/cargo-build-bpf --sbf-sdk "/home/klykov/.local/share/solana/install/releases/1.14.18/solana-release/bin/sdk/bpf" |
|
@dmakarov might be able to assist if this indeed is a toolchain issue |
|
I was facing the same problem here, I fixed it by downgrading Solana Tool Suite from |
|
I get the same problem with GPL, tests work with |
How to run GPL tests? Is running |
|
|
@acheroncrypto From discussion with @dmakarov, looks like the problem is with the inconsistency of the dependencies version used by metaplex program, solana installation and anchor. I've created a branch https://github.com/KirillLykov/metaplex-program-library/tree/klykov/sol-issue-31960 based on the commit |
|
@KirillLykov thanks but I'll note that those versions hasn't changed in ~2 years and they were also tested with every Solana release within the last ~2 years(without having to change versions). This will hurt composability quite a bit because not only you'd have to upgrade your own dependencies but you'd also potentially need to wait for all of the subdependencies to upgrade as well. |
It looks like it is not on our side directly, we suspect that the problem is related to the borsh update (#30975 (comment)) In this particular case, it was enough to update only direct dependencies of MPL. It looks like the master of this repo already uses quite recent version of solana / anchor so updating is hopefully not a problem (if it is only about mpl) metaplex-foundation/metaplex-program-library#1137 |
|
After downgrading to |
is this an opensource project? I want to check the cargo tree to see what do you mean. I guess your project depends on some crates that are requiring |
It's one of the most reported problems and the error message is not very helpful(#31428). Here are some examples from the StackExchange:
It's a miracle if you can compile a program with |
I was confused with the original message, I thought it is about solana and borsh update which lead to the current problem. |
The problem in #31428 is not directly related to the current issue but it's still related in a sense that when people upgrade to |
|
@KirillLykov I finally have my problem solved by following #31775. I used to think the solution can either be upgrading to 1.16 or downgrading to 1.14. However, for upgrading. For downgrading and using stable toolchain. Due to underlying libraries updated their dependencies, now they conflict with old cargo(1.62). |
Short summary of the aboveUpgrade of
My understanding is that it happens because the client program which triggers execution of the on-chain program serializes instruction data using borsh 0.9, while on-chain program uses newer version to deserialize it. (correct me if I'm wrong). Do we know if borsh serialization was used to write serialized data to on-chain accounts? Like program with borsh 0.9 writes data in one format, while a program with borsh 0.10 reads this data and misinterprets it? How it can be solved
Other issuesThere is another issue explained in The error message for MSRV mentioned here. My understanding is that it happens when the user updated their installation to the latest but later returned back to 1.14 so that some crates use versions that cannot be compiled with the rustc version used internally by cargo-build-sbf. (Correct me if I'm wrong) @acheroncrypto could you confirm that this summary is correct? |
Yes, it's a good summary overall but I'm not fully convinced of the analysis for the following reasons.
Anchor version is irrelevant here, as I mentioned in my script post, Anchor is only being used for the client side tests and not as a program crate. Here are the dependencies that the program in the reproducable script example uses: https://github.com/metaplex-foundation/metaplex/blob/cbb590d53a3a9718ecf2b526580952fc6d10baaf/rust/token-metadata/program/Cargo.toml#L15-L23
Even if that's the case, why does the on-chain program use
To my knowledge, Solana build tools is not opinionated on which de/serialization was used in the program, so if there is a program with a lock file, that used to work, should continue to work independent of the build tools/CLI version used. Why? Because the program doesn't even have a dependency to the new |
|
@acheroncrypto thanks for the comments, I somehow missed the part of why borsh has been upgraded after cli upgrade. Should be lock file which controls that, need to dig a bit more into that |
|
I figured this out finally! I was able to reproduce this reliably by upgrading my tools to 1.16, and then building and running a program that uses the 1.14 crates. The program just does a token transfer in a CPI: https://github.com/solana-labs/solana-program-library/tree/master/examples/rust/transfer-tokens If you build the program with 1.16 tools, 1.14 crates, and dump the raw byte representation of the transfer instruction, ie: You'll get: And with 1.14 tools, 1.14 crates, you get: What's going on here? This time, we can blame the Rust language! (But it's not their fault, because they explicitly say that they can change the internals) The Solana tools v1.16 use a Rust compiler v1.68. In Rust v1.66, the byte layout of To preserve ABI compatibility, when you call solana/sdk/program/src/program.rs Line 296 in b7c2ad5 But in our case, we're using the 1.16 compiler, which breaks Until we can backport those fixes to 1.14, the workaround for this issue is to be sure to always use the 1.16 crates with the 1.16 compiler, otherwise your program will break |
|
This explanation makes sense, nice spot @joncinque!
The issue has been:
It was a perfect storm for dependency issues. Glad it's being resolved now! |
|
@acheroncrypto can we close this issue if it looks resolved? |
The root cause of the issue has been identified but many people are still blocked with an older |
|
After a discussion with the team offline, we've decided that the change is too risky to backport. Releases are coordinated with a certain version of the Rust compiler, and it is dangerous to encourage mixing crate and Rust versions in any capacity. It's a bigger risk for programs to behave unexpectedly or incorrectly, so even though it's painful in the short term, people must find a way to get their dependencies up to date to use the newer compiler. If you still want to use old versions and get errors from dependencies that are automatically updated, I encourage you to use an old working lockfile. https://github.com/solana-labs/solana-program-library/tree/ca7d3876313ed3ea6138aa3c1e00c23896e91949 is the last version of SPL that works with 1.14. To properly run tests, I used this old version, where all the dependencies are properly pinned to avoid pulling in newer crates. And while I don't recommend it for real program deployments, you can always patch the Solana crates with a local copy that has #32797 applied. |
|
This issue is dangerous and should be highlighted in the core documentation. It can cause issues with redeployment of old, but extremely stable programs. |
|
Based on the discussion here, my understanding is that the safest way to proceed is to use latest 1.16 version for all three: solana-cli, runtime, and solana-program. Am I right, @jon? |
Yep, that's right -- at the very least, they need to use 1.16 for |
|
Got this error again:
|
Problem
Some programs raise
Invoked an instruction with data that is too largeerror with1.16.0release.I wasn't able to reproduce this error in a smaller program but I can share 3 different programs where this happens.
Logs from this instruction with
1.16.0(platform-tools v1.37):Pre
1.16.0(platform-tools v1.29):All examples work pre
1.16.0and don't work with1.16.0and seems like others who have used the1.16.0version have run into this issue as well. See #31337 (comment), #30271 (comment).The text was updated successfully, but these errors were encountered: