Unable to index starting from block 4000000

[Walter-Phillips]

Issue

Thanks for opening an issue on the Fuel Indexer project.

*If you're an internal FuelLabs contributor, please feel free to remove this template when creating issues, as this template is meant for external users and contributors.

Prerequisites

• Can you reproduce the problem?
• Do you know which version of the indexer you're using?
• Did you already check for an answer in previous Discussions?

Description

• When I put a start block in the indexer manifest file that is 4000000 or greater I get a wasm error

Detailed information helps us narrow down any potential causes.

Steps to Reproduce

Please list the steps required to reproduce your issue in detail.

1. Change the manifest.yaml to start at some block, I've been using 4500000
2. Deploy the indexer (I'm using some absolute paths you'll have to change, sorry)
3. After a few seconds you get this error:
   
   This runs 10 times then fail like the message says

Expected behavior: Able to index from any block

Actual behavior: Can only index from blocks less than 4000000

Versions

Please list what version of indexer components you're using. Latest is currently .

• fuel-indexer: 0.21.0
• forc-index: 0.21.0

The staging branch should have everything you will need

[deekerno]

@Walter-Phillips What network are you on? And just to be sure, you can't start indexing at blocks greater than 4,000,000, but you're still able to index blocks greater than that, right?

[Walter-Phillips]

Hey @deekerno we're on beta-4. I haven't let the indexer run long enough to get to blocks past 4,000,000 from block 1, so I'm not sure.

[deekerno]

Can you start indexing from a number lower than 4,000,000 but still close to it (e.g. 3.9M, 3.5M, etc.) and see if it indexes past that point?

[Walter-Phillips]

@deekerno, doing that now and will let you know how it goes.🫡

[Walter-Phillips]

@deekerno This is what ended up happening, didn't make it very far. Tried form 3.9 and 3.99.

[deekerno]

I'll take a look at your repo and see if I can replicate the issue.

[Walter-Phillips]

Thank you, appreciate it🙏🏾

[deekerno]

@Walter-Phillips: Oddly enough, I was able to replicate your issue with the main branch of your repository on a few blocks:

block 3909602
2023-10-20T19:25:45.727653Z ERROR fuel_indexer::executor: 980: Indexer(sail.sail_indexer) WASM execution failed: Panic.
2023-10-20T19:25:45.730670Z ERROR fuel_indexer::service: 396: Indexer(sail.sail_indexer) terminated with an error: panicked at src/lib.rs:4:1:
Failed decoding.: InvalidData("tried to read 64 bytes from response but only had 32 remaining!")

block 3911066
2023-10-20T19:30:17.896587Z ERROR fuel_indexer::executor: 980: Indexer(sail.sail_indexer) WASM execution failed: Panic.
2023-10-20T19:30:17.898488Z ERROR fuel_indexer::service: 396: Indexer(sail.sail_indexer) terminated with an error: panicked at src/lib.rs:4:1:
Failed decoding.: InvalidData("tried to read 64 bytes from response but only had 40 remaining!")

block 3911068
2023-10-20T19:33:01.166614Z ERROR fuel_indexer::executor: 980: Indexer(sail.sail_indexer) WASM execution failed: Panic.
2023-10-20T19:33:01.168039Z ERROR fuel_indexer::service: 396: Indexer(sail.sail_indexer) terminated with an error: panicked at src/lib.rs:4:1:
Failed decoding.: Utf8Error(Utf8Error { valid_up_to: 14, error_len: Some(1) })

block 3911093
2023-10-20T19:34:18.374232Z ERROR fuel_indexer::executor: 980: Indexer(sail.sail_indexer) WASM execution failed: Panic.
2023-10-20T19:34:18.375747Z ERROR fuel_indexer::service: 396: Indexer(sail.sail_indexer) terminated with an error: panicked at src/lib.rs:4:1:
Failed decoding.: InvalidData("tried to read 64 bytes from response but only had 40 remaining!")

But I do not see the same errors when running a more complex indexer (in this case, that would be our block explorer). That leads me to believe that the indexer module on the main branch has something that's causing these errors. I'm not exactly sure what that is yet, but I figured I'd share it with you first.

[Walter-Phillips]

@deekerno Cool, thank you. I'll remake the indexer that we have from scratch and see what happens

[Walter-Phillips]

@deekerno I remade an indexer from the quickstart guide. I incrementally made changes and it failed whenever I added an abi. I've tried two abis but they are both ones from projects I've built. Going to try it with something from Sway applications. Could it be an incompatibility with the version of sway the abi gets made from and the indexer?

[deekerno]

@Walter-Phillips I'll look into the ABI idea today. 👌🏽

[deekerno]

@Walter-Phillips Can you invite @lostman to your repository as well? That way, you can have the entire indexer team looking into the problem (I'm also up to my neck in revamping our GraphQL functionality 😭).

[Walter-Phillips]

@deekerno no problem. Just sent the invite to @lostman. Good luck with the revamp :)).

[deekerno]

@Walter-Phillips How are str[64] ID fields constructed for your structs? I adjusted our codegen to output the type upon a failed decode:

block 3909602
Failed decoding.: InvalidData("tried to read 64 bytes from response but only had 32 remaining!")
-> Decoding CancelLimitOrder

block 3911066
Failed decoding.: InvalidData("tried to read 64 bytes from response but only had 40 remaining!")
-> Decoding TakeMarketOrder

block 3911068
Failed decoding.: Utf8Error(Utf8Error { valid_up_to: 14, error_len: Some(1) })
-> Decoding CreatePostOnly

block 3911093
Failed decoding.: InvalidData("tried to read 64 bytes from response but only had 40 remaining!")
-> Decoding TakeMarketOrder

It seems that there may be some issue around your strings?

[Walter-Phillips]

@deekerno This is the sway struct. Right now in the client, we use the predicate address and convert it to a string. We're calling the uid helper on it in the handler which we no longer need, which could be the problem. I'll drop the uid helper and try indexing again from 4,000,000.

I am a bit confused about it trying to decode Market Orders and Post Onlys, we haven't created any on testnet

[Walter-Phillips]

Also, how were you able to edit the codegen?

[deekerno]

That's certainly interesting. 🤔 and I'm fairly sure that dropping the UID helper (I'd recommend to use something like the get_or_create() method here [which I have yet to document]) will help as the decoding failure is happening in the #[indexer] macro at the ABI decoding step from the SDK; specifically, it's happening here:

                let decoded = ABIDecoder::decode_single(&#type_tokens::param_type(), &data).expect("Failed decoding.");
                let obj = #type_tokens::from_token(decoded).expect("Failed detokenizing.");
                self.#name.push(obj);

So it's definitely within code that we wrote; I'm just wondering if there's something special that you guys have done that we should try to replicate.

re: editing the codegen -- I just adjusted the above part of the code in the fuel-indexer repository and rebuilt your indexer module to log the tokens for the failed type.

[Walter-Phillips]

@deekerno Tried doing it like that and got the same error, without using the id from our client. There isn't anything special we're doing as far as I know. I also attached examples of us creating the limit order struct in rust and ts, in our tests.

[deekerno]

Yeah, I think it's definitely something on our side. We'll put our proverbial heads together and see what we can find in the next day or so. 👍🏽

[Walter-Phillips]

Thank you. I know you guys are busy so I really appreciate it. Let me know if you need anything from me.

[ra0x3]

@Walter-Phillips Could you share the link to your repo btw? (I know you invited us to the repo, but I don't remember the link)

[Walter-Phillips]

@ra0x3 https://github.com/Sail-Exchange/Sail-Core

[deekerno]

@Walter-Phillips I noticed that your contracts are using forc versions ranging from v0.42.1 to v0.45.0. There were a few changes to how strings are represented starting with v0.46.0. Additionally, the indexer does its best to keep up with the latest versions of the Fuel ecosystem, so when there's a disparity between a contract's ABI and what the indexer has been developed for, it can potentially result in things like this. Do you mind updating your contracts to use the latest forc version available through the beta-4 channel through fuelup? You may need to adjust your contracts to incorporate the String improvements.

[Walter-Phillips]

@deekerno I'll look into that and see if it helps, would explain why it's an issue even for types we haven't logged to testnet yet

[Walter-Phillips]

@deekerno updated to 0.46 and had the same error. I am still using string arrays for the IDs, should I be using string slices?

[ra0x3]

@deekerno Where are we on this? @Walter-Phillips Did you ever get this figured out?

[Walter-Phillips]

@ra0x3 I was able to fix this by changing how we were handling logs in our contracts and updating everything to newer versions. @deekerno Thanks for the help.