RPC-only indexer restarts continuously and drains our Alchemy API consumption

[PaulRBerg]

Indexer URL: https://envio.dev/app/sablier-labs/lockup-envio/c54f4cb

The affected chain is Sei Network (chain ID 1329). Our RPC config for this chain looks like this:

rpc:
  - for: "sync"
    url: "https://sei-mainnet.g.alchemy.com/v2/${ENVIO_ALCHEMY_API_KEY}"
  - for: "fallback"
    initial_block_interval: 2000
    interval_ceiling: 2000
    url: "https://evm-rpc.sei-apis.com"
  - for: "fallback"
    initial_block_interval: 2000
    interval_ceiling: 2000
    url: "https://sei.drpc.org"

The indexer has been running for some time now, and it's drained our Alchemy consumption:

Despite that, the indexer (appears to) restarts continuously (or at least it appears to be doing that). It's never at 100%.

[DZakh]

I'll appreciate some steps to run it locally for debugging 🙏

[PaulRBerg]

I can DM you the Alchemy key if you want

git clone https://github.com/sablier-labs/indexers && cd indexers
git switch prb-refactor
pnpm install
just codegen-envio
cd envio/lockup
just envio dev

[DZakh]

No need, I'll use mine. Looking at it now.

[DZakh]

@PaulRBerg Could you share a full Sei config. I don't see it in the branch

[PaulRBerg]

  - contracts:
      - address: "0x736a6e895790e089aec2bf76b2d7f368ce6efff5"
        name: "SablierLockup_v2_0"
    id: 1329
    rpc:
      - for: "sync"
        url: "https://sei-mainnet.g.alchemy.com/v2/${ENVIO_ALCHEMY_API_KEY}"
      - for: "fallback"
        initial_block_interval: 2000
        interval_ceiling: 2000
        url: "https://evm-rpc.sei-apis.com"
      - for: "fallback"
        initial_block_interval: 2000
        interval_ceiling: 2000
        url: "https://sei.drpc.org"
    start_block: 138913911

[DZakh]

I couldn't reproduce it locally. It's indexing perfectly fine for me, there is a percentage of wasteful queries, but it shouldn't affect the indexing progress. Additionally, alchemy returns a maximum of 500 blocks in the response, which makes indexing very slow.

I'd recommend finding an RPC provider that can return more blocks in a single response. Usually, these are some specialized on reads.

ps. There's a project to speed up RPC sync, I'll take a look at wasteful queries when I'm at it

[PaulRBerg]

Strange. Why did we then see 0% indexing on Hosted Service UI? And why did it consume so much of our Alchemy API quota (or is that expected?)?

At any rate, thanks for looking into it.

[DZakh]

Try to run with LOG_LEVEL=trace if you have the issue again (locally). Or let me know when you have it deployed on the Hosted Service, the deployment was already deleted when I checked.

[PaulRBerg]

alchemy returns a maximum of 500 blocks in the response

Actually, it looks like they return 2000 blocks: