eth_subscribe logs subcription skipping logs

[adytzu2007]

Hey, we've found an issue with subscriptions for logs in erigon. Basically, if there are more than 128 logs, it's very likely that some of them will be dropped because the channel is too small. We made a hotfix in a private branch and verified that this was indeed the root cause. You can find the commit here.

We were thinking about opening a PR and making this configurable but I'm not sure where the config should reside and whether this is the proper fix so I wanted to gather some feedback before starting to work on a more permanent solution.

[somnergy]

Please add example(s)

[adytzu2007]

I was using this script: https://gist.github.com/adytzu2007/f3adf156a2229f85d5e660f24354bddb
It's more apparent on polygon where blocks are bigger but I could easily reproduce for ethereum as well. I sent a PR to make it configurable but the feedback is that it would be better to use a growing buffer instead.

[somnergy]

If you could reproduce on Ethereum mainnet, it would be greatly helpful

[adytzu2007]

You can reproduce this by using https://ethereum.blockpi.network/v1/rpc/public in the script I linked above. It's a public RPC running erigon.
An example of the run:

{'jsonrpc': '2.0', 'method': 'eth_subscription', 'params': {'subscription': '0xf89c82be5f7e0980ff8bd49db4dcc90e', 'result': {'address': '0x76611eea26af8842281b56bb742129e77133592f', 'topics': ['0x9563ff6035b973f2e4514ad9315010c220eb74b0c33a782a18118a199a97e442', '0x00000000000000000000000017c07e0c232f2f80dfdbd7a95b942d893a4c5acb', '0x000000000000000000000000535469eb1e1a4338fc9085202fe2758b8071278a'], 'data': '0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c097ce7bc90715b34b9f1000000000', 'blockNumber': '0x12b04af', 'transactionHash': '0xabfa17d83f4d01b338b71893c26622ee1dce0be31697fbfc171c0e4dd422876d', 'transactionIndex': '0x5d', 'blockHash': '0x5f580672cef50c7ba6b7835fab73bde40e00d7d2ae6b121ac583b75edcbaa948', 'logIndex': '0x80', 'removed': False}}}
{'jsonrpc': '2.0', 'method': 'eth_subscription', 'params': {'subscription': '0xf89c82be5f7e0980ff8bd49db4dcc90e', 'result': {'address': '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2', 'topics': ['0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef', '0x000000000000000000000000975b2383190bbe232918bbc0df75dfe884c3b370', '0x0000000000000000000000001111111254eeb25477b68fb85ed929f73a960582'], 'data': '0x00000000000000000000000000000000000000000000000000534c3591168530', 'blockNumber': '0x12b04af', 'transactionHash': '0x8213afb97bc1c8ec56d1f23b56754efd23a88b92377c8425d4bcf5516f9acb9f', 'transactionIndex': '0x6d', 'blockHash': '0x5f580672cef50c7ba6b7835fab73bde40e00d7d2ae6b121ac583b75edcbaa948', 'logIndex': '0xbf', 'removed': False}}}

The logIndex jumps from 0x80 to 0xbf. The script will output the following when it happens

AssertionError: (19596463, 19596463, 'expected next log index to be 129, got 191')

[adytzu2007]

The problems stems from here. The subscription channel can hold at most 128 entries at a time. The channel is allocated here and the behavior when the channel is full is to drop messages

[yperbasis]

Fixed by PR #9810