Event-polling across consecutive blocks is broken

[therealjmj]

Ethers Version

6.4.0

Search Terms

No response

Describe the Problem

It's patched in my local version and working to scan my "Shield" event which occurs across 2 consecutive blocks during automated tests. The event was completely lost - it took me about 6 hours today to track it down.

I believe this is a logical off-by-one bug - the fix is below. Hope it helps :)

Code Snippet

inside PollingEventSubscriber.#poll:

-        filter.fromBlock = this.#blockNumber + 1;
+        filter.fromBlock = this.#blockNumber; // Updated to fix a bug in polling scans for the same filtered events across consecutive blocks (ie. during automated tests).

Contract ABI

No response

Errors

No response

Environment

No response

Environment (Other)

No response

[ricmoo]

Thanks! I look into it shortly. There was a change in v5 to v6 how the last scanned block was indexed (lastScanned vs nextToScan).

Thanks for your research! :)

[thisisyou33-publishing-networks]

Thank You 🙏

…

On Fri, Jun 2, 2023, 6:25 PM Richard Moore ***@***.***> wrote: Thanks! I look into it shortly. There was a change in v5 to v6 how the last scanned block was indexed (lastScanned vs nextToScan). Thanks for your research! :) — Reply to this email directly, view it on GitHub <#4119 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMTUCNKMVPIUBNJZTRG3W4LXJJY6DANCNFSM6AAAAAAYY3KETE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ricmoo]

What network are you on?

I've traced through the logic and tried a few experiments, and I think the current logic makes sense. There was a small problem on networks which are "fast" (like BNB, Polygon and Arbitrum) that I've fixed locally.

But trying to track down what might have been the cause of your issue...

[ricmoo]

I made a change that might also help with this, if it was caused by networks with non-consistent events. Can you try out v6.4.2?

[therealjmj]

This was on Ethereum.
Not Fixed: v6.4.2 has the same issue, my change is still necessary to get tests to pass correctly.

If you're interested in trying it yourself, in context, see:

https://github.com/Railgun-Community/cookbook

To test:

1. Isolate the beefy fork test (add only to describe.only(... in FORK-beefy-vault-recipes)
2. Run anvil locally: ./run-anvil Ethereum https://your/eth/rpc/url
3. Run tests: env NETWORK_NAME=Ethereum yarn test-fork (note, it takes about 40 sec to run)

If you try without the patch, the tests will fail with incorrect values. This is because the proper "Shield" event was not synced (the block gets skipped by the poller).

Note that the current version of ethers is set to 6.4.0, and the above code patch is in the patches folder.

[ricmoo]

But that patch will duplicate events from blocks, since you are fetching logs for [ x, x + y ] and then [x + y, x + y + z ]. Since x + y was processed twice, and logs in that block would be repeated.

Can you add a console.log inside there to print the ranges being fetched, and include the number of logs fetched?