Client's SubscribeFilterLogs stops reporting events after some time

[beeb]

System information

go-ethereum v1.10.8
OS & Version: linux

Expected behaviour

When using ethclient.Client.SubscribeFilterLogs, all events matching the filter are caught.

Actual behaviour

After some time, the script stops reporting new events and it needs to be restarted to start working again.

Steps to reproduce the behaviour

var EthClient *ethclient.Client
// ...
currentBlock, _ := EthClient.BlockNumber(context.Background())
query := ethereum.FilterQuery{
	Addresses: []common.Address{myAddress},
	Topics:    [][]common.Hash{{myTopic}},
	FromBlock: big.NewInt(int64(currentBlock)),
}
logs := make(chan types.Log)
sub, err := EthClient.SubscribeFilterLogs(context.Background(), query, logs)
if err != nil {
	log.Fatal(err)
	return
}

for {
	select {
	case err := <-sub.Err():
		log.Error(err)
		continue
	case vLog := <-logs:
		event, err := MyContract.ParseSomeEvent(vLog)
		if err != nil {
			log.Error(err)
			continue
		}
		// do something here with event

	}
}

Hard for me to convey what needs to be done. I guess try to leave a script running for days and see if it stops catching new events after a while.

I'm not sure what could cause this issue. I automated script restart with cron to try to mitigate this issue but it still happens that the SubscribeFilterLogs stops reporting new events.

[ligi]

Any chance you could provide some of the geth log around the time it happens?

[beeb]

Hard for me to identify when it starts failing as I monitor for a few events a day only... Might try to find something by finding events that were missed manually and looking at the time.

[Vanclief]

Having the same issue.

Could it be related to a disconnection and the function is not notified? Maybe keep making a ping to the WS to avoid disconnection?

[beeb]

Having the same issue.

Could it be related to a disconnection and the function is not notified? Maybe keep making a ping to the WS to avoid disconnection?

Not for me as I'm connected through IPC.

[Vanclief]

Weird, I am connecting over an Alchemy/Infura websocket and it will intermittently drop events after some time.

I redeployed my service today while including the Subscription call inside the for, to see if by recreating the subscription on each loop, it would not drop. I am not sure if it works on the long term but so far it hasn't given me an issue.

var EthClient *ethclient.Client
// ...
currentBlock, _ := EthClient.BlockNumber(context.Background())
query := ethereum.FilterQuery{
	Addresses: []common.Address{myAddress},
	Topics:    [][]common.Hash{{myTopic}},
	FromBlock: big.NewInt(int64(currentBlock)),
}
logs := make(chan types.Log)

for {
        sub, err := EthClient.SubscribeFilterLogs(context.Background(), query, logs)
        if err != nil {
	        log.Fatal(err)
	        return
        }
	select {
	case err := <-sub.Err():
		log.Error(err)
		continue
	case vLog := <-logs:
		event, err := MyContract.ParseSomeEvent(vLog)
		if err != nil {
			log.Error(err)
			continue
		}
		// do something here with event

	}
}

[beeb]

Thanks @Vanclief , looks like it could be a helpful workaround if the subscription ends up in a bad state somehow after an event is picked up.

I have for now abandonned the project that was using the code above so I won't be able to test it immediately, but I plan on running it again at some point.

[yelhouti]

@Vanclief i am facing a similar issue, and it seems to me that the when an error happen, the subscription get stuck in an infinite look emit and error to the channel with null and logging error happened

[xiezm]

https://github.com/hydroscan/hydroscan-api/blob/a8882653a5aaf2dd2f9fa397400c160cef0488b3/task/event_subscriber.go

https://github.com/SmartMeshFoundation/photon-mobile/blob/2641db709d0aad553a8e46d406c4aa6539c16627/network/helper/safeethclient.go

Here's a solution that , hope will help you

[yelhouti]

@xiezm thanks a lot, this could definitely help.

I still think that this issue should be fixed here.

[Vanclief]

@xiezm Thanks for sharing, that definitely looks like the correct way to do it instead of just spamming new ws connections.