Configurable wsMessageSizeLimit for websocket RPC connections (websocket: read limit exceeded)

[flyte]

Rationale

Currently, messages are limited to a size of 15MB using a wsMessageSizeLimit = 15 * 1024 * 1024 constant.
It would be helpful if this would be configurable (as well as perhaps some of the other constants like timeouts).

I'm running my own geth node on one machine and querying it from another machine on the local network using the RPC client. I'm deliberately performing large queries on my node and the network transport isn't a bottleneck, so I'd like to configure my RPC client to accept much larger response sizes (it has no problem when using an IPC socket when running locally on the node) without returning an error:

websocket: read limit exceeded

I haven't tried the HTTP connection, but that has an even smaller limit of 5MB and is reportedly slower, which doesn't work for my case in which I'm making millions of calls to my node.

Implementation

As long as it's somehow configurable, I can't say what the best implementation would be (only been writing Go for a week or so).

[ligi]

Can you elaborate on your use-case?

[flyte]

I'm pulling logs out of my Eth node for a range of contract addresses, getting their receipts and getting the blocks that they were confirmed in so that I can calculate the gas used (receipt.GasUsed) for all of those contracts per day (block.Time()) vs how many ERC-20/ERC-721 and ERC-1155 Transfer events (log.Topics) had taken place.

I've been using GraphQL for these queries since of course it makes it easy to tie the three types together into a single query, but sadly it's not easy to find a hosted Eth node that provides GraphQL (Infura no longer do it without paying for a whole node at $300+/m)

This obviously means I'm making millions of queries, so I'm using batch queries for this (IIRC, this means sometimes grabbing a few thousand receipts in a single query). Understandably this puts a reasonable load on my Eth node, but since it's dedicated to this task it's not an issue. As I mentioned, this works well when connected over IPC but what would ordinarily be fine and (relatively) reasonable over a websocket too, is artificially limited by this fixed 15MB limit on the websocket client.

[vr-devil]

i need it. @ligi

[vr-devil]

my log filter.

func (job *ScanJob) CreateFilterQuery(fromBlock, toBlock uint64) ethereum.FilterQuery {
	return ethereum.FilterQuery{
		FromBlock: big.NewInt(int64(fromBlock)),
		ToBlock:   big.NewInt(int64(toBlock)),
		Topics: [][]common.Hash{
			{
				common.HexToHash(chain.TopicTransfer),
				common.HexToHash(chain.TopicTransferSingle),
				common.HexToHash(chain.TopicTransferBatch),
			},
		},
	}
}

read limit exceeded, when just read 2 blocks.

time="2022-08-17T01:38:34Z" level=info msg="get logs from: 15260605 to 15260606, remain: 95247."
time="2022-08-17T01:38:35Z" level=error msg="failed to filter logs, err: websocket: read limit exceeded."

[tarik0]

Just like @flyte, I'm also dealing with mass queries and I also need to change the 15MB size. I'm developing a simulation application and I have to call debug_traceCall multiple times. Some of the logs are massive and sometimes they take more than 15 MB. Are there any way to change this variable ?

[tarik0]

Until a implementation, I've did a workaround using function hooks with github.com/brahma-adshonor/gohook. You can link the newWebsocketCodec using Go's go:linkname and hook it. It's a nasty workaround but I couldn't find any other way lol.

package main

import (
	"fmt"
	"github.com/brahma-adshonor/gohook"
	"github.com/ethereum/go-ethereum/rpc"
	"github.com/gorilla/websocket"
	"net/http"
)

// wsMessageSizeLimit is 50 MB
const wsMessageSizeLimit = 50 * 1024 * 1024

//go:linkname newWebsocketCodec github.com/ethereum/go-ethereum/rpc.newWebsocketCodec
func newWebsocketCodec(*websocket.Conn, string, http.Header) rpc.ServerCodec

// newWebsocketCodecHook is a hook for the newWebsocketCodec.
func newWebsocketCodecHook(conn *websocket.Conn, host string, req http.Header) rpc.ServerCodec {
	codec := newWebsocketCodecTramp(conn, host, req)
	conn.SetReadLimit(wsMessageSizeLimit)
	return codec
}

// newWebsocketCodecTramp is a tramp for the newWebsocketCodec.
func newWebsocketCodecTramp(*websocket.Conn, string, http.Header) rpc.ServerCodec {
        // add "-gcflags=-l" to disable inline functions. 
	panic("hooking failed")
}

func main() {
	// Hook the websocket.
	err := gohook.Hook(newWebsocketCodec, newWebsocketCodecHook, newWebsocketCodecTramp)
	if err != nil {
		panic(err)
	}
	
	// Connect to the RPC...
}

Also, to silence missing function body create a empty file that ends with the .s extension in the same folder.

[quentinlesceller]

👍 Would be great to have configurable wsMessageLimit.

[tylerni7]

Just bumping this issue... It's pretty obnoxious to not have this configurable. The low, hardcoded value has led to issues:

#26883 (realized it needed to be bumped)
#26967 (oops still too low!)

Would the maintainers be amenable to a PR with this being configurable?

[holiman]

Fixed by #27801 (thanks @tylerni7 )