Erigon 0.40.1 mainnet + embeded CL stopped syncing

[zekchan]

System information

Erigon version: ./erigon --version
erigon version 2.40.1-stable-94b4cee5
OS & Version: Windows/Linux/OSX
ubuntu 22.04 Docker
Commit hash:

Erigon Command (with flags/config):

"--txpool.disable",
                "--v5disc",
                "--lightclient.discovery.addr=***",
                "--log.dir.path=/data/logs",
                "--chain",
                "mainnet",
                "--metrics",
                "--rpc.returndata.limit=999999999",
                "--rpc.batch.limit=2000",
                "--metrics.addr",
                "***",
                "--metrics.port",
                "6060",
                "--datadir",
                "/data",
                "--torrent.download.rate=1000mb",
                "--http.api=eth,erigon,web3,net,debug,trace,txpool",
                "--http.addr",
                "***",
                "--http.port",
                "***:,
                "--http.vhosts",
                "*",
                "--ws",
                "--http"

Concensus Layer:
Embeded
Concensus Layer Command (with flags/config):

Chain/Network:
Mainnet

Expected behaviour

Syncing

Actual behaviour

I have two erigons in docker with same setups. Both stoppped 16850951-16850951 and started do somehing only in several minutes with long lags

Logs

Nothing just that in both nodes:

[WARN] [03-18|00:44:35.849] Could not validate update                err="invalid sync committee"
[WARN] [03-18|00:44:36.252] Could not validate update                err="invalid sync committee"
[WARN] [03-18|00:44:38.259] Could not send beacon block to ETH1      err="missing attestation data"
[WARN] [03-18|00:44:41.850] Could not validate update                err="invalid sync committee"
[WARN] [03-18|00:44:44.781] Could not validate update                err="invalid sync committee"

Sync stages graph:

[tirumerla]

I'm seeing similar issues on one of the node with 2.40.1

[fernandezpaco]

Same here but different logs, i created an issue but i have closed it in favor of this one just in case it is the same problem

#7131

[egorFiNE]

2.40.1, mainnet.

Same issue. Node stopped at 16850994.

[whidrasl]

same. block 16854519

  |   | [WARN] [03-18\|16:32:32.195] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:32.833] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:33.033] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:33.434] Could not validate update                err="invalid sync committee"
  |   | [INFO] [03-18\|16:32:33.963] [p2p] GoodPeers                          eth68=102 eth66=9 eth67=91
  |   | [WARN] [03-18\|16:32:34.035] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:58.668] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:59.268] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:59.473] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:59.673] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:32:59.873] Could not validate update                err="invalid sync committee"

...

  |   | [WARN] [03-18\|16:33:00.274] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:33:01.075] Could not validate update                err="invalid sync committee"
  |   | [INFO] [03-18\|16:33:01.727] [LightClient] P2P                        peers=180
  |   | [INFO] [03-18\|16:33:01.741] [txpool] stat                            block=16854519 pending=10000 baseFee=3044 queued=30000 alloc=3.9GB sys=8.3GB
  |   | [WARN] [03-18\|16:33:02.327] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:33:02.527] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:33:03.401] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:33:04.201] Could not validate update                err="invalid sync committee"
  |   | [WARN] [03-18\|16:33:05.403] Could not validate update                err="invalid sync committee"

[whidrasl]

same on Goerli

[whidrasl]

same for v2.38.1 (with embedded Consensus layer)

[egorFiNE]

Sometimes it syncs a few blocks then continues.

[wainach]

Seeing same issue.

[winchell]

Same. Goerli. I have another Goerli node using Lighthouse CL and it's fine. v2.40.1 for both.

[Giulio2002]

#7134 fixed this

[whidrasl]

there is no more such logs since 19:50 GMT. Even without fix/without restarting nodes/with v2.40.1/v2.38.1

[egorFiNE]

Same here, it stopped without fixes and happily runs for at least 10h now.

[egorFiNE]

...and we're back.

@Giulio2002 could you please release a stable version with this fix included?