Skip to content

Comments

Add logging on DataColumnSidecars recovery failure#9486

Merged
zilm13 merged 2 commits intoConsensys:dasfrom
zilm13:recovery-logging
May 27, 2025
Merged

Add logging on DataColumnSidecars recovery failure#9486
zilm13 merged 2 commits intoConsensys:dasfrom
zilm13:recovery-logging

Conversation

@zilm13
Copy link
Contributor

@zilm13 zilm13 commented May 24, 2025

PR Description

We've got such exception in FULU:

2025-05-23 14:44:10.212 ERROR - PLEASE FIX OR REPORT | Unexpected exception thrown for beaconchain-async-2
java.util.concurrent.CompletionException: java.util.NoSuchElementException: No value present
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source) ~[?:?]
at tech.pegasys.teku.infrastructure.async.stream.AsyncIteratorCollector.onError(AsyncIteratorCollector.java:44) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.stream.MapStreamHandler.onNext(MapStreamHandler.java:33) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.stream.FlattenStreamHandler$1.onNext(FlattenStreamHandler.java:32) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.stream.FutureAsyncIteratorImpl.lambda$iterate$1(FutureAsyncIteratorImpl.java:30) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$finish$33(SafeFuture.java:396) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at java.base/java.util.concurrent.CompletableFuture.uniHandle(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.complete(Unknown Source) ~[?:?]
at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$propagateResult$3(SafeFuture.java:152) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.complete(Unknown Source) ~[?:?]
at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$propagateToAsync$24(SafeFuture.java:357) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.SafeFuture.of(SafeFuture.java:84) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.AsyncRunner.lambda$runAsync$2(AsyncRunner.java:47) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.SafeFuture.of(SafeFuture.java:76) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at tech.pegasys.teku.infrastructure.async.ScheduledExecutorAsyncRunner.lambda$createRunnableForAction$1(ScheduledExecutorAsyncRunner.java:124) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
at java.base/java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.util.NoSuchElementException: No value present
at java.base/java.util.Optional.get(Unknown Source) ~[?:?]
at tech.pegasys.teku.infrastructure.async.stream.MapStreamHandler.onNext(MapStreamHandler.java:31) ~[teku-infrastructure-async-develop.jar:25.4.0+267-g092d6a2170]
... 20 more

I've no ideas how it happens, but I've added more logging in the place where I think it occurs. When it happens again it should be a bit easier to track it.

Fixed Issue(s)

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

Copy link
Member

@lucassaldanha lucassaldanha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm approving it but we should evaluate if this error is something that might pop up too often and pollute our logs.

@zilm13 zilm13 merged commit c32dc4d into Consensys:das May 27, 2025
14 checks passed
@zilm13 zilm13 deleted the recovery-logging branch May 27, 2025 14:07
lucassaldanha pushed a commit to lucassaldanha/teku that referenced this pull request May 27, 2025
@lucassaldanha
Copy link
Member

Cherry-picked on master-das today 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants