Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SEVERE error in integration tests due to update of OpenTelemetry #290

Closed
zakkak opened this issue Oct 17, 2024 · 4 comments · Fixed by #291
Closed

SEVERE error in integration tests due to update of OpenTelemetry #290

zakkak opened this issue Oct 17, 2024 · 4 comments · Fixed by #291
Assignees
@zakkak
Labels
bug Something isn't working

Comments

@zakkak
Copy link
Collaborator

zakkak commented Oct 17, 2024

Since quarkusio/quarkus#43752 the integration tests fail with:

2024-10-14 11:07:57,743 WARNING [io.qua.ope.run.exp.otl.sen.VertxGrpcSender] (vert.x-eventloop-thread-5) Failed to export TraceRequestMarshalers. The request could not be executed. Full error message: Connection refused: localhost/127.0.0.1:4317
2024-10-14 11:07:57,743 SEVERE [io.ope.exp.int.grp.GrpcExporter] (vert.x-eventloop-thread-5) Failed to export spans. The request could not be executed. Error message: Connection refused: localhost/127.0.0.1:4317 [Error Occurred After Shutdown]: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:4317
	Suppressed: java.lang.IllegalStateException: Retries exhausted: 3/3
		at io.smallrye.mutiny.helpers.ExponentialBackoff$1.lambda$apply$0(ExponentialBackoff.java:46)
		at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
		at io.smallrye.mutiny.groups.MultiOnItem.lambda$transformToUni$6(MultiOnItem.java:268)
		at io.smallrye.mutiny.operators.multi.MultiConcatMapOp$MainSubscriber.onItem(MultiConcatMapOp.java:117)
		at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
		at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drainWithDownstream(UnicastProcessor.java:108)
		at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drain(UnicastProcessor.java:139)
		at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.onNext(UnicastProcessor.java:205)
		at io.smallrye.mutiny.operators.multi.processors.SerializedProcessor.onNext(SerializedProcessor.java:104)
		at io.smallrye.mutiny.subscription.SerializedSubscriber.onItem(SerializedSubscriber.java:74)
		at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
		at io.smallrye.mutiny.operators.multi.MultiRetryWhenOp$RetryWhenOperator.onFailure(MultiRetryWhenOp.java:127)
		at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
		at io.smallrye.mutiny.converters.uni.UniToMultiPublisher$UniToMultiSubscription.onFailure(UniToMultiPublisher.java:104)
		at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.dispatch(UniOnFailureFlatMap.java:85)
		at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.onFailure(UniOnFailureFlatMap.java:60)
		at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:60)
		at java.base@21.0.4/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
		at java.base@21.0.4/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
		at java.base@21.0.4/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
		at java.base@21.0.4/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
		at io.vertx.core.Future.lambda$toCompletionStage$3(Future.java:603)
		at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
		at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
		at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
		at io.vertx.core.impl.future.Mapping.onFailure(Mapping.java:45)
		at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
		at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
		at io.vertx.core.http.impl.HttpClientImpl.lambda$doRequest$4(HttpClientImpl.java:398)
		at io.vertx.core.net.impl.pool.Endpoint.lambda$getConnection$0(Endpoint.java:52)
		at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:162)
		at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:123)
		at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:328)
		at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:321)
		at io.vertx.core.net.impl.pool.SimpleConnectionPool$ConnectFailed$1.run(SimpleConnectionPool.java:380)
		at io.vertx.core.net.impl.pool.Task.runNextTasks(Task.java:43)
		at io.vertx.core.net.impl.pool.CombinerExecutor.submit(CombinerExecutor.java:91)
		at io.vertx.core.net.impl.pool.SimpleConnectionPool.execute(SimpleConnectionPool.java:244)
		at io.vertx.core.net.impl.pool.SimpleConnectionPool.lambda$connect$2(SimpleConnectionPool.java:258)
		at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint.lambda$connect$2(SharedClientHttpStreamEndpoint.java:104)
		at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
		at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
		at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
		at io.vertx.core.impl.future.Composition$1.onFailure(Composition.java:66)
		at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
		at io.vertx.core.impl.future.FailedFuture.addListener(FailedFuture.java:98)
		at io.vertx.core.impl.future.Composition.onFailure(Composition.java:55)
		at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
		at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
		at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:328)
		at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:321)
		at io.vertx.core.net.impl.NetClientImpl.failed(NetClientImpl.java:352)
		at io.vertx.core.net.impl.NetClientImpl.lambda$connectInternal2$6(NetClientImpl.java:324)
		at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
		at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
		at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
		at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
		at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
		at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:110)
		at io.vertx.core.net.impl.ChannelProvider.lambda$handleConnect$0(ChannelProvider.java:157)
		at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
		at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
		at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
		at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
		at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
		at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
		at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118)
		at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:326)
		at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:342)
		at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
		at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
		at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
		at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
		at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
		at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
		at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
		at java.base@21.0.4/java.lang.Thread.runWith(Thread.java:1596)
		at java.base@21.0.4/java.lang.Thread.run(Thread.java:1583)
		at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:896)
		at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:872)
	Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:4317
		Suppressed: [CIRCULAR REFERENCE: java.lang.IllegalStateException: Retries exhausted: 3/3]
	Caused by: java.net.ConnectException: Connection refused
		at java.base@21.0.4/sun.nio.ch.Net.pollConnect(Native Method)
		at java.base@21.0.4/sun.nio.ch.Net.pollConnectNow(Net.java:682)
		at java.base@21.0.4/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:973)
		at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:336)
		at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)
		at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
		at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
		at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
		at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
		at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
		at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
		at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
		at java.base@21.0.4/java.lang.Thread.runWith(Thread.java:1596)
		at java.base@21.0.4/java.lang.Thread.run(Thread.java:1583)
		at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:896)
		at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:872)
Caused by: [CIRCULAR REFERENCE: java.net.ConnectException: Connection refused]

This is due to the OpenTelemetry update in quarkusio/quarkus#43752. After this change we see a SEVERE error instead of a WARNING we used to get due to not setting up any OpenTracing collector.
This was referenced Oct 17, 2024
Merged
Merged
zakkak added a commit to zakkak/mandrel-integration-tests that referenced this issue Oct 18, 2024
@zakkak
Start Jaeger to avoid errors related to opentracing/opentelemetry
e9eeb77 
Closes Karm#290
This was referenced Oct 18, 2024
Merged
Closed
@jerboaa
Copy link
Collaborator

jerboaa commented Oct 21, 2024

On Mandrel for JDK 23 integration tests this manifests as:

Error:  Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 295.745 s <<< FAILURE! - in org.graalvm.tests.integration.RuntimesSmokeTest
Error:  quarkusFullMicroProfile{TestInfo}  Time elapsed: 185.388 s  <<< FAILURE!
org.opentest4j.AssertionFailedError: 
build-and-run.log log should not contain error or warning lines that are not whitelisted. See /home/runner/work/mandrel/mandrel/mandrel-integration-tests/testsuite/target/archived-logs/org.graalvm.tests.integration.RuntimesSmokeTest/quarkusFullMicroProfile/build-and-run.log and check these offending lines: 
		at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73) ==> expected: <true> but was: <false>
	at org.graalvm.tests.integration.RuntimesSmokeTest.testRuntime(RuntimesSmokeTest.java:155)
	at org.graalvm.tests.integration.RuntimesSmokeTest.quarkusFullMicroProfile(RuntimesSmokeTest.java:199)

[INFO]

See #292 for details.

This was referenced Oct 21, 2024
Closed
Closed
@jerboaa jerboaa added the bug Something isn't working label Oct 21, 2024
@jerboaa
Copy link
Collaborator

jerboaa commented Oct 29, 2024

Can we allow-list those warnings so as to reduce test noise, perhaps? At least for JDK 23 and JDK 21 based builds this would be helpful.

@zakkak
Copy link
Collaborator Author

zakkak commented Oct 29, 2024

Can we allow-list those warnings so as to reduce test noise, perhaps? At least for JDK 23 and JDK 21 based builds this would be helpful.

IMO we should better merge #291, but I am OK with whitelisting them. Pinged Karm to see if there are anyupdates on #291

@jerboaa
Copy link
Collaborator

jerboaa commented Oct 29, 2024

Can we allow-list those warnings so as to reduce test noise, perhaps? At least for JDK 23 and JDK 21 based builds this would be helpful.

IMO we should better merge #291, but I am OK with whitelisting them. Pinged Karm to see if there are anyupdates on #291

OK. Thanks!

@Karm Karm closed this as completed in #291 Oct 29, 2024
@Karm Karm closed this as completed in 0811c8d Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zakkak zakkak

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Start Jaeger to avoid errors related to opentracing/opentelemetry zakkak/mandrel-integration-tests
2 participants
@jerboaa @zakkak