chore: Upgrade deps #103

findingrish · 2021-08-31T13:19:25Z

No description provided.

findingrish · 2021-08-31T13:26:37Z

Ran into following exception in data-query-service while trying out k8s deployment

com.google.common.util.concurrent.UncheckedExecutionException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Credentials should use fail() instead of throwing exceptions
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946) ~[guava-30.1.1-jre.jar:?]
	at org.hypertrace.gateway.service.common.AttributeMetadataProvider.getAttributeMetadata(AttributeMetadataProvider.java:117) ~[gateway-service-impl.jar:?]
	at org.hypertrace.gateway.service.common.util.AttributeMetadataUtil.getTimestampAttributeMetadata(AttributeMetadataUtil.java:65) ~[gateway-service-impl.jar:?]
	at org.hypertrace.gateway.service.common.util.AttributeMetadataUtil.getTimestampAttributeId(AttributeMetadataUtil.java:73) ~[gateway-service-impl.jar:?]
	at org.hypertrace.gateway.service.entity.EntityService.getEntities(EntityService.java:130) ~[gateway-service-impl.jar:?]
	at org.hypertrace.gateway.service.GatewayServiceImpl.getEntities(GatewayServiceImpl.java:216) [gateway-service-impl.jar:?]
	at org.hypertrace.gateway.service.GatewayServiceGrpc$MethodHandlers.invoke(GatewayServiceGrpc.java:618) [gateway-service-api.jar:?]
	at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) [grpc-stub-1.40.0.jar:1.40.0]
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) [grpc-api-1.40.0.jar:1.40.0]
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) [grpc-api-1.40.0.jar:1.40.0]
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40) [grpc-api-1.40.0.jar:1.40.0]
	at io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86) [grpc-api-1.40.0.jar:1.40.0]
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331) [grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:866) [grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) [grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) [grpc-core-1.40.0.jar:1.40.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Credentials should use fail() instead of throwing exceptions
	at io.grpc.Status.asRuntimeException(Status.java:535) ~[grpc-api-1.40.0.jar:1.40.0]
	at io.grpc.stub.ClientCalls$BlockingResponseStream.hasNext(ClientCalls.java:648) ~[grpc-stub-1.40.0.jar:1.40.0]
	at org.hypertrace.gateway.service.common.AttributeMetadataProvider$2.load(AttributeMetadataProvider.java:71) ~[gateway-service-impl.jar:?]
	at org.hypertrace.gateway.service.common.AttributeMetadataProvider$2.load(AttributeMetadataProvider.java:59) ~[gateway-service-impl.jar:?]
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1.1-jre.jar:?]
	... 21 more
Caused by: java.lang.AbstractMethodError: Receiver class io.grpc.netty.NettyClientTransport does not define or inherit an implementation of the resolved method 'abstract io.grpc.internal.ClientStream newStream(io.grpc.MethodDescriptor, io.grpc.Metadata, io.grpc.CallOptions, io.grpc.ClientStreamTracer[])' of interface io.grpc.internal.ClientTransport.
	at io.grpc.internal.MetadataApplierImpl.apply(MetadataApplierImpl.java:74) ~[grpc-core-1.40.0.jar:1.40.0]
	at org.hypertrace.core.grpcutils.client.RequestContextAsCreds.applyRequestContext(RequestContextAsCreds.java:44) ~[grpc-client-utils-0.6.1.jar:?]
	at org.hypertrace.core.grpcutils.client.ContextKeyBasedCreds.applyRequestMetadata(ContextKeyBasedCreds.java:19) ~[grpc-client-utils-0.6.1.jar:?]
	at io.grpc.internal.CallCredentialsApplyingTransportFactory$CallCredentialsApplyingTransport.newStream(CallCredentialsApplyingTransportFactory.java:147) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.ForwardingConnectionClientTransport.newStream(ForwardingConnectionClientTransport.java:51) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.InternalSubchannel$CallTracingTransport.newStream(InternalSubchannel.java:673) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.ManagedChannelImpl$ChannelStreamProvider$1RetryStream.newSubstream(ManagedChannelImpl.java:584) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.RetriableStream.createSubstream(RetriableStream.java:234) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.RetriableStream.start(RetriableStream.java:366) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.ClientCallImpl.startInternal(ClientCallImpl.java:287) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:189) ~[grpc-core-1.40.0.jar:1.40.0]
	at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:332) ~[grpc-stub-1.40.0.jar:1.40.0]
	at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:306) ~[grpc-stub-1.40.0.jar:1.40.0]
	at io.grpc.stub.ClientCalls.blockingServerStreamingCall(ClientCalls.java:204) ~[grpc-stub-1.40.0.jar:1.40.0]
	at org.hypertrace.core.attribute.service.v1.AttributeServiceGrpc$AttributeServiceBlockingStub.findAttributes(AttributeServiceGrpc.java:509) ~[attribute-service-api.jar:?]
	at org.hypertrace.core.attribute.service.client.AttributeServiceClient.lambda$findAttributes$8(AttributeServiceClient.java:80) ~[attribute-service-client.jar:?]
	at io.grpc.Context.call(Context.java:564) ~[grpc-context-1.40.0.jar:1.40.0]
	at org.hypertrace.core.grpcutils.context.RequestContext.call(RequestContext.java:77) ~[grpc-context-utils-0.6.1.jar:?]
	at org.hypertrace.core.grpcutils.client.GrpcClientRequestContextUtil.executeWithHeadersContext(GrpcClientRequestContextUtil.java:42) ~[grpc-client-utils-0.6.1.jar:?]
	at org.hypertrace.core.attribute.service.client.AttributeServiceClient.execute(AttributeServiceClient.java:62) ~[attribute-service-client.jar:?]
	at org.hypertrace.core.attribute.service.client.AttributeServiceClient.findAttributes(AttributeServiceClient.java:80) ~[attribute-service-client.jar:?]
	at org.hypertrace.gateway.service.common.AttributeMetadataProvider$2.load(AttributeMetadataProvider.java:64) ~[gateway-service-impl.jar:?]
	at org.hypertrace.gateway.service.common.AttributeMetadataProvider$2.load(AttributeMetadataProvider.java:59) ~[gateway-service-impl.jar:?]
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1.1-jre.jar:?]
	... 21 more

Trying to see if upgrading deps could help (which anyways need upgrade)

aaron-steinfeld

I expect it to fix it. That error reads to me like we're running grpc 1.40 (which we are) over an older grpc-netty (which we were).

aaron-steinfeld · 2021-08-31T13:34:46Z

hypertrace-service/build.gradle.kts

  // Config
  implementation("com.typesafe:config:1.4.1")
  // Logging
  implementation("org.slf4j:slf4j-api:1.7.30")
  runtimeOnly("org.apache.logging.log4j:log4j-slf4j-impl:2.14.1")

  // GRPC
-  runtimeOnly("io.grpc:grpc-netty:1.36.1")
+  runtimeOnly("io.grpc:grpc-netty:1.40.0")


that said, I would have expected this to show up in the docker compose flavor too.

yeah, so not having high hopes from this change, but wanted to give it a try.

For me, with the existing hypertrace-service image, docker-compose is running fine.

I think it should still work - I just noticed you said that it was happening in data query service, not in hypertrace service. Because the wrapper services are pulling in all the containers (a mistake, IMO), of the services they're depending on, I suspect that the individual services going into data query are not bringing in grpc-netty 1.40, but one of the ones coming in here is.

I suspect that the individual services going into data query are not bringing in grpc-netty 1.40

Yeah, this turned out to be the reason. grpc-netty 1.39 was being used.

Because the wrapper services are pulling in all the containers (a mistake, IMO)

The container dependency would be automatically pulled in with other dependencies?

we're bringing our own container and stuffing a number of impls in it. So we should be depending on
something like implementation("org.hypertrace.config.service:config-service-impl") only rather than implementation("org.hypertrace.config.service:config-service") - right now we depend on both (not sure if the code is currently written in such a way to require that, or this can be an easy fix).

The downside to doing is what we just saw - we're getting container runtime dependencies like jetty, grpc-netty etc. from each child service, throwing them into a big pile and then doing conflict resolution across all of them - makes it easy to lead to mismatches.

not sure if the code is currently written in such a way to require that, or this can be an easy fix

Filed #104, will check

rish691 added 3 commits August 31, 2021 18:48

Upgrade deps for data-config service

9cbf4ef

Upgrade deps for data-query service

acd6b8d

Upgrade deps for hypertrace-service

ac4e8df

findingrish requested review from kotharironak, laxmanchekka and ravisingal as code owners August 31, 2021 13:19

findingrish requested a review from aaron-steinfeld August 31, 2021 13:24

aaron-steinfeld approved these changes Aug 31, 2021

View reviewed changes

aaron-steinfeld reviewed Aug 31, 2021

View reviewed changes

findingrish merged commit f86179c into main Aug 31, 2021

findingrish deleted the upgrade-deps branch August 31, 2021 13:35

findingrish mentioned this pull request Sep 1, 2021

Wrapper services should depend on implementation of child services #104

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: Upgrade deps #103

chore: Upgrade deps #103

Uh oh!

findingrish commented Aug 31, 2021

Uh oh!

findingrish commented Aug 31, 2021

Uh oh!

aaron-steinfeld left a comment

Uh oh!

aaron-steinfeld Aug 31, 2021

Uh oh!

findingrish Aug 31, 2021 •

edited

Loading

Uh oh!

kotharironak Aug 31, 2021

Uh oh!

aaron-steinfeld Aug 31, 2021

Uh oh!

findingrish Aug 31, 2021 •

edited

Loading

Uh oh!

aaron-steinfeld Aug 31, 2021

Uh oh!

findingrish Sep 1, 2021

Uh oh!

Uh oh!

chore: Upgrade deps #103

chore: Upgrade deps #103

Uh oh!

Conversation

findingrish commented Aug 31, 2021

Uh oh!

findingrish commented Aug 31, 2021

Uh oh!

aaron-steinfeld left a comment

Choose a reason for hiding this comment

Uh oh!

aaron-steinfeld Aug 31, 2021

Choose a reason for hiding this comment

Uh oh!

findingrish Aug 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kotharironak Aug 31, 2021

Choose a reason for hiding this comment

Uh oh!

aaron-steinfeld Aug 31, 2021

Choose a reason for hiding this comment

Uh oh!

findingrish Aug 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aaron-steinfeld Aug 31, 2021

Choose a reason for hiding this comment

Uh oh!

findingrish Sep 1, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

findingrish Aug 31, 2021 •

edited

Loading

findingrish Aug 31, 2021 •

edited

Loading