Skip to content

Commit

Permalink
[improve] AuthenticationProviderOpenID k8s error logs (apache#20135)
Browse files Browse the repository at this point in the history
### Motivation

The `AuthenticationProviderOpenID` error logs from the Kubernetes client are not very helpful in certain cases because we only get the error's message and not the error's response body. See kubernetes-client/java#2066 for details on the solution.

Here is an example of a problematic error:

```
org.apache.pulsar.broker.authentication.AuthenticationProviderList - Authentication failed for auth provider class org.apache.pulsar.broker.authentication.oidc.AuthenticationProviderOpenID:
javax.naming.AuthenticationException: Error retrieving OpenID Provider Metadata from Kubernetes API server:
	at org.apache.pulsar.broker.authentication.oidc.OpenIDProviderMetadataCache$1.onFailure(OpenIDProviderMetadataCache.java:174) ~[org.apache.pulsar-pulsar-broker-auth-oidc-3.0.0.jar:3.0.0]
	at io.kubernetes.client.openapi.ApiClient$1.onResponse(ApiClient.java:927) ~[io.kubernetes-client-java-api-17.0.2.jar:?]
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519) ~[com.squareup.okhttp3-okhttp-4.9.3.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
```

When I enable debug logging out of the API Client, I can see:

```
INFO: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:serviceaccount:michael-test:superuser\" cannot get path \"/.well-known/openid-configuration/\"","reason":"Forbidden","details":{},"code":403}

Apr 19, 2023 2:50:25 AM okhttp3.internal.platform.Platform log
INFO: <-- END HTTP (246-byte body)
2023-04-19T02:50:25,832+0000 [pulsar-web-40-1] DEBUG 
```

(Note: the solution to this problem is to update the `system:service-account-issuer-discovery` `ClusterRole` to include endpoints with trailing slashes. I created kubernetes/kubernetes#117455 to help solve the permission problem in kubernetes.)

### Modifications

* Use both the message and the response body when converting a Kubernetes client error into a Pulsar Authentication error.

### Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

### Documentation

- [x] `doc-not-needed`

### Matching PR in forked repository

PR in forked repository: no need for a forked PR
  • Loading branch information
michaeljmarshall authored Apr 19, 2023
1 parent fc17c1d commit c9c99aa
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,10 @@ private CompletableFuture<List<Jwk>> getJwksFromKubernetesApiServer() {
@Override
public void onFailure(ApiException e, int statusCode, Map<String, List<String>> responseHeaders) {
incrementFailureMetric(AuthenticationExceptionCode.ERROR_RETRIEVING_PUBLIC_KEY);
// We want the message and responseBody here: https://github.com/kubernetes-client/java/issues/2066.
future.completeExceptionally(
new AuthenticationException("Failed to retrieve public key from Kubernetes API server: "
+ e.getMessage()));
new AuthenticationException("Failed to retrieve public key from Kubernetes API server. "
+ "Message: " + e.getMessage() + " Response body: " + e.getResponseBody()));
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,9 +165,10 @@ private CompletableFuture<OpenIDProviderMetadata> loadOpenIDProviderMetadataForK
@Override
public void onFailure(ApiException e, int statusCode, Map<String, List<String>> responseHeaders) {
incrementFailureMetric(AuthenticationExceptionCode.ERROR_RETRIEVING_PROVIDER_METADATA);
// We want the message and responseBody here: https://github.com/kubernetes-client/java/issues/2066.
future.completeExceptionally(new AuthenticationException(
"Error retrieving OpenID Provider Metadata from Kubernetes API server: "
+ e.getMessage()));
"Error retrieving OpenID Provider Metadata from Kubernetes API server. Message: "
+ e.getMessage() + " Response body: " + e.getResponseBody()));
}

@Override
Expand Down

0 comments on commit c9c99aa

Please sign in to comment.