Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core - Add build information to all requests sent by HTTPClient as headers #5238

Merged

Conversation

kbendick
Copy link
Contributor

@kbendick kbendick commented Jul 11, 2022

The HTTPClientFactory is used to build the HTTP Client that will be used by the REST Catalog.

This PR injects the following two headers into each request, for debugging purposes:

  • X-Client-Version
  • X-Client-Git-Commit-Short

To test for these headers, TestHTTPClient was updated to use the HTTPClientFactory. The headers, with their expected values, are placed on each expected request, which is an assertion.

Notice that we don't have to add the headers onto the request that is sent, as they are default headers for each request.

This PR needs relies on work in #5237 and needs to be rebased once that is merged.

@kbendick
Copy link
Contributor Author

kbendick commented Jul 11, 2022

Here's a sample of the headers sent for each request, per the mock server tests:

Relevant portions:
X-Client-Git-Commit-Short: "a850705" X-Client-Version: ""Apache Iceberg 0.14.0-SNAPSHOT (commit a850705)"`

We can update these header names or which values are used (as well as where they're placed and how, but I think HTTPClientFactory makes sense).

Probably using the short version of 0.14.0-SNAPSHOT would be just as sufficient for debugging while reducing unnecessary header size. But I'll let others chime in.

The headers can technically be overridden via the config header.X-Client-Git-Commit-Short in the configuration, although we can code against that (make it a reserved property) if desired.

   "headers" : {
      "Accept" : [ "application/json" ],
      "Content-Type" : [ "application/json" ],
      "X-Client-Git-Commit-Short" : [ "a850705" ],
      "X-Client-Version" : [ "Apache Iceberg 0.14.0-SNAPSHOT (commit a8507057f778efba6f1750ae814749f52a4cd7bc)" ],
      "Authorization" : [ "Bearer auth_token" ],
      "Accept-Encoding" : [ "gzip, x-gzip, deflate" ],
      "Host" : [ "127.0.0.1:1080" ],
      "Connection" : [ "keep-alive" ],
      "User-Agent" : [ "Apache-HttpClient/5.1 (Java/11.0.15)" ],
      "content-length" : [ "0" ]
    }

Comment on lines 120 to 121
String.format("header.%s", HTTPClientFactory.CLIENT_GIT_COMMIT_SHORT_HEADER), IcebergBuild.gitCommitShortId(),
String.format("header.%s", HTTPClientFactory.CLIENT_VERSION_HEADER), IcebergBuild.version()));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't really need to be set because HTTPClientFactory isn't used here - the underlying JDBC client is what builds this.

We can however still override the version etc by sending back a configuration property of header.X-Client-Version with the desired value to be used as the value (though I don't see any reason to do that).

@kbendick kbendick changed the title Core - Add build information to all requests sent by HTTPClient Core - Add build information to all requests sent by HTTPClient as headers Jul 11, 2022
Comment on lines +176 to +177
.withHeader(HTTPClientFactory.CLIENT_VERSION_HEADER, icebergBuildFullVersion)
.withHeader(HTTPClientFactory.CLIENT_GIT_COMMIT_SHORT_HEADER, icebergBuildGitCommitShort);
Copy link
Contributor Author

@kbendick kbendick Jul 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We add the headers that we expect to see as assertions on the mock request. The actual HTTPClient is used, generated by the HTTPClientFactory, which injects the headers.

If requests are not generated which have these headers, they will not match and the assertion of this mock request will fail.

Notice that we didn't have to add the request headers anywhere else (like we do with Authorization elsewhere in this test suite when generating the actual requests on the updated line 205). That signifies that the headers are added automatically if the HTTPClient is created via HTTPClientFactory.

@kbendick kbendick marked this pull request as ready for review July 11, 2022 08:22
@kbendick kbendick force-pushed the kb-configure-http-client-build-info-headers branch from b90e54b to 55ff696 Compare July 11, 2022 16:21
@github-actions github-actions bot removed the API label Jul 11, 2022
@rdblue rdblue merged commit 3754cbd into apache:master Jul 11, 2022
@rdblue
Copy link
Contributor

rdblue commented Jul 11, 2022

Thanks, @kbendick!

@kbendick kbendick deleted the kb-configure-http-client-build-info-headers branch July 11, 2022 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants