Skip to content

Error logs when clients connect over HTTP instead of HTTPS #77391

Closed

Description

Kibana version:

7.9 (tested, this likely affects many older versions)

Describe the bug:

When Kibana has HTTPS enabled, but a client tries to connect over HTTP, the client gets redirected to HTTPS as expected an empty response as expected*. However, ugly error messages show up in the logs:

server   error  [13:44:35.450] [error][client][connection] Error: 4388156864:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:

We have code that is supposed to downgrade these error messages to the debug log level:

downgradeIfHTTPWhenHTTPS(event) {
return downgradeIfErrorMessage(GET_CLIENT_HELLO, event);
}

However, that code is broken. It was originally introduced in Kibana 6.0 (#11209), back when we were using Node 6.10.2 and Hapi 14.2.0. Now, we are using Node 10.22.0 (which includes bundled OpenSSL 1.1.1) and Hapi 17.6.0.

Steps to reproduce:

  1. Start Kibana with SSL enabled (server.ssl.enabled: true)
  2. Connect to Kibana using a client via HTTP
  3. Observe the aforementioned error messages in the Kibana logs

Expected behavior:

These error messages should be downgraded to the debug log level. The error message has simply changed at some point since this log interceptor was originally added.

Any additional context:

I added some additional logging to the log interceptor to verify:

index 07545570445..644f18afba5 100644
--- a/src/legacy/server/logging/log_interceptor.js
+++ b/src/legacy/server/logging/log_interceptor.js
@@ -58,6 +58,15 @@ function downgradeIfErrorMessage(match, event) {
   const errorMessage = get(event, 'error.message');
   const matchesErrorMessage = isClientError && doesMessageMatch(errorMessage, match);
 
+  if (isClientError && !matchesErrorMessage && match === GET_CLIENT_HELLO) {
+    const { error } = event;
+    console.log(
+      `Not downgraded (match ${match.toString()}): ${JSON.stringify(
+        error,
+        Object.getOwnPropertyNames(error)
+      )}`
+    );
+  }
   if (!matchesErrorMessage) return null;
 
   return {

Resulting logs show this:

Not downgraded (match /GET_CLIENT_HELLO:http/): {"stack":"Error: 4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n","message":"4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n"}
server   error  [14:52:58.712] [error][client][connection] Error: 4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:

So, it looks like this is an easy fix, but unfortunately we'll still have to resort to error message matching.


Note: tagging Platform as they are technically the code owner, but also Security because I'm about to submit a PR to fix it 🙂

*Update: redirect does not happen / has nothing to do with it, that was just my browser being helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

Team:CoreCore services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etcTeam:SecurityTeam focused on: Auth, Users, Roles, Spaces, Audit Logging, and more!bugFixes for quality problems that affect the customer experience

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions