Description
openedon Sep 14, 2020
Kibana version:
7.9 (tested, this likely affects many older versions)
Describe the bug:
When Kibana has HTTPS enabled, but a client tries to connect over HTTP, the client gets redirected to HTTPS as expected an empty response as expected*. However, ugly error messages show up in the logs:
server error [13:44:35.450] [error][client][connection] Error: 4388156864:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:
We have code that is supposed to downgrade these error messages to the debug log level:
kibana/src/legacy/server/logging/log_interceptor.js
Lines 126 to 128 in 003fcb1
However, that code is broken. It was originally introduced in Kibana 6.0 (#11209), back when we were using Node 6.10.2 and Hapi 14.2.0. Now, we are using Node 10.22.0 (which includes bundled OpenSSL 1.1.1) and Hapi 17.6.0.
Steps to reproduce:
- Start Kibana with SSL enabled (
server.ssl.enabled: true
) - Connect to Kibana using a client via HTTP
- Observe the aforementioned error messages in the Kibana logs
Expected behavior:
These error messages should be downgraded to the debug log level. The error message has simply changed at some point since this log interceptor was originally added.
Any additional context:
I added some additional logging to the log interceptor to verify:
index 07545570445..644f18afba5 100644
--- a/src/legacy/server/logging/log_interceptor.js
+++ b/src/legacy/server/logging/log_interceptor.js
@@ -58,6 +58,15 @@ function downgradeIfErrorMessage(match, event) {
const errorMessage = get(event, 'error.message');
const matchesErrorMessage = isClientError && doesMessageMatch(errorMessage, match);
+ if (isClientError && !matchesErrorMessage && match === GET_CLIENT_HELLO) {
+ const { error } = event;
+ console.log(
+ `Not downgraded (match ${match.toString()}): ${JSON.stringify(
+ error,
+ Object.getOwnPropertyNames(error)
+ )}`
+ );
+ }
if (!matchesErrorMessage) return null;
return {
Resulting logs show this:
Not downgraded (match /GET_CLIENT_HELLO:http/): {"stack":"Error: 4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n","message":"4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n"}
server error [14:52:58.712] [error][client][connection] Error: 4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:
So, it looks like this is an easy fix, but unfortunately we'll still have to resort to error message matching.
Note: tagging Platform as they are technically the code owner, but also Security because I'm about to submit a PR to fix it 🙂
*Update: redirect does not happen / has nothing to do with it, that was just my browser being helpful.