Structured Access Logs

[josephearl]

Description

As an SRE
I would like the access logs to emit structured information when using quarkus-logging-json
So that I can filter logs more efficiently

At the moment when using Quarkus access logging with something like Splunk or Kibana, important information like response status code, request URI, response size etc is only available in the message

These should be available as separate JSON properties on the log event so that I can easily filter the logs

Implementation ideas

logback-access does this well when combined with logstash-logback-encoder and emits something along the lines of:

{
  "@timestamp": "2021-05-04T18:24:25.720+01:00",
  "status_code": 404,
  "method": "GET",
  "protocol": "HTTP/1.1",
  "requested_uri": "/api/v1/resource/12345678",
  "remote_host": "127.0.0.1",
  "content_length": -1,
  "elapsed_time": 76,
  "level": "DEBUG",
  "logger_name": "access.log",
  "message": "127.0.0.1 - - [2021-05-04T18:24:25.720+0100] \"GET /api/v1/resource/12345678 HTTP/1.1\" 404 -1",
}

[mkouba]

Have you tried the custom log handler from the io.quarkiverse.loggingjson:quarkus-logging-json?

[gsmet]

I'm not sure we want to include this feature in the core extension given we have io.quarkiverse.loggingjson:quarkus-logging-json but if somebody is up to it and it's a simple patch, we can include it.

[josephearl]

@mkouba i have not tried using that to format the access logs, can give it a go and see if it works

[xsajkha]

I have enabled the access logs and I also fixed the log pattern so that we can get the info in separate fields in the json payload:
`
#Access logs config

quarkus.http.access-log.enabled=true

quarkus.http.record-request-start-time=true

quarkus.http.access-log.pattern={ "client":"%{REMOTE_HOST}","method": "%{METHOD}","request_url": %{REQUEST_URL}. "status":"%{RESPONSE_CODE}" , "duration":"%{RESPONSE_TIME}", "bytes":"%{BYTES_SENT}" }

quarkus.http.access-log.log-to-file=true

quarkus.http.access-log.base-file-name=access

quarkus.http.access-log.log-directory=/tmp
`
This will generate output like:

{ "client":"xxx.xxx.xxx","method": "GET","request_url": /auth/resources/gw46h/login/keycloak/css/login.css. "status":"200" , "duration":"7", "bytes":"0" }

[josephearl]

Thanks @xsajkha will give that a go