custom_fields_by_lua not being parsed properly

[answerquest]

Hello, thanks for this program. That this is pure-frontend interface to kong gateway itself is quite nice.

King seems to be having a similar custom_fields_by_lua problem as Konga does, see for reference: pantsel/konga#809

Screenshot:

Looking into the api call made to Kong by King, I can see where the problem lies: here's the raw payload:

{"name":"file-log","instance_name":"filelog1","route":null,"service":null,"consumer":null,"protocols":["http","https"],"enabled":true,"tags":[],"config":{"path":"/data/hello.txt","reopen":true,"custom_fields_by_lua":{"{ \"request":{"headers.postman-token\"":" \"return nil\", \"authenticated_entity\""}}}}

Seeing the config part:

"config": {
    "path": "/data/hello.txt",
    "reopen": true,
    "custom_fields_by_lua": {
      "{ \"request": {
        "headers.postman-token\"": " \"return nil\", \"authenticated_entity\""
      }
    }
  }

I guess custom_fields_by_lua is not being processed properly.

This is what I entered into the field :

{ "request.headers.postman-token": "return nil", "authenticated_entity": "return nil" }

This is what the api call to Kong needs to have:

"config": {
    "path": "/data/hello.txt",
    "reopen": true,
    "custom_fields_by_lua": {
          "request.headers.postman-token": "return nil",
          "authenticated_entity": "return nil"
    }
  }

Basically the json we enter needs to faithfully make its way through without mutilation.

Maybe there's some script trying to parse things in a different way, but that doesn't make sense to me : Sticking to the format that the original Kong gateway expects seems to be the best way forward IMHO.

[ligreman]

Hi @answerquest

I'll take a look at it, the problem is that the interface (HTML) parses that JSON as a string (it does that automatically, I do not control it). And sometimes thats the way to go, but other times not. So I must check that other plugin fields of that type do not get broken with this change.

I think I've faced this problem before... but I don't remember the exact format to use in this fields. The hint I used "Format to use -> field.subfield:valor" is not very good to understand the format, I must improve it :P

Thanks!

[answerquest]

Looking at how "request.headers.postman-token" is being changed into "{ \"request": {"headers.postman-token\"", I reckon there's some advanced functionality that's being mis-applied here. And it's breaking the data up in a way that we can't really put it back together later.

One way out would be: change the field type to a simple text one same as other simple text fields; see what happens. There, even if the json gets stringified, there's hope for injecting an action before api send that parses the field back into json.

[ligreman]

@answerquest I've reworked the "map" fields in plugin configuration, and just treated them as JSON objects. Previously they were parsed and validated as arrays of strings and converted in a weird way to JSON format.

Now those fields are just a string, then they are JSON.parsed and sent to the API. So, now you must provide a valid JSON for them. I've tested them with many plugins (file_log, http_log, loggly, opentelemetry, syslog, tcp_log and udp_log) and I think it is fixed, but please if you find any weird behaviour or error, open an issue.