Adding a "field set" or some other identifer for JSON formatted log lines

[VimCommando]

Related to:

1. Log format not detected when piped to stdin #729
2. feat(json): multiple line-format in json logs #1065

Can I suggest a JSON format option like field-set to help differentiate different line formats within the same JSON file (or no file at all from stdin).

The way I imagine this working is putting this in the same priority as the current regex filters. As I understand it when you open a file with lnav it:

1. Decompresses the file (if needed)
2. Checks the filename against the file-pattern
3. (if not JSON) Evaluates lines against the regex patterns to find the most specific match

For JSON files, a field-set would be an array of expected property names. These would follow the same field and value conventions of being text or JSON pointers. An example format might look like this:

{
    "$schema": "https://lnav.org/schemas/format-v1.schema.json",
    "ecs-log": {
        "title": "Elastic Common Schema",
        "url": "https://github.com/elastic/ecs",
        "description": "A generic ECS log format",
        "file-pattern": ".*json",
        // New proposed selector
        "field-set": [
            "ecs/version",
            "@timestamp",
            "log/level",
            "message"
        ],
        "multiline": false,
        "json": true,
        "hide-extra": true,
        "body-field": "message",
        "timestamp-field": "@timestamp",
        "level-field": "log/level",
        "log.level": {
            "error": "ERROR",
            "debug": "DEBUG",
            "warning": "WARN",
            "info": "INFO",
            "critical": "CRIT",
            "fatal": "FATAL"
        },
        "line-format": [
            { "field": "__timestamp__" }, " ",
            { "field": "log/level", "min-width": 6, "text-transform": "uppercase" },
            "[", { "field": "log/logger", "default-value": " " }, "] ",
            { "field": "message" }, " ",
            { "field": "error/stack_trace", "default-value": "" }
        ],
        "value": {
            "log/logger": { "kind": "string", "identifier": true },
            "error/stack_trace": { "kind": "json" }
        },
        "sample": [
            {
                "line": "{\"ecs\":{\"version\":\"8.0.0\"},\"@timestamp\":\"2022-07-05T17:29:35.054+00:00\",\"message\":\"Session is no longer available and cannot be re-authenticated.\",\"log\":{\"level\":\"WARN\",\"logger\":\"plugins.security.authenticator\"},\"process\":{\"pid\":18},\"span\":{\"id\":\"8d108c2937c874b9\"},\"trace\":{\"id\":\"1914226e1716c8693184d71a6a0f4336\"}}",
                "level": "warning"
            }
        ]
    }
}

And if I need a different set of fields rendered, because there are different properites in the JSON, I can use a different field-set:

// Alternate selector for a different line format
{
    "ecs-http": {
        "description": "An http-based ECS log format",
        "field-set": [
            "ecs/version",
            "@timestamp",
            "http/request",
            "http/response",
            "message"
        ],
        // ...
    }
}

This way if I have intermixed logs in a single JSON stream I have a way to distinguish them. This could also solve the limitation of not being able to format JSON streamed to stdin.