For boolean values, should we recommend using tags or a boolean field?

[webmat]

Boolean fields are meant to represent a binary value, but the absence of the field can add a third state to consider, making the field actually ternary (true, false, absent).

Alternately we could represent this binary value by the presence or absence of a specific tag in tags.

Of course tags is largely meant to let people add free form tags to their event stream, without needing them to be documented formally. But that doesn't prevent us from documenting some commonly tags, just like we're documenting field names & types.

This idea came up in elastic/beats#7991, discussing whether we should have a boolean field log.truncated or a tag truncated.

If we think using tags as "true" binary booleans should be the preferred approach:

• we could document this convention in the "ECS Conventions" of the readme
• and add a list of "official" tags we suggest people consider using.

At the time of writing this issue, there are no boolean fields in ECS.

[ruflin]

I would delay this discussion until we have actual boolean values. In general I see the tags to be used by our users and should not be polluted by ECS to prevent any conflicts.

[webmat]

Agreed. I created the "deferred" label, and gave it a boring color so we stop noticing it :-)

[urso]

@ruflin @webmat We've got 2 PRs blocked. Let's not defer it for too long please.

The proposal for tags was not to use the tags field, but a custom field that acts like tags. In the use-case given we could introduce a log.status or log.flags fields, with type []string.

Assuming reader pipeline in beats uses fields we always have to generate this:

{
   ...
  "log": {
    "truncated": false,
    "multiline": false,
  },
}

Using tags the event would become:

{
   ...
  "log": {
    "status": [],
  },
}

The event does not exist yet in filebeat, as we're still processing the input.

Having tags we're somewhat flexible, as we only need to define/add tags locally + document the tag only.

Using fields we would have to pass a 'state'-struct, transform the struct the event fields, update fields.yml + documentation.

My impression is that using tags would be somewhat more flexible and require less places/files to be updated (less things to forget).

[ruflin]

We have one similar example in ECS which is network.direction: https://github.com/elastic/ecs#-network-fields It sounds like we are more talking about an enum type then actually tags even though from the Elasticsearch perspective it's the same.