Compose ECS objects vs reuse objects

[ruflin]

In several open issues / PRs (for example #51) the host objects are reused in different places. To reuse objects we can either repeat the name of the object or we could use composition. Examples:

Composition: Field c is composed out of host and geo, means it contains all the fields in host and geo. Example:

c.ip
c.mac
c.location

Reusing the object:

c.host.ip
c.host.mac
c.geo.location

Composition has the advantage that the names are shorter but it's not directly know out of what c is composed, meaning which fields it contains. Reusing the objects reduces the number of objects in ECS that look different. c could have a field d that is specific to c and still have all host and and geo fields. As soon as geo or host are used in any place it is clear which fields it will contain. This should simplify the mental model around ECS. To handle the longer field names with reusing objects it would be nice if Kibana would "understand" ECS and could potentially add some magic shortening here.

Most important is that we are consistent across ECS and use one or the other and don't mix them as we do at the moment.

[MikePaquette]

@ruflin ECS how would this composition be implemented? My first reaction is that the composition strings are actually additional things to remember, and might make ECS harder to understand.

[ruflin]

We already do a bit of composition at the moment for example in source.* and destination.* and use the host fields. From an "implementation" perspective using composition could be as simple as defining the following in yaml:

- name: source
  composed_of: geo, host

Then the generator would do all the magic.

My bigger concern is how we make that understandable and visible to the user. This is where I think the reusing of objects becomes nice. We can mention where we recommend to use the host object but we don't have to repeat all the fields every time. And users adding non ECS fields on the top level can inside still reuse for example host.

[webmat]

I would go for simplicity and have the reused objects present and visible, so I would not go for composition, personally.

[ruflin]

If we make ip also a top level field like message, would this solve the issue of having too many nesting? This means for the above it could be c.ip or c.host.ip, meaning host.ip is just reusing the field.

[vbohata]

I think composing to c.ip, ... could lead to a lower number of reusable objects. Yes, for this use case it is clear and for the first point of view nice but I vote for the longer names. Everyone can see here from which parts it is composed from.

[vbohata]

New example. For some logs from the switch I have following fields:

• source.host.mac
• client.host.mac
• switch.port.number

Where source.host.mac is the source mac of the traffic, client.host.mac is mac stored in switch, switch.port.number is port of the switch.

So by assigning ip, mac, port, ... under host, it is clear these values are related to the host interface which initialised the connection, switch.port.* is related to physical port of switch (e.g. switch.port.number, switch.port.link_status). So "port" can not be top level field because once it has numeric value (host usage), once it is object.
Also switch itself can have management IP, so if someone would need to mix it to one event, there would be switch.host.ip, switch.host.port, switch.port ...

[ruflin]

Based on the decision that we have reusable objects like os we are reusing some of the objects and define the ones that can be reused. Closing.