Improve semantics of network.direction

[dcode]

network.direction is currently defined ambiguously. The semantics depend on the source of the data making it very difficult to analyze.

We need a specifier that indicates whether the record is using host-level semantics or network-level semantics. Namely, if an event is marked as outbound, from a detection perspective, we need to know if that means the traffic is leaving our internal trust boundary to the Internet, or if it simply means that we're sending packets out of the local network interface to perhaps a local server without our internal trust boundary.

[ebeahan]

@dcode Agree from a detection perspective identifying if the flow is trust-to-untrust, trust-to-trust, untrust-to-untrust is indeed useful. Do you see a gap in the current schema beyond the existing network.direction fields? Or would this focus on how ECS is implemented and how those directions are determined?

[dcode]

We had a discussion on March 27th that fell through the cracks. A proposed way forward was to simply add "ingress" and "egress" to the existing network.direction field, which would indicate perspective of traffic entering or leaving a network interface, respectively. This captures the semantics used today by auditbeat, metricbeat, packetbeat, and winlogbeat (from sysmon) for network traffic observed from an endpoint host where global network boundaries may not be known.

On the other hand, existing semantics of "inbound" and "outbound" would capture the semantics of traffic crossing a network boundary. This would be used when packetbeat is run as an observer via a tap or span port. It also aligns with semantics used by Zeek.

Related discussion: elastic/beats#11147

[webmat]

So the idea is to define these two new values and clarify each pair's role:

• ingress/egress: host or endpoint POV
• inbound/outbound: network perimeter POV, based on specific configuration by admins on which networks are internal and which are external

I think this is simple enough, if we do not try to mix this with trust levels or other ways of classifying network exchanges.

[dainperkins]

On the field value side I'm actually starting to think @rw-access had good points on terminology...

ingress/egress seems more aligned to a concept of entering/exiting something that you would eventually exit (router, switch, highway, etc) where inbound/outbound seem to define a state of reaching/leaving a particular destination (thanks merriam-webster...)

That being said I think the definition matters more:

network.direction:

• provided only by local host reporting (packetbeat*, log ingest, socket analysis, etc.)
• literal direction of traffic from a host's local frame of reference (as a client / source / server / destination)
• e.g. TCP: host syn = outbound/egress, host syn-ack = inbound/ingress
• e.g. UDP: dns request sent = outbound/egress, dns request received: inbound/ingress
• e.g. UDP: dns response sent = outbound/egress, dns response received: inbound/ingress
• does not include logical / organizational qualifiers (internal/external, zones, etc.)
• easy, simple implementation

Observers would not populate network.direction, populating observer.[egress & | ingress].interface instead

• Packetbeat- would be nice to add code to determine if the traffic being reported on is host related (to/from one of my ips) vs. running as an observer on a span port (not to/from one of my ips -> populate observer.ingress.interface)

Then we could close this out and start a separate discussion on the perimeter / groupings question...

[leehinman]

@dainperkins @dcode

Observers would not populate network.direction, populating observer.[egress & | ingress].interface instead

• Packetbeat- would be nice to add code to determine if the traffic being reported on is host related (to/from one of my ips) vs. running as an observer on a span port (not to/from one of my ips -> populate observer.ingress.interface)

Would it be acceptable for the [File,Audit,Packet]beat to have in it's
config file "observer" as a tag. Then if this tag is present we
populate the observer fields, if not we populate it as a host?

[dainperkins]

@leehinman
I don't see why that wouldn't work.

I'd think Filebeat would choose based on source integration, tho having the option to set wouldn't be bad for custom logs...

Auditbeat is a weird one, not knowing how it would deal with promiscuous sniffing on an interface (I can check on my pi-corelight and see if its being reported if I can get auditbeat running on the pi version of ubuntu (no promises...)

[webmat]

@leehinman I think a config flag is a good idea to flip from perimeter inbound/outbound vs host ingress/egress.

Although there may be subtleties around whether that beat will "only" capture perimeter events, when that switch is flipped. Perhaps there should be a way to change this per pipeline / module / protocol?

[dainperkins]

are you thinking the same field for either - that seems like a really bad idea (as it could be either/both)