[meta] Add support for proxies in ECS

[webmat]

We'd like to have support for proxies in ECS. We think the following could fall into this:

• load balancers (self-managed or services)
• caching proxies, both self-managed (e.g. Varnish) or CDNs
• level 4 load balancers

More details to come...

Past issues on proxies

#158, #387, #504, #544, #549, #626, sub-discussion in #51 (comment)

IP Forwarding

The network.forwarded_ip field as currently designed and documented has issues. They may be addressed separately, or could be part of this work on proxies.

Past issues around XFF or network.forwarded_ip: #523, #771, #874, #880, #895

[leehinman]

Would this just be limited to proxies or would any device in the middle of a network connection (not source or destination) be applicable? Things like NAT device, Router, Firewall, Switch, Wireless Access Point, etc?

[ebeahan]

I see proxy applying to any proxy-like devices like load balancers, API gateways, CDNs, web proxies, caching appliances, etc. A device that receives requests from the original source (device acts as a server), perhaps does some operations, and then passes the request upstream to an origin destination (device now acts as client).

If we use that criteria, I see layer two and three networking devices like routers, switches, or wireless APs excluded. While a NAT gateway are rewriting source and/or destination IPs and ports, I'd still not consider it a proxy. Depending on the feature set used on the firewall, we can get into some fuzzier area: URL filtering, SSL/TLS interception, DLP.

[webmat]

My thinking also reflects Eric's.

@leehinman do you see things that are missing, for the additional devices you mention?

[leehinman]

I'm OK with the proxy definition and scope.

I just thought some of the verbal discussions included a more generic definition (device in middle of a network connection) and wanted to see if those were in or if we were just focusing on proxies.

[dainperkins]

I don't think just "in the middle" qualifies, the difference being plain nat will usually just rewrite ips, ports (and maybe some app layer ip/port references, iirc seq/acks are unchanged) proxies will terminate a session and start a new one on behalf of the client.

Thinking it thru I think we'll need (assuming proxy as observer for global truth):

network fields

• proxy ip/vip (e.g. destination.ip from original client's perspective)
• snat ip (e.g. source "nat" ip, could be NAT, could also be a vip used as source for observer managed connection)
• proxied server ip (e.g. target system proxy/lb sends connection to)
• maybe an indicator for "nat / proxy" to minimally drive related.ip[s], related.community_id processing for e.g. ingest processing?

observer fields

• vip pool name/id/ip pool (e.g. load balancer log might reference)
• nat pool name/id/ip pool (e.g. load balancer log might reference)
• snat pool name/id/ip pool (e.g. load balancer log might reference)

/d