network.forwarded_ip should be an ordered list of IP

[dainok]

Description of the issue:

network.forwarded_ip is used to map X-Forwarded-For addresses. XFF is an ordered list of IP, because a single request can pass through multiple proxies.

Any additional context or examples:

X-Forwarded-For: client, proxy1, proxy2

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For

[dainperkins]

@dainok,

Good catch, tho the network.forwarded_ip was meant for NAT translations, which are only locally significant.

What do you think about an http addition (http.forwarded_for)?

/d

[dainok]

http.forwarded_for is the best option I guess, because it's a HTTP header. But at this point I'm not sure what network.forwarded_ip should be.
This bug request is because filebeat-panw module is using network.forwarded_ip to store multiple http x-forwarded-for IP.

[dainperkins]

Apologies - I was thinking source.nat.ip, NOT network.forwarded_ip, which is specifically labeled as a proxy forward.

I still think that x-forwarded-for belongs under http, as in every implementation I can recall a proxied connection is, at ground truth, two totally separate network connections, whereas nat is inline replacement of ip/ports - but the connection is not terminated at any higher layers on the device handling translation.

I also believe, @webmat correct me if I am wrong, array of IPs is fine for an IP field

[dainok]

Agree with you but I still don't get when network.forwarded_ip and http.forwarded_for should contains different data.
Try to speculate:

• if http sesssion is observed, then http.forwared_for = network.forwarded_ip
• if L4 traffic is observed, then network.forwarded_ip is set, http.forwarded_for is not.
  What do you think about?

[ebeahan]

As documented today network.forwarded_ip is intended to be the original client host IP of the connection being proxied. My understanding is the device/service/process which is performing the proxying would be the authoritative source for this field (@dainperkins @webmat can help keep me honest if that's inaccurate 😄 ) .

For example - the following flow of a HTTP request:

[client] => [proxyA] => [originServer]

From proxyA's perspective, network.forwarded_ip would equal client IP.

proxyA (if configured to do so) would append client IP to an X-Forwarded-For HTTP header in the HTTP request proxied to originServer.

In a more complex scenario:

[client] => [proxyA] => [proxyB] => [originServer]

proxyA network.forwarded_ip will remain client IP. proxyA could optionally add a X-Forwarded-For HTTP header.

proxyB network.forwarded_ip will be proxyA IP used as the source of the connection between proxyA and proxyB. proxyB could also optionally add an additional X-Forwarded-For HTTP header or append to the existing X-Forwarded-For.

There are also proxy implementations leveraging the X-Real-IP header to retain the original client IP vs X-Forwarded-For list to be considered in practice.

Reading over the proposed http.forwarded_for field, this field would be a list of all IPs appearing in the X-Forwarded-For HTTP header(s). A field for proxied protocols outside http could also be considered (e.g. HAProxy's proxy protocol or SMTP XCLIENT) in a broader, separate discussion.

[dainok]

Ok, see #880

[ebeahan]

Additional note: these IP addresses would be good values to populate related.ip.

[webmat]

We created meta-issue #938 to add proxy support in ECS, and potentially address issues around network.forwarded_ip in the same work.

[ebeahan]

Closing in favor of #938