Feature - Reverse DNS Enrichment

[andrewkroh]

As a user I would like to be able to resolve IP addresses in events to hostnames by using reverse DNS lookups (query PTR records). My hosts send data directly to Elastic Cloud. Many (but not all) of the IP addresses in my events are private IP addresses that are only resolvable via internal DNS servers (so the enrichment cannot be done in Elastic Cloud).

Ideally I'd like it to be possible to only enrich events associated with my private network. So a condition that works on IP ranges would be nice, but I could also use a regex (less ideal) to an IP prefix.

processors:
# When destination.ip is in a private network (RFC 1918 or RFC 4193).
- dns:
    reverse:
      - destination.ip: destination.hostname
    when.network:
      destination.ip: private

# Enrich ip and client_ip for all DNS events.
- dns:
    reverse:
      - ip: hostname
      - client_ip: client_hostname
    when.equals:
      type: dns

[ph]

I have the same need locally :)

[webmat]

Ooh, I love the enrichment for private addresses :-)

The classification of the IP address in itself* would also be great to have as part of the event. Do we already have a way to do this classification in Beats? Or were you just laying out an expressive example and we don't actually have this yet?

I want that as part of the events because down the line, I can then do my GeoIP and reverse lookups centrally only on the public ones :-)

* By classification I mean: is it local, private, multicast, public or other.

[andrewkroh]

The configuration is just an idea at the moment. But you mention a use case I had not considered -- adding a classification to the event itself. I was only considering using that information in the processor's conditional logic rather than adding it to the event. I'll take that into consideration.

[webmat]

In case this helps, in the draft for Suricata in Logstash, I winged the classification based on a few Wikipedia resources, and skipped multicast & carrier grade networks (they were incorrectly flagged public, the fallback if it didn't match the local or private CIDRs).

Here's what I used:

  if [destination][ip] {
    cidr {
      address => '%{[destination][ip]}'
      network => ['10.0.0.0/8', '192.168.0.0/16', '172.16.0.0/12', 'fd00::/8']
      add_field => { '[destination][type]' => 'private' }
    }
    cidr {
      address => '%{[destination][ip]}'
      network => ['169.254.0.0/16', 'fe80::/10', '127.0.0.0/8', '0.0.0.0/8']
      add_field => { '[destination][type]' => 'local' }
    }
    if ![destination][type] { # Simplification. Not caring about carrier, multicast, etc.
      mutate { add_field => { '[destination][type]' => 'public'} }
    }

There must be a better way than this, though. Do you think Go network libraries can help directly with this?

In case they don't here's a few of the resources I used to wing it :-)

• Reserved IP addresses -- The most complete listing of all types of CIDRs I found, including carrier grade.
• Private & Local
• Multicast

[andrewkroh]

I have an implementation of the condition that I was experimenting with that allows for named classes in addition to custom CIDRs. andrewkroh@8ca8a03

[webmat]

Ah great, looks like Go libs are helping you classify them. So what I called public would be "unicast" and "global_unicast", in there, right?

I wonder if for tagging the events we shouldn't map all of these back to something simpler. Here's an example:

• loopback, link_local_unicast, link_local_multicast, interface_local_multicast => local
• unicast, global_unicast => public
• private => private
• multicast => multicast
• unspecified => unspecified

We'd lose a bit details, but I think for the purposes of operational and security monitoring, the distinction between all of the local types of communications is not that interesting.