[cisco_asa] Fix Event code 106023 - Source/Destination IP not being parsed into respective source.ip or destination.ip field when interface nameif has a full colon (:)

[aleksmaus]

[cisco_asa] Fix Event code 106023 - Source/Destination IP not being parsed into respective source.ip or destination.ip field when interface nameif has a full colon (:)

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

• Closes cisco_asa: Event code 106023 - Source/Destination IP not being parsed into respective source.ip or destination.ip field when interface nameif has a full colon (:) #9184

[elasticmachine]

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

[pkoutsovasilis]

can you add also an example with ipv6 ips which are full of colons (please include also the port)? just to make sure that nothing breaks in the former case

[aleksmaus]

there is already one, search

Apr 27 2020 02:03:03 dev01: %ASA-4-106023: Deny icmp src OUTSIDE:2a02:cf40:add:4002:91f2:a9b2:e09a:6fc6 dst OUTSIDE:2a02:cf40:add:4002:91f2:a9b2:e09a:6fc6 (type 128, code 0) by access-group "OUTSIDE_in"

[pkoutsovasilis]

yes one already with an iface without colon it exists, but you are fixing support for ifaces with colons in them right?! So can we add one that covers this case iface with colon + ipv6 + port?

[aleksmaus]

hmmm. found another case (assuming if ipv6 was customized with prefix containing ':' ), works in grok dev tool

but breaks when regenerating with the elastic-package .... ughhh

Apr 27 2020 02:03:03 dev01: %ASA-4-106023: Deny icmp src v2:OUTSIDE:2a02:cf40:add:4002:91f2:a9b2:e09a:6fc6 dst v3:OUTSIDE:2a02:cf40:add:4002:91f2:a9b2:e09a:6fc6 (type 128, code 0) by access-group "OUTSIDE_in"

            "cisco": {
                "asa": {
                    "destination_interface": "v3:OUTSIDE:2a02:cf40:add:4002:91f2:a9b2:e09a",
                    "rule_name": "OUTSIDE_in",
                    "source_interface": "v2:OUTSIDE:2a02:cf40:add:4002:91f2:a9b2:e09a"
                }
            },

[elasticmachine]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[aleksmaus]

After doing some digging around, it doesn't look like we have a robust support for parsing ipv6 case if the hostname has prefix that contains ":".
Problem is recognizing the ":" for IPv6 address from the possible ":" in the the hostname. At the moment we don't have immediate cases where hostname contains ":" in the name with IPv6. So will attempt to cover the immediate case with IPv4 and then will see if IPv6 case ever materializes.

Currently having an issue with Grok. The expression works fine in the "Grok" debugger.
Sample Data:

Deny udp src outside:10.8.1.9 dst v3:inside:172.16.1.3/53 by access-group "outside_acl" [0x0, 0x0]

Grok pattern:

^Deny ((protocol %{POSINT:network.iana_number})|%{NOTSPACE:network.transport}) src %{IFACENAME:_temp_.cisco.source_interface}:%{IPV4ORHOST:source.address}(/%{NUMBER:source.port})?\s*(\(%{CISCO_USER_OR_SGT_SRC}\) )?dst %{IFACENAME:_temp_.cisco.destination_interface}:%{IPV4ORHOST:destination.address}(/%{NUMBER:destination.port})?%{DATA}by access-group "%{NOTSPACE:_temp_.cisco.list_id}"

Custom Patterns:

HOSTNAME \b(?:[0-9A-Za-z][0-9A-Za-z\-_]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z\-_]{0,62}))*(\.?|\b)
IPORHOST (?:%{IP}|%{HOSTNAME})
IPV4ORHOST (?:%{IPV4}|%{HOSTNAME})
NOTCOLON [^:]*
IFACENAME (.*:)?[^:]*
USERNAME [a-zA-Z0-9._'-]+
CISCO_USER_OR_SGT_SRC (?:%{CISCO_USER:_temp_.cisco.source_user_or_sgt}|%{CISCO_SGT:_temp_.cisco.source_user_or_sgt}|\((?:%{CISCO_USER:_temp_.cisco.source_user_or_sgt}|%{CISCO_SGT:_temp_.cisco.source_user_or_sgt})\))
CISCO_USER (?:\*\*\*\*\*|(?:(?:LOCAL\\)?(?:%{HOSTNAME}\\)?%{USERNAME}\$?(?:@%{HOSTNAME})?%{CISCO_SGT}?)|[^$]+)
CISCO_SGT (?:, *)?%{NUMBER}(?::%{WORD})?

Returns:

{
  "destination": {
    "port": "53",
    "address": "172.16.1.3"
  },
  "_temp_": {
    "cisco": {
      "destination_interface": "v3:inside",
      "list_id": "outside_acl",
      "source_interface": "outside"
    }
  },
  "source": {
    "address": "10.8.1.9"
  },
  "network": {
    "transport": "udp"
  }
}

But the same thing doesn't work in the ingest pipeline, cliping the destination_interface:

            "cisco": {
                "asa": {
                    "destination_interface": "v3",
                    "rule_name": "outside_acl",
                    "source_interface": "outside"
                }
            },

It's interesting, if I add prefixes to both src and dst or only src the pipeline works fine.
It's not helpful that the same processor with ingest pipeline returns different result than the grok debugger output.

Will get back to this later.

[aleksmaus]

KK. Figured out the problem with my previous attempt. The original regex

^Deny ((protocol %{POSINT:network.iana_number})|%{NOTSPACE:network.transport}) src %{NOTCOLON:_temp_.cisco.source_interface}:%{IPORHOST:source.address}(/%{NUMBER:source.port})?\s*(\(%{CISCO_USER_OR_SGT_SRC}\) )?dst %{NOTCOLON:_temp_.cisco.destination_interface}:%{IPORHOST:destination.address}(/%{NUMBER:destination.port})?%{DATA}by access-group "%{NOTSPACE:_temp_.cisco.list_id}"

actually matches when the dst has prefix:

Deny udp src outside:10.8.1.9 dst v3:inside:172.16.1.3/53 by access-group \"outside_acl\" [0x0, 0x0]

and doesn't match any other permutation of src or dst prefixes

That's why I saw the different result in the pipeline simulator.
Example playground

POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "_description",
    "processors": [
      {
        "grok": {
          "field": "message",
          "patterns": [
            "^Deny ((protocol %{POSINT:network.iana_number})|%{NOTSPACE:network.transport}) src %{NOTCOLON:_temp_.cisco.source_interface}:%{IPORHOST:source.address}(/%{NUMBER:source.port})?\\s*(\\(%{CISCO_USER_OR_SGT_SRC}\\) )?dst %{NOTCOLON:_temp_.cisco.destination_interface}:%{IPORHOST:destination.address}(/%{NUMBER:destination.port})?%{DATA}by access-group \"%{NOTSPACE:_temp_.cisco.list_id}\""         ],
          "pattern_definitions": {
            "HOSTNAME": "\\b(?:[0-9A-Za-z][0-9A-Za-z\\-_]{0,62})(?:\\.(?:[0-9A-Za-z][0-9A-Za-z\\-_]{0,62}))*(\\.?|\\b)",
            "IPORHOST": "(?:%{IP}|%{HOSTNAME})",
            "NOTCOLON": "[^:]*",
            "IFACENAME": "(.*)?%{NOTCOLON}",
            "USERNAME": "[a-zA-Z0-9._'-]+",
            "CISCO_USER_OR_SGT_SRC": "(?:%{CISCO_USER:_temp_.cisco.source_user_or_sgt}|%{CISCO_SGT:_temp_.cisco.source_user_or_sgt}|\\((?:%{CISCO_USER:_temp_.cisco.source_user_or_sgt}|%{CISCO_SGT:_temp_.cisco.source_user_or_sgt})\\))",
            "CISCO_USER": "(?:\\*\\*\\*\\*\\*|(?:(?:LOCAL\\\\)?(?:%{HOSTNAME}\\\\)?%{USERNAME}\\$?(?:@%{HOSTNAME})?%{CISCO_SGT}?)|[^$]+)",
            "CISCO_SGT": "(?:, *)?%{NUMBER}(?::%{WORD})?"
          }
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "message0": "Deny udp src outside:10.8.1.9 dst inside:172.16.1.3/53 by access-group \"outside_acl\" [0x0, 0x0]",
        "message1": "Deny udp src v2:outside:10.8.1.9 dst inside:172.16.1.3/53 by access-group \"outside_acl\" [0x0, 0x0]",
        "message": "Deny udp src outside:10.8.1.9 dst v3:inside:172.16.1.3/53 by access-group \"outside_acl\" [0x0, 0x0]",
        "message3": "Deny udp src v2:outside:10.8.1.9 dst v3:inside:172.16.1.3/53 by access-group \"outside_acl\" [0x0, 0x0]"
      }
    }
  ]
}

Output:

{
  "docs": [
    {
      "doc": {
        "_index": "index",
        "_version": "-3",
        "_id": "id",
        "_source": {
          "_temp_": {
            "cisco": {
              "destination_interface": "v3",
              "list_id": "outside_acl",
              "source_interface": "outside"
            }
          },
          "destination": {
            "address": "inside"
          },
          "message3": """Deny udp src v2:outside:10.8.1.9 dst v3:inside:172.16.1.3/53 by access-group "outside_acl" [0x0, 0x0]""",
          "source": {
            "address": "10.8.1.9"
          },
          "message0": """Deny udp src outside:10.8.1.9 dst inside:172.16.1.3/53 by access-group "outside_acl" [0x0, 0x0]""",
          "message": """Deny udp src outside:10.8.1.9 dst v3:inside:172.16.1.3/53 by access-group "outside_acl" [0x0, 0x0]""",
          "message1": """Deny udp src v2:outside:10.8.1.9 dst inside:172.16.1.3/53 by access-group "outside_acl" [0x0, 0x0]""",
          "network": {
            "transport": "udp"
          }
        },
        "_ingest": {
          "timestamp": "2024-08-30T01:20:09.769757584Z"
        }
      }
    }
  ]
}

Will update PR shortly

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[aleksmaus]

Hold on on reviewing this one. I'm not too happy that there is an additional fairly complex grok/regex for that special case, that now is checked for match first. I'll see if there is anything that can be done.

[aleksmaus]

Refined the existing regex to match better the existing data as well as the new logs when interface name contains :

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: fc25571

History

• 💚 Build #15212 succeeded 771b94c249eb5d42cb1e883bb358ff532e8379f3
• 💚 Build #15137 succeeded 4ef14996c63a0fce6208887e332d96ef3982e4f5
• 💔 Build #15130 failed dc0f88974ffb6bcbac76abdf519b0ec8862d0f3e

cc @aleksmaus