CTI 1.11 Addendum

rfcs/text/0008-threat-intel.md

@@ -2,7 +2,7 @@
 <!-- Leave this ID at 0000. The ECS team will assign a unique, contiguous RFC number upon merging the initial stage of this RFC. -->
 
 - Stage: **2 (candidate)** <!-- Update to reflect target stage. See https://elastic.github.io/ecs/stages.html -->
-- Date: **2021-06-23** <!-- The ECS team sets this date at merge time. This is the date of the latest stage advancement. -->
+- Date: **2021-07-06** <!-- The ECS team sets this date at merge time. This is the date of the latest stage advancement. -->
 
 Elastic Security Solution will be adding the capability to ingest, process and utilize threat intelligence information for increasing detection coverage and helping analysts make quicker investigation decisions. Threat intelligence can be collected from a number of sources with a variety of structured and semi-structured data representations. This makes threat intelligence an ideal candidate for ECS mappings. Threat intelligence data will require ECS mappings to normalize it and make it usable in our security solution. This RFC is focused on identifying new field sets and values that need to be created for threat intelligence data. Existing ECS field reuse will be prioritized where possible. If new fields are required we will utilize [STIX Cyber Observable data model](https://docs.oasis-open.org/cti/stix/v2.1/cs01/stix-v2.1-cs01.html#_mlbmudhl16lr) as guidance.
 
@@ -37,6 +37,8 @@ threat.indicator.port | long | 443 | Identifies a threat indicator as a port num
 threat.indicator.email.address | keyword | phish@evil.com | Identifies a threat indicator as an email address (irrespective of direction).
 threat.marking.tlp | keyword | RED | Data markings represent restrictions, permissions, and other guidance for how data can be used and shared. Examples could be TLP (WHITE, GREEN, AMBER, RED).
 threat.indicator.scanner_stats | long | 4 | Count of Anti virus/EDR that successfully detected malicious file or URL. Sources like VirusTotal, Reversing Labs often provide these statistics.
+threat.indicator.reference | keyword | https://feodotracker.abuse.ch/ | URL to the intelligence source
+threat.indicator.provider | keyword | lrz_urlhaus | The name of the indicator's provider
 
 ### Proposed New Values for Event Fieldset
 
@@ -122,8 +124,6 @@ Network Example
             "email-addr"
         ],
         "description": "Email address, domain, port, and IP address observed using an Angler EK campaign.",
-        "dataset": "threatintel.abuseurl",
-        "module": "threatintel",
         "provider": "Abuse.ch",
         "reference": "https://urlhaus.abuse.ch/url/1292596/",
         "confidence": "High",
@@ -182,8 +182,6 @@ File Example
             "file"
         ],
         "description": "Implant used during an Angler EK campaign.",
-        "dataset": "threatintel.malwarebazaar",
-        "module": "threatintel",
         "provider": "Abuse.ch",
         "reference": "https://bazaar.abuse.ch/sample/f3ec9a2f2766c6bcf8c2894a9927c227649249ac146aabfe8d26b259be7d7055",
         "confidence": "High",
@@ -361,6 +359,7 @@ Some examples of commercial intelligence include:
   * Stage 1 correction: https://github.com/elastic/ecs/pull/1100
 * Stage 1 (originally stage 2 prior to removal of RFC stage 4): https://github.com/elastic/ecs/pull/1127
 * Stage 2: https://github.com/elastic/ecs/pull/1293
+  * Stage 2 addendum: https://github.com/elastic/ecs/pull/1502
 
 
 <!--

rfcs/text/0008/as.yml

@@ -0,0 +1,5 @@
+---
+- name: as
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0008/file.yml

@@ -0,0 +1,5 @@
+---
+- name: file
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0008/geo.yml

@@ -0,0 +1,5 @@
+---
+- name: geo
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0008/hash.yml

@@ -0,0 +1,5 @@
+---
+- name: hash
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0008/pe.yml

@@ -0,0 +1,5 @@
+---
+- name: pe
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0008/registry.yml

@@ -0,0 +1,5 @@
+---
+- name: registry
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0008/threat.yml

@@ -142,3 +142,19 @@
         * RED
 
     example: White
+
+  - name: indicator.reference
+    level: extended
+    type: keyword
+    short: Indicator reference URL
+    description: >
+      Reference URL linking to additional information about this indicator.
+    example: https://system.example.com/indicator/0001234
+
+  - name: indicator.provider
+    level: extended
+    type: keyword
+    short: Indicator provider
+    description: >
+      The name of the indicator's provider.
+    example: lrz_urlhaus

rfcs/text/0008/url.yml

@@ -0,0 +1,5 @@
+---
+- name: url
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0008/x509.yml

@@ -0,0 +1,5 @@
+---
+- name: x509
+  reusable:
+    expected:
+      - threat.indicator

rfcs/text/0021-threat-enrichment.md

@@ -1,7 +1,7 @@
 # 0021: Threat Enrichment
 
 - Stage: **2 (candidate)** <!-- Update to reflect target stage. See https://elastic.github.io/ecs/stages.html -->
-- Date: **2021-06-24** <!-- The ECS team sets this date at merge time. This is the date of the latest stage advancement. -->
+- Date: **2021-07-06** <!-- The ECS team sets this date at merge time. This is the date of the latest stage advancement. -->
 
 <!--
 Stage 0: Provide a high level summary of the premise of these changes. Briefly describe the nature, purpose, and impact of the changes. ~2-5 sentences.
@@ -24,11 +24,10 @@ Stage 1: If the changes include field additions or modifications, please create
 Stage 1: Describe at a high level how this change affects fields. Include new or updated yml field definitions for all of the essential fields in this draft. While not exhaustive, the fields documented here should be comprehensive enough to deeply evaluate the technical considerations of this change. The goal here is to validate the technical details for all essential fields and to provide a basis for adding experimental field definitions to the schema. Use GitHub code blocks with yml syntax formatting, and add them to the corresponding RFC folder.
 -->
 
-As these fields represent the enrichment of an existing event with indicator information, they are comprised of three categories of data:
+As these fields represent the enrichment of an existing event with indicator information, they are comprised of two categories of data:
 
 1. The indicator's indicator fields, as defined in RFC 0018
-2. Other relevant ECS fields from the indicator (see below)
-3. Fields representing the context of the enrichment itself
+2. Fields representing the context of the enrichment itself
 
 ### Proposed new fields
 
@@ -84,26 +83,20 @@ If it is determined that an event matches a given indicator, that event can be e
             "tlp": "WHITE"
           },
           "first_seen": "2020-10-01",
+          "file": {
+            "hash": {
+              "sha256": "0c415dd718e3b3728707d579cf8214f54c2942e964975a5f925e0b82fea644b4",
+              "md5": "1eee2bf3f56d8abed72da2bc523e7431"
+            },
+            "size": 656896,
+            "name": "invoice.doc"
+          },
           "last_seen": "2020-11-01",
+          "reference": "https://system.example.com/event/#0001234",
           "sightings": 4,
           "type": ["sha256", "md5", "file_name", "file_size"],
           "description": "file last associated with delivering Angler EK"
         },
-        // event and file fields are copied from the indicator doc, if relevant
-        "event": {
-          "provider": "Abuse.ch",
-          "dataset": "threatintel.abusemalware",
-          "module": "threatintel",
-          "reference": "https://system.example.com/event/#0001234"
-        },
-        "file": {
-          "hash": {
-            "sha256": "0c415dd718e3b3728707d579cf8214f54c2942e964975a5f925e0b82fea644b4",
-            "md5": "1eee2bf3f56d8abed72da2bc523e7431"
-          },
-          "size": 656896,
-          "name": "invoice.doc"
-        },
         /* `matched` will provide context about which of the indicators above matched on this
               particular enrichment. If multiple matches for this indicator object, this could
               be a list */
@@ -140,25 +133,25 @@ If it is determined that an event matches a given indicator, that event can be e
      - policy file-sha256-policy:
        "match": {
          "indices": "threat-\*",
-         "match_field": "file.hash.sha256",
-         "enrich_fields": ["event", "file", "indicator"]
+         "match_field": "threat.indicator.file.hash.sha256",
+         "enrich_fields": ["threat.indicator"]
        }
    - set:
-     field: "threat_match.matched.type"
+     field: "threat_match.threat.matched.type"
      value: "file-sha256-policy"
    - set:
-     field: "threat_match.matched.field"
+     field: "threat_match.threat.matched.field"
      value: "file.hash.sha256"
    - set:
-     field: "threat_match.matched.atomic"
+     field: "threat_match.threat.matched.atomic"
      value: "{{ file.hash.sha256 }}"
    - set:
      field: "threat.enrichments"
      value: []
      override: false
    - append:
      field: "threat.enrichments"
-     value: "{{ threat_match }}"
+     value: "{{ threat_match.threat }}"
    - remove:
      field: "threat_match"
 
@@ -241,6 +234,7 @@ e.g.:
 * Stage 0: https://github.com/elastic/ecs/pull/1386
 * Stage 1: https://github.com/elastic/ecs/pull/1400
 * Stage 2: https://github.com/elastic/ecs/pull/1460
+  * Stage 2 addendum: https://github.com/elastic/ecs/pull/1502
 
 <!--
 * Stage 1: https://github.com/elastic/ecs/pull/NNN

rfcs/text/0021/as.yml

@@ -2,4 +2,4 @@
 - name: as
   reusable:
     expected:
-      - threat.enrichments
+      - threat.enrichments.indicator

rfcs/text/0021/file.yml

@@ -2,4 +2,4 @@
 - name: file
   reusable:
     expected:
-      - threat.enrichments
+      - threat.enrichments.indicator

rfcs/text/0021/geo.yml

@@ -2,4 +2,4 @@
 - name: geo
   reusable:
     expected:
-      - threat.enrichments
+      - threat.enrichments.indicator

rfcs/text/0021/hash.yml

@@ -2,4 +2,4 @@
 - name: hash
   reusable:
     expected:
-      - threat.enrichments
+      - threat.enrichments.indicator

rfcs/text/0021/pe.yml

@@ -2,4 +2,4 @@
 - name: pe
   reusable:
     expected:
-      - threat.enrichments
+      - threat.enrichments.indicator

rfcs/text/0021/registry.yml

@@ -2,4 +2,4 @@
 - name: registry
   reusable:
     expected:
-      - threat.enrichments
+      - threat.enrichments.indicator