Merge pull request #2973 from splunk/gitlab_release_v4.26.0

Gitlab release v4.26.0

.gitlab-ci.yml

@@ -2,15 +2,18 @@ default:
   image: docker-hub.repo.splunkdev.net/python:3.9
 
 variables:
+  EXTRACTO_VERSION:
+    value: "1.0.60"
+    description: "The version of extracto to use"
   SKIP_DOWNSTREAM_TESTING:
     value: "False"
     description: "If true, downstream testing will be suppressed (useful for debugging or forcing a release in an emergency)."
 
 stages:
   - validate
   - generate
-  - app_inspect
   - test
+  - app_inspect
   - release
 
 include:

README.md

@@ -25,103 +25,53 @@ Welcome to the Splunk Security Content
 
 This project gives you access to our repository of Analytic Stories, security guides that provide background on tactics, techniques and procedures (TTPs), mapped to the MITRE ATT&CK Framework, the Lockheed Martin Cyber Kill Chain, and CIS Controls. They include Splunk searches, machine learning algorithms and Splunk Phantom playbooks (where available)—all designed to work together to detect, investigate, and respond to threats.
 
-# Get Content🛡
-The latest Splunk Security Content can be obtained via:
-
-#### [SSE App](https://splunkbase.splunk.com/app/3435/)
-Grab the latest release of Splunk Security Essentials App and install it on a Splunk instance. You can download it from [splunkbase](https://splunkbase.splunk.com/app/3435/), it is a Splunk Supported App. SSE Splunk app today supports push updates for security content release, this is the **preferred way** to get content!
-
-#### [ESCU App](https://github.com/splunk/security_content/releases)
-Grab the latest release of DA-ESS-ContentUpdate.spl and install it on a Splunk instance. Alternatively, you can download it from [splunkbase](https://splunkbase.splunk.com/app/3449/), it is currently a Splunk Supported App.
-
-#### [API](https://docs.splunkresearch.com/?version=latest)
-```
-curl -s https://content.splunkresearch.com | jq
-{
-  "hello": "welcome to Splunks Research security content api"
-}
-```
-
-# Usage 🧰
-### contentctl.py
-The Content Control tool allows you to manipulate Splunk Security Content via the following actions:
-
-0. **init** - Initilialize a new repo from scratch so you can easily add your own content to a custom application.  Note that this requires a large number of command line arguments, so use python _contentctl.py init --help_ for documentation around those arguments.
-1. **new_content** - Creates new content (detection, story, baseline)
-2. **validate** - Validates written content
-3. **generate** - Generates a deployment package for different platforms (splunk_app)
-4. **build** - Builds an application suitable for deployment on a search head using Slim, the Splunk Packaging Toolkit
-5. **inspect** - Uses a local version of appinspect to ensure that the app you built meets basic quality standards.
-6. **cloud_deploy** - Using ACS, deploy your custom app to a running Splunk Cloud Instance.
-7. **convert** - Convert a detection rule with sigma syntax to a Splunk SPL detection
-8. **content_changer** - Perform changes on security content
-
-### pre-requisites
-Make sure you use python version 3.9.
+**Note:** We have sister projects that enable us to build the industry's best security content. These projects are the Splunk Attack Range, an attack simulation lab built around Splunk, and Contentctl, the tool that enables us to build, test, and package our content for distribution.
 
-```
-git clone git@github.com:splunk/security_content.git
-cd security_content
-pip install virtualenv
-virtualenv venv
-source venv/bin/activate
-pip install -r requirements.txt
-```
-### Architecture details for the tooling
-- [WIKI](https://github.com/splunk/security_content/wiki/Security-Content-Code)
+- [Splunk Attack Range](https://github.com/splunk/attack_range): An attack simulation lab built around Splunk.
+- [Contentctl](https://github.com/splunk/contentctl): The tool that enables us to build, test, and package our content for distribution.
 
-### create a new detection
-`python contentctl.py -p . new_content -t detection`
+# Get Content🛡
+The latest Splunk Security Content can be obtained via:
+
+### 🌐 [Website](https://www.notion.so/Splunk-c0afd5a0c59242a19f7ab555630b493d?pvs=21)
 
-for a more indepth write up on how to write content see our [guide](https://github.com/splunk/security_content/wiki/Developing-Content).
+Best way to discover and access our content is by using the [research.splunk.com](https://research.splunk.com/) website.
 
-### validate security content
-`python contentctl.py -p . validate -pr ESCU`
+### 🖥️ [Splunk Enterprise Security (ES) Content Update](https://docs.splunk.com/Documentation/ES/latest/Admin/Usecasecontentlibrary?#Update_the_Analytic_Stories)
 
-### generate a splunk app from current content
-`python contentctl.py -p . generate -o dist/escu -pr ESCU`
+Splunk security content ships as part of ESCU directly into, if you are an ES user, good news, you already have it!
 
-### convert a Sigma search into a Splunk detection
-Detection rule using tstats and cim datamodel:
-`python contentctl.py -p . convert -dm cim -o detections/endpoint/ -dp dev/endpoint/attempted_credential_dump_from_registry_via_reg_exe.yml`
+### 📦 [ESCU App](https://github.com/splunk/security_content/releases)
 
-Detection rule using raw:
-`python contentctl.py -p . convert -dm raw -o detections/endpoint/ -dp dev/endpoint/attempted_credential_dump_from_registry_via_reg_exe.yml`
+To manually download the latest release of Splunk Security Content (named DA-ESS-ContentUpdate.spl), you can visit the [splunkbase](https://splunkbase.splunk.com/app/3449/) page or the [release page](https://github.com/splunk/security_content/releases) on GitHub.
 
-Detection rule converted to Windows Security Event Code 4688:
-`python contentctl.py -p . convert -dm raw -lo "Windows Security 4688" -o detections/endpoint/ -dp dev/endpoint/attempted_credential_dump_from_registry_via_reg_exe.yml`
+# Tools 🧰
+The key tool that drives our content development is [contentctl](https://github.com/splunk/contentctl). Contentctl offers the following features:
 
-### perform changes on security content
-Content changer will perform a change function defined in [here](https://github.com/splunk/security_content/blob/add_content_changer/bin/contentctl_project/contentctl_core/application/use_cases/content_changer.py) on all content or the content defined through the filter condition:
-`python contentctl.py -p detections/endpoint content_changer --change_function update_description --filter_key name --filter_value "3CX Supply Chain Attack Network Indicators" "Hello World"`
+- Creating new detections
+- Validating the correctness of all necessary components for detections
+- Testing detections
+- Generating deployable apps from detections
 
+To learn more about contentctl and its capabilities, please visit the [contentctl repository](https://github.com/splunk/contentctl).
 
 # MITRE ATT&CK ⚔️
 ### Detection Coverage
-To view an up-to-date detection coverage map for all the content tagged with MITRE techniques visit: [https://mitremap.splunkresearch.com/](https://mitremap.splunkresearch.com/) under the **Detection Coverage** layer. Below is a snapshot in time of what technique we currently have some detection coverage for. The darker the shade of blue the more detections we have for this particular technique. This map is automatically updated on every release and generated from the [generate-coverage-map.py](https://github.com/splunk/security_content/blob/develop/bin/generate-coverage-map.py).
+To view an up-to-date detection coverage map for all the content tagged with MITRE techniques visit: [https://mitremap.splunkresearch.com/](https://mitremap.splunkresearch.com/) under the **Detection Coverage** layer. Below is a snapshot in time of what technique we currently have some detection coverage for.
 
 ![](docs/mitre-map/coverage.png)
 
-# Customize to your Environment 🏗
-Customize your content to change how [often detections run](https://github.com/splunk/security_content/wiki/Customize-to-Your-Environment#customizing-scheduling-and-alert-actions-with-deployments), or what the right source type for [sysmon](https://github.com/splunk/security_content/wiki/Customize-to-Your-Environment#customizing-source-types-with-macros) in your environment is please follow this [guide](https://github.com/splunk/security_content/wiki/Customize-to-Your-Environment).  
-
-# What's in an Analytic Story? 🗺
-A complete use case, specifically built to detect, investigate, and respond to a specific threat like [Credential Dumping](https://github.com/splunk/security_content/blob/develop/stories/credential_dumping.yml) or [Ransomware](https://github.com/splunk/security_content/blob/develop/stories/ransomware.yml). A group of detections and a response make up an analytic story, they are associated with the tag `analytic_story: <name>`.  
-
 # Content Parts 🧩
 
-* [detections/](detections/): Contains all 209 detection searches to-date and growing.
+* [detections/](detections/): Contains all detection searches to-date and growing.
 * [stories/](stories/): All Analytic Stories that are group detections or also known as Use Cases
 * [deployments/](deployments/): Configuration for the schedule and alert action for all content
 * [playbooks/](playbooks/): Incident Response Playbooks/Workflow for responding to a specific Use Case or Threat.
 * [baselines/](baselines/): Searches that must be executed before a detection runs. It is specifically useful for collecting data on a system before running your detection on the collected data.
-* [investigations/](investigations/): Investigations to further analysis the output from detections.
-* [dashboards/](dashboards/): JSON definitions of Mission Control dashboards, to be used as a response task. Currently not used.
+* [investigations/](investigations/): Investigations to further analyze the output from detections. For more information, you can refer to the [Splunk Enterprise Security documentation on timelines](https://docs.splunk.com/Documentation/ES/7.3.0/User/Timelines).
 * [macros/](macros/): Implements Splunk’s search macros, shortcuts to commonly used search patterns like sysmon source type. More on how macros are used to customize content below.
 * [lookups/](lookups/): Implements Splunk’s lookup, usually to provide a list of static values like commonly used ransomware extensions.
-* [security_content_automation/](security_content_automation/): It contains script for enriching detection with relevant supported TAs and also contains script for publishing release build to [Pre-QA artifactory](https://repo.splunk.com/artifactory/Solutions/DA/Pre-QA/) on every tag release.
-
-
+* [data_sources/](data_sources/): Defines the data sources, the necessary TA or App to collect them and the fields provided that can be used by the detections.
 
 # Contribution 🥰
 We welcome feedback and contributions from the community! Please see our [contributing to the project](https://github.com/splunk/security_content/wiki/Contributing-to-the-Project) for more information on how to get involved.
@@ -136,8 +86,6 @@ If you have questions or need support, you can:
 * Post a question to [Splunk Answers](http://answers.splunk.com)
 * Join the [#security-research](https://splunk-usergroups.slack.com/archives/C1S5BEF38) room in the [Splunk Slack channel](http://splunk-usergroups.slack.com)
 
-
-
 ## License
 Copyright 2022 Splunk Inc.
 

contentctl.yml

@@ -6,7 +6,7 @@ build:
   path_root: dist
   prefix: ESCU
   build: 004210
-  version: 4.25.0
+  version: 4.26.0
   label: ES Content Updates
   author_name: Splunk Threat Research Team
   author_email: research@splunk.com

detections/cloud/aws_ami_atttribute_modification_for_exfiltration.yml → detections/cloud/aws_ami_attribute_modification_for_exfiltration.yml

@@ -1,8 +1,8 @@
-name: AWS AMI Atttribute Modification for Exfiltration
+name: AWS AMI Attribute Modification for Exfiltration
 id: f2132d74-cf81-4c5e-8799-ab069e67dc9f
 version: 2
 date: '2023-03-31'
-author: BHavin Patel, Splunk
+author: Bhavin Patel, Splunk
 status: production
 type: TTP
 data_source: []
@@ -12,7 +12,7 @@ search: '`cloudtrail` eventName=ModifyImageAttribute (requestParameters.launchPe
 |  rename requestParameters.launchPermission.add.items{}.userId as accounts_added
  | eval ami_status=if(match(group_added,"all") ,"Public AMI", "Not Public")  | stats
   count min(_time) as firstTime max(_time) as lastTime  values(group_added) values(accounts_added) as accounts_added values(ami_status) by  src_ip region eventName userAgent user_arn aws_account_id userIdentity.principalId 
-|  `security_content_ctime(firstTime)`| `security_content_ctime(lastTime)` | `aws_ami_atttribute_modification_for_exfiltration_filter`'
+|  `security_content_ctime(firstTime)`| `security_content_ctime(lastTime)` | `aws_ami_attribute_modification_for_exfiltration_filter`'
 how_to_implement: You must install splunk AWS add on and Splunk App for AWS. This
   search works with AWS CloudTrail logs.
 known_false_positives: It is possible that an AWS admin has legitimately shared a

detections/cloud/aws_iam_delete_policy.yml

@@ -5,7 +5,7 @@ date: '2021-04-01'
 author: Michael Haag, Splunk
 status: production
 type: Hunting
-description: The following detection identifes when a policy is deleted on AWS. This
+description: The following detection identifies when a policy is deleted on AWS. This
   does not identify whether successful or failed, but the error messages tell a story
   of suspicious attempts. There is a specific process to follow when deleting a policy.
   First, detach the policy from all users, groups, and roles that the policy is attached

detections/cloud/cloud_security_groups_modifications_by_user.yml

@@ -0,0 +1,65 @@
+name: Cloud Security Groups Modifications by User
+id: cfe7cca7-2746-4bdf-b712-b01ed819b9de
+version: 1
+date: '2024-02-21'
+author: Bhavin Patel, Splunk 
+data_source: []
+type: Anomaly
+status: production
+description: The following analytic identifies users who are unsually modifying security group in your cloud enriovnment,focusing on actions such as modifications, deletions, or creations performed by users over 30-minute intervals. Analyzing patterns of modifications to security groups can help in identifying anomalous behavior that may indicate a compromised account or an insider threat.\
+
+  The detection calculates the standard deviation for each host and leverages the
+  3-sigma statistical rule to identify an unusual number of users. To customize this
+  analytic, users can try different combinations of the `bucket` span time and the
+  calculation of the `upperBound` field. This logic can be used for real time security
+  monitoring as well as threat hunting exercises.\
+
+  This detection will only trigger on all user and service accounts that have created/modified/deleted a security group .\
+
+  The analytics returned fields allow analysts to investigate the event further by
+  providing fields like source ip and values of the security objects affected.
+search: '| tstats dc(All_Changes.object) as unique_security_groups values(All_Changes.src) as src values(All_Changes.user_type) as user_type values(All_Changes.object_category) as object_category values(All_Changes.object) as objects
+    values(All_Changes.action) as action  values(All_Changes.user_agent) as user_agent values(All_Changes.command) as command from datamodel=Change WHERE All_Changes.object_category = "security_group" (All_Changes.action = modified OR All_Changes.action = deleted OR All_Changes.action = created)  by All_Changes.user  _time span=30m 
+|  `drop_dm_object_name("All_Changes")` 
+| eventstats avg(unique_security_groups) as avg_changes
+    , stdev(unique_security_groups) as std_changes by user 
+| eval upperBound=(avg_changes+std_changes*3) 
+| eval isOutlier=if(unique_security_groups > 2 and unique_security_groups >= upperBound, 1, 0) 
+| where isOutlier=1| `cloud_security_groups_modifications_by_user_filter`'
+how_to_implement: This search requries the Cloud infrastructure logs such as AWS Cloudtrail, GCP Pubsub Message logs, Azure Audit logs to be ingested into an accelerated Change datamodel. It is also recommended that users can try different combinations of the `bucket` span time and outlier conditions to better suit with their environment. 
+known_false_positives: It is possible that legitimate user/admin may modify a number of security groups  
+references:
+- https://attack.mitre.org/techniques/T1578/005/
+tags:
+  analytic_story:
+  - Suspicious Cloud User Activities
+  asset_type: Cloud Instance
+  confidence: 50
+  impact: 70
+  message:  Unsual number cloud security group modifications detected by user - $user$
+  mitre_attack_id:
+  - T1578.005
+  observable:
+  - name: user
+    type: User
+    role:
+    - Victim
+  product:
+  - Splunk Enterprise
+  - Splunk Enterprise Security
+  - Splunk Cloud
+  required_fields:
+  - _time
+  - All_Changes.object_id
+  - All_Changes.action
+  - All_Changes.status
+  - All_Changes.object_category
+  - All_Changes.user
+  risk_score: 35
+  security_domain: threat
+tests:
+- name: True Positive Test
+  attack_data:
+  - data: https://media.githubusercontent.com/media/splunk/attack_data/master/datasets/attack_techniques/T1578.005/aws_authorize_security_group/aws_authorize_security_group.json
+    sourcetype: aws:cloudtrail
+    source: aws_cloudtrail

detections/cloud/o365_multiple_users_failing_to_authenticate_from_ip.yml

@@ -7,7 +7,7 @@ status: production
 type: TTP
 data_source: []
 description: This analytic identifies instances where multiple users (more than 10 unique accounts) have failed to authenticate from a single IP address within a short time span (5 minutes). Such a pattern can be indicative of malicious activities, such as brute-force attacks or password spraying attempts. The detection leverages O365 audit logs, specifically focusing on Azure Active Directory login failures (AzureActiveDirectoryStsLogon). By aggregating these failures based on the source IP address and time, the analytic captures patterns where multiple unique user accounts have authentication failures from the same IP within a 5-minute window. Multiple authentication failures from a single IP address targeting various accounts can be a strong indicator of an attacker trying to gain unauthorized access. It could represent a brute-force attack, password spraying, or other malicious login attempts. Identifying and responding to such patterns promptly is crucial to prevent potential account compromises and unauthorized access to organizational resources. If the detection is a true positive, it suggests that an external entity is actively trying to breach the security by targeting multiple user accounts. While the attempts have been unsuccessful (as indicated by the login failures), it's a clear sign of malicious intent. Immediate action is required to block or monitor the suspicious IP, investigate the nature of the attempts, and potentially notify affected users to take precautionary measures like password changes or enabling multi-factor authentication.
-search: ' `o365_management_activity` Workload=AzureActiveDirectory Operation=UserLoginFailed record_type=AzureActiveDirectoryStsLogon 
+search: ' `o365_management_activity` Workload=AzureActiveDirectory Operation=UserLoginFailed ErrorNumber=50126 
   |  bucket span=5m _time
   |  stats dc(user) as unique_accounts values(user) as user values(LogonError) as LogonError values(signature) as signature values(UserAgent) as UserAgent by _time, src_ip
   |  where unique_accounts > 10
@@ -52,6 +52,7 @@ tags:
   - LogonError
   - signature
   - UserAgent
+  - ErrorNumber
   security_domain: identity
 tests:
 - name: True Positive Test