Skip to content

Conversation

@imays11
Copy link
Contributor

@imays11 imays11 commented Nov 12, 2025

Pull Request

Issue link(s):

Summary - What I changed

Came across some false positives as I was rule testing.

Temporary credentials are also created for AWS Internal operations and when an AWS service operates on your behalf. These both should be excluded from results. I saw this in my own testing, in prod data and in our alert telemetry. These cases can be excluded by looking for source.ip:* as this field is not populated for those internal AWS operations. This change reduced alert telemetry by ~40% over the last 30 days.

NOTE: Another false positive instance has been highlighted in the investigation guide. A legitimate AWS console login session is given temporary (ASIA) credentials which are populated for all the operations performed during that session. The only way to distinguish between these events and other temporary session token events, like those granted via AssumeRole or GetSessionToken, is with a field populated as sessionCredentialFromConsole: true. Right now our Integration does not map this field and it can only be found in event.original. I am putting in an Integration request to populate this field, which we could then use to further reduce false positives for rules like this.

  • updated description , false positives, and investigation guide
  • added source.ip:* to query

How To Test

Data in our stack to see both true and false positives over the last 30 days if you run the original query
script for executing the rule as expected: trigger_persistence_iam_api_calls_via_user_session_token.py

Screenshot of event.original with sessionCredentialFromConsole: true populated

Screenshot 2025-11-12 at 4 46 23 PM

Screenshot of both true and false positives w/ source.ip not populated for internal operations

Screenshot 2025-11-12 at 4 09 18 PM

Came across some false positives as I was rule testing.

Temporary credentials are also created for AWS Internal operations and when an AWS service operates on your behalf. These both should be excluded from results. I saw this in my own testing, in prod data and in our alert telemetry. These cases can be excluded by looking for `source.ip:*` as this field is not populated for those internal AWS operations.

NOTE: Another false positive instance has been highlighted in the investigation guide. A legitimate AWS console login session is given temporary (ASIA) credentials which are populated for all the operations performed during that session. The only way to distinguish between these events and other temporary session token events, like those granted via AssumeRole or GetSessionToken, is with a field populated as `sessionCredentialFromConsole: true`. Right now our Integration does not map this field and it can only be found in `event.original`. I am putting in an Integration request to populate this field, which we could then use to further reduce false positives for rules like this.

- updated description , false positives, and investigation guide
- added `source.ip:*` to query
@imays11 imays11 self-assigned this Nov 12, 2025
@imays11 imays11 added Integration: AWS AWS related rules Rule: Tuning tweaking or tuning an existing rule Team: TRADE Domain: Cloud labels Nov 12, 2025
@github-actions
Copy link
Contributor

Rule: Tuning - Guidelines

These guidelines serve as a reminder set of considerations when tuning an existing rule.

Documentation and Context

  • Detailed description of the suggested changes.
  • Provide example JSON data or screenshots.
  • Provide evidence of reducing benign events mistakenly identified as threats (False Positives).
  • Provide evidence of enhancing detection of true threats that were previously missed (False Negatives).
  • Provide evidence of optimizing resource consumption and execution time of detection rules (Performance).
  • Provide evidence of specific environment factors influencing customized rule tuning (Contextual Tuning).
  • Provide evidence of improvements made by modifying sensitivity by changing alert triggering thresholds (Threshold Adjustments).
  • Provide evidence of refining rules to better detect deviations from typical behavior (Behavioral Tuning).
  • Provide evidence of improvements of adjusting rules based on time-based patterns (Temporal Tuning).
  • Provide reasoning of adjusting priority or severity levels of alerts (Severity Tuning).
  • Provide evidence of improving quality integrity of our data used by detection rules (Data Quality).
  • Ensure the tuning includes necessary updates to the release documentation and versioning.

Rule Metadata Checks

  • updated_date matches the date of tuning PR merged.
  • min_stack_version should support the widest stack versions.
  • name and description should be descriptive and not include typos.
  • query should be inclusive, not overly exclusive. Review to ensure the original intent of the rule is maintained.

Testing and Validation

  • Validate that the tuned rule's performance is satisfactory and does not negatively impact the stack.
  • Ensure that the tuned rule has a low false positive rate.

excluding AWS Internal user agent
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants