Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix powershell error on events 40961 and 40962 #10792

Merged
merged 4 commits into from
Sep 12, 2024

Conversation

bjmcnic
Copy link
Contributor

@bjmcnic bjmcnic commented Aug 14, 2024

Proposed commit message

Fix pipeline_error for powershell_operational events 40961 and 40962.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@bjmcnic bjmcnic added Integration:windows Windows Team:Security-Windows Platform Security Windows Platform Team [elastic/sec-windows-platform] labels Aug 14, 2024
@bjmcnic bjmcnic self-assigned this Aug 14, 2024
@bjmcnic bjmcnic requested a review from a team as a code owner August 14, 2024 17:31
@elasticmachine
Copy link

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

@bjmcnic bjmcnic requested a review from a team as a code owner August 14, 2024 17:36
Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing changes to manifest.yml that match the change in changelog.yml

@@ -537,6 +537,7 @@ processors:
description: Remove all empty values from event_data.
lang: painless
source: ctx?.winlog?.event_data?.entrySet().removeIf(entry -> entry.getValue() == null || entry.getValue().equals(""));
if: ctx?.winlog?.event_data != null
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure we need to add this to:

forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml

but it probably applies to:

  • powershell/elasticsearch/ingest_pipeline/default.yml
  • sysmon_operational/elasticsearch/ingest_pipeline/default.yml
  • forwarded/elasticsearch/ingest_pipeline/powershell.yml
  • forwarded/elasticsearch/ingest_pipeline/sysmon_operational.yml

Do you happen to have any sample events you could add to powershell_operational/_dev/test/pipeline/test-events.json ? That way the tests will catch this if we break it in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leehinman Thanks for the tips. I was able to find sample events for powershell_operational and added them to tests. I also made the analogous fix powershell and sysmon_operational as you'd suggested, but was unable to create real events that would trigger the issue for those.

@andrewkroh andrewkroh added the bugfix Pull request that fixes a bug issue label Aug 19, 2024
@jamiehynds
Copy link

Hi @bjmcnic - a customer has been in touch looking for an update on this PR. Their main focus is 40961 and 40962 event support within the powershell_operational pipeline? Is it possible to prioritise that pipeline first, and handle the forwarded events (and other) pipelines separately?

@bjmcnic
Copy link
Contributor Author

bjmcnic commented Sep 10, 2024

@jamiehynds I added the tests for powershell_operational and the forwarded events for that. I was unable to naturally generate erring events for powershell or sysmon_operational, but I put the fixes in anyway. We'll see what reviewers think.

@elasticmachine
Copy link

🚀 Benchmarks report

Package windows 👍(4) 💚(2) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
applocker_msi_and_script 9433.96 7462.69 -1971.27 (-20.9%) 💔
applocker_packaged_app_deployment 14705.88 11904.76 -2801.12 (-19.05%) 💔
powershell_operational 4347.83 2976.19 -1371.64 (-31.55%) 💔

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @bjmcnic

Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for adding the test cases.

@bjmcnic bjmcnic merged commit 8711ef8 into elastic:main Sep 12, 2024
5 checks passed
@elasticmachine
Copy link

Package windows - 2.0.2 containing this change is available at https://epr.elastic.co/search?package=windows

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:windows Windows Team:Security-Windows Platform Security Windows Platform Team [elastic/sec-windows-platform]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants