Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Sync with zeek package #20696

Merged
merged 1 commit into from
Aug 20, 2020
Merged

Conversation

leehinman
Copy link
Contributor

What does this PR do?

Syncs zeek filebeat module with zeek packages dataset. Specifically:

  • connection
    • remove redundant source.ip & destination.ip mapping
    • add null check for append to related.ip
    • remove duplicate setting of event.kind & event.category
  • dce_rpc
    • add null check for append to related.ip
  • dnp3
    • add append to related.ip
  • dns
    • move conversion of zeek.dns.ts to @timestamp to ingest pipeline
    • add event.created to ingest pipeline
  • intel
    • move conversion of zeek.intel.ts to @timestamp to ingest pipeline
  • socks
    • change processor for event.outcome from append to set
  • ssl
    • fix so event.type is correctly set
    • fix quoting of = in ingest pipeline
  • x509
    • convert pipeline to yaml format

Why is it important?

We should get same behavior when using Filebeat module vs package
dataset

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=zeek mage -v pythonIngetTest

Related issues

@leehinman leehinman added enhancement Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:SIEM labels Aug 19, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Aug 19, 2020
@botelastic
Copy link

botelastic bot commented Aug 19, 2020

This pull request doesn't have a Team:<team> label.

- connection
  + remove redundant source.ip & destination.ip mapping
  + add null check for append to related.ip
  + remove duplicate setting of event.kind & event.category
- dce_rpc
  + add null check for append to related.ip
- dnp3
  + add append to related.ip
- dns
  + move conversion of zeek.dns.ts to @timestamp to ingest pipeline
  + add event.created to ingest pipeline
- intel
  + move conversion of zeek.intel.ts to @timestamp to ingest pipeline
- socks
  + change processor for event.outcome from append to set
- ssl
  + fix so event.type is correctly set
  + fix quoting of = in ingest pipeline
- x509
  + convert pipeline to yaml format
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20696 updated]

  • Start Time: 2020-08-19T21:48:53.086+0000

  • Duration: 63 min 15 sec

Test stats 🧪

Test Results
Failed 0
Passed 2466
Skipped 388
Total 2854

@leehinman leehinman merged commit 7de72d6 into elastic:master Aug 20, 2020
@leehinman leehinman added v7.10.0 and removed needs_backport PR is waiting to be backported to other branches. labels Aug 20, 2020
leehinman added a commit to leehinman/beats that referenced this pull request Aug 20, 2020
- connection
  + remove redundant source.ip & destination.ip mapping
  + add null check for append to related.ip
  + remove duplicate setting of event.kind & event.category
- dce_rpc
  + add null check for append to related.ip
- dnp3
  + add append to related.ip
- dns
  + move conversion of zeek.dns.ts to @timestamp to ingest pipeline
  + add event.created to ingest pipeline
- intel
  + move conversion of zeek.intel.ts to @timestamp to ingest pipeline
- socks
  + change processor for event.outcome from append to set
- ssl
  + fix so event.type is correctly set
  + fix quoting of = in ingest pipeline
- x509
  + convert pipeline to yaml format

(cherry picked from commit 7de72d6)
leehinman added a commit that referenced this pull request Aug 24, 2020
- connection
  + remove redundant source.ip & destination.ip mapping
  + add null check for append to related.ip
  + remove duplicate setting of event.kind & event.category
- dce_rpc
  + add null check for append to related.ip
- dnp3
  + add append to related.ip
- dns
  + move conversion of zeek.dns.ts to @timestamp to ingest pipeline
  + add event.created to ingest pipeline
- intel
  + move conversion of zeek.intel.ts to @timestamp to ingest pipeline
- socks
  + change processor for event.outcome from append to set
- ssl
  + fix so event.type is correctly set
  + fix quoting of = in ingest pipeline
- x509
  + convert pipeline to yaml format

(cherry picked from commit 7de72d6)
@leehinman leehinman deleted the zeek_sync branch September 18, 2020 03:12
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
- connection
  + remove redundant source.ip & destination.ip mapping
  + add null check for append to related.ip
  + remove duplicate setting of event.kind & event.category
- dce_rpc
  + add null check for append to related.ip
- dnp3
  + add append to related.ip
- dns
  + move conversion of zeek.dns.ts to @timestamp to ingest pipeline
  + add event.created to ingest pipeline
- intel
  + move conversion of zeek.intel.ts to @timestamp to ingest pipeline
- socks
  + change processor for event.outcome from append to set
- ssl
  + fix so event.type is correctly set
  + fix quoting of = in ingest pipeline
- x509
  + convert pipeline to yaml format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants