Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Digital Fingerprinting 2.0 Pipelines #315

Merged
0 commits merged into from
Sep 13, 2022

Conversation

efajardo-nv
Copy link
Contributor

@efajardo-nv efajardo-nv commented Aug 2, 2022

  • Update existing HAMMAH pipeline to new DFP 2.0 requirements.
  • Add source stages for Azure and Duo.
  • Add Azure and Duo DFP pipelines to Morpheus CLI.
  • Example on how to run CloudTrail DFP pipeline using Python API.
  • Add options to save trained user models to file and load later for inference-only workflow.
  • Make AE source stages extensible to allow for deriving new features.
  • Derived features for Azure and Duo logs. See [FEA] Digital Fingerprinting Updates for Feature Engineering #207
  • Option to use generic model for user who don't have enough training data
  • Add README with information about environment setup and running DFP pipelines.
  • Update dfencoder dependency to new branch which includes breaking changes. Updated dfencoder branch to not break current/previous Morpheus versions. Can now be merged into nv-updates.
  • Update HAMMAH validation scripts

Closes #207
Closes #208
Closes #339

@efajardo-nv efajardo-nv added enhancement Additional functionality added to an existing feature non-breaking Non-breaking change feature request New feature or request 2 - In Progress labels Aug 2, 2022
@efajardo-nv efajardo-nv changed the base branch from branch-22.08 to branch-22.09 August 16, 2022 16:33
@efajardo-nv efajardo-nv marked this pull request as ready for review August 29, 2022 10:57
@efajardo-nv efajardo-nv requested review from a team as code owners August 29, 2022 10:57
Copy link
Contributor

@gbatmaz gbatmaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@efajardo-nv efajardo-nv marked this pull request as draft September 7, 2022 17:59
@efajardo-nv
Copy link
Contributor Author

rerun tests

@efajardo-nv efajardo-nv marked this pull request as ready for review September 13, 2022 01:48
@mdemoret-nv mdemoret-nv added this to the Digital Fingerprinting milestone Sep 13, 2022
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in a good enough state to start building off of it with the other PRs that need to go in and can be worked in parallel

@mdemoret-nv
Copy link
Contributor

@gpucibot merge

@ghost ghost merged commit acec47d into nv-morpheus:branch-22.09 Sep 13, 2022
mdemoret-nv pushed a commit that referenced this pull request Sep 15, 2022
- Update existing HAMMAH pipeline to [new DFP 2.0 requirements](#208).
- Add source stages for Azure and Duo.
- Add Azure and Duo DFP pipelines to Morpheus CLI.
- Example on how to run CloudTrail DFP pipeline using Python API.
- Add options to save trained user models to file and load later for inference-only workflow.
- Make AE source stages extensible to allow for deriving new features.
- Derived features for Azure and Duo logs. See #207
- Option to use generic model for user who don't have enough training data
- Add README with information about environment setup and running DFP pipelines.
- ~Update `dfencoder` dependency to [new branch](https://github.com/efajardo-nv/dfencoder/tree/morpheus-22.08) which includes breaking changes.~ Updated `dfencoder` branch to not break current/previous Morpheus versions. Can now be merged into `nv-updates`.
- Update HAMMAH validation scripts

Closes #207 
Closes #208
Closes #339

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)
  - Michael Demoret (https://github.com/mdemoret-nv)

Approvers:
  - https://github.com/gbatmaz
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #315
ghost pushed a commit that referenced this pull request Sep 17, 2022
This builds off of PR #315 

Still a work in progress.

Closes #340 
Closes #341
Closes #342 
Closes #210

Authors:
  - Michael Demoret (https://github.com/mdemoret-nv)
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #352
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Additional functionality added to an existing feature feature request New feature or request non-breaking Non-breaking change
Projects
Archived in project
3 participants