Skip to content

Commit

Permalink
Merge branch 'branch-23.01' into 564-doc-helm-chart-reference-names
Browse files Browse the repository at this point in the history
  • Loading branch information
dagardner-nv authored Jan 6, 2023
2 parents 717039c + 9eca388 commit 426b2ee
Show file tree
Hide file tree
Showing 7 changed files with 56 additions and 28 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ docker run --rm -ti --gpus=all -p8000:8000 -p8001:8001 -p8002:8002 \
--strict-readiness=false \
--disable-auto-complete-config \
--model-control-mode=explicit \
--load-model phishing-bert-onnx
--load-model=phishing-bert-onnx
```

Once we have Triton running, we can verify that it is healthy using [curl](https://curl.se/). The `/v2/health/live` endpoint should return a 200 status code:
Expand Down
4 changes: 2 additions & 2 deletions examples/abp_nvsmi_detection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ In this example, we will be using Morpheus' provided ABP NVSMI Detection model.

### The Dataset

The dataset that this workflow was designed to process contains NVIDIA GPU metrics at regular time intervals and is extracted by a NetQ agent and serialized into JSON. Each line in the dataset contains much of the same information that is returned by the `nvidia-smi` utility. We won't examine at a full message directly since each line contains 176 different columns, but it's possible to get a idea of how the dataset was generated using the `nvidia-smi dmon` command. If you run this yourself, the output similar to the following:
The dataset that this workflow was designed to process contains NVIDIA GPU metrics at regular time intervals and is extracted by a NetQ agent and serialized into JSON. Each line in the dataset contains much of the same information that is returned by the `nvidia-smi` utility. We won't examine at a full message directly since each line contains 176 different columns, but it's possible to get an idea of how the dataset was generated using the `nvidia-smi dmon` command. If you run this yourself, the output similar to the following:

```bash
$ nvidia-smi dmon
Expand Down Expand Up @@ -85,7 +85,7 @@ Once Triton has loaded the model, the following will be displayed:
+-------------------+---------+--------+
```

If this is not present in the output, check the Triton log for any error messages related to loading the model.
> **Note**: If this is not present in the output, check the Triton log for any error messages related to loading the model.
## Running the Pipeline

Expand Down
11 changes: 8 additions & 3 deletions examples/gnn_fraud_detection_pipeline/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-FileCopyrightText: Copyright (c) 2021-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -18,14 +18,19 @@ limitations under the License.

## Requirements

Prior to running the gnn fraud detection pipeline, additional requirements must be installed in to your Conda environment. A supplemental requirements file has been provided in this example directory.
Prior to running the GNN fraud detection pipeline, additional requirements must be installed in to your Conda environment. A supplemental requirements file has been provided in this example directory.

```bash
mamba env update -n ${CONDA_DEFAULT_ENV} -f examples/gnn_fraud_detection_pipeline/requirements.yml
```

## Running

##### Setup Env Variable
```bash
export MORPHEUS_ROOT=$(pwd)
```

Use Morpheus to run the GNN fraud detection Pipeline with the transaction data. A pipeline has been configured in `run.py` with several command line options:

```bash
Expand Down Expand Up @@ -53,7 +58,7 @@ Options:
--help Show this message and exit.
```

To launch the configured Morpheus pipeline with the sample data that is provided at `<MORPHEUS_ROOT>/models/dataset`, run the following:
To launch the configured Morpheus pipeline with the sample data that is provided at `$MORPHEUS_ROOT/models/dataset`, run the following:

```bash

Expand Down
19 changes: 13 additions & 6 deletions examples/log_parsing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,20 @@ Pull Docker image from NGC (https://ngc.nvidia.com/catalog/containers/nvidia:tri

Example:

```
```bash
docker pull nvcr.io/nvidia/tritonserver:22.08-py3
```

##### Start Triton Inference Server container
##### Setup Env Variable
```bash
export MORPHEUS_ROOT=$(pwd)
```
cd ${MORPHEUS_ROOT}/models

docker run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v $PWD:/models nvcr.io/nvidia/tritonserver:22.08-py3 tritonserver --model-repository=/models/triton-model-repo --model-control-mode=explicit --load-model log-parsing-onnx
##### Start Triton Inference Server Container
From the Morpheus repo root directory, run the following to launch Triton and load the `log-parsing-onnx` model:

```bash
docker run --rm -ti --gpus=all -p8000:8000 -p8001:8001 -p8002:8002 -v $PWD/models:/models nvcr.io/nvidia/tritonserver:22.08-py3 tritonserver --model-repository=/models/triton-model-repo --exit-on-error=false --model-control-mode=explicit --load-model log-parsing-onnx
```

##### Verify Model Deployment
Expand All @@ -47,6 +52,8 @@ Once Triton server finishes starting up, it will display the status of all loade
+------------------+---------+--------+
```

> **Note**: If this is not present in the output, check the Triton log for any error messages related to loading the model.
### Run Log Parsing Pipeline

Run the following from the `examples/log_parsing` directory to start the log parsing pipeline:
Expand All @@ -67,7 +74,7 @@ python run.py \
Use `--help` to display information about the command line options:

```bash
python ./examples/log_parsing/run.py --help
python run.py --help

Options:
--num_threads INTEGER RANGE Number of internal pipeline threads to use
Expand Down Expand Up @@ -95,7 +102,7 @@ Options:
```

### CLI Example
The above example is illustrative of using the Python API to build a custom Morpheus Pipeline. Alternately the Morpheus command line could have been used to accomplish the same goal. To do this we must ensure that the `examples`/log_parsing directory is available in the `PYTHONPATH` and each of the custom stages are registered as plugins.
The above example is illustrative of using the Python API to build a custom Morpheus Pipeline. Alternately the Morpheus command line could have been used to accomplish the same goal. To do this we must ensure that the `examples/log_parsing` directory is available in the `PYTHONPATH` and each of the custom stages are registered as plugins.

From the root of the Morpheus repo run:
```bash
Expand Down
6 changes: 3 additions & 3 deletions examples/nlp_si_detection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ This example illustrates how to use Morpheus to automatically detect Sensitive I

## Background

The goal of this example is to identify potentially sensitive information in network packet data as quickly as possible to limit exposure and take corrective action. Sensitive information is a broad term but can be generalized to any data that should be guarded from unautorized access. Credit card numbers, passwords, authorization keys, and emails are all examples of sensitive information.
The goal of this example is to identify potentially sensitive information in network packet data as quickly as possible to limit exposure and take corrective action. Sensitive information is a broad term but can be generalized to any data that should be guarded from unauthorized access. Credit card numbers, passwords, authorization keys, and emails are all examples of sensitive information.

In this example, we will be using Morpheus' provided NLP SI Detection model. This model is capable of detecting 10 different categories of sensitive information:

Expand Down Expand Up @@ -93,7 +93,7 @@ Once Triton has loaded the model, the following should be displayed:
| sid-minibert-onnx | 1 | READY |
+-------------------+---------+--------+
```

> **Note**: If this is not present in the output, check the Triton log for any error messages related to loading the model.
## Running the Pipeline

Expand Down Expand Up @@ -203,7 +203,7 @@ The output file `detections.jsonlines` will contain the original PCAP messages w
* secret_keys
* user

The value for these fields will either be a `1` indicating a decection and a `0` indicating no detection. An example row with a detection is:
The value for these fields will either be a `1` indicating a detection and a `0` indicating no detection. An example row with a detection is:
```json
{
"timestamp": 1616381019580,
Expand Down
37 changes: 26 additions & 11 deletions examples/ransomware_detection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,18 @@ Pull Docker image from NGC (https://ngc.nvidia.com/catalog/containers/nvidia:tri

Example:

```
```bash
docker pull nvcr.io/nvidia/tritonserver:22.08-py3
```

##### Start Triton Inference Server container
##### Setup Env Variable
```bash
cd ${MORPHEUS_ROOT}/examples/ransomware_detection
export MORPHEUS_ROOT=$(pwd)
```

##### Start Triton Inference Server Container
Run the following from the `examples/ransomware_detection` directory to launch Triton and load the `ransomw-model-short-rf` model:

```bash
# Run Triton in explicit mode
docker run --rm -ti --gpus=all -p8000:8000 -p8001:8001 -p8002:8002 -v $PWD/models:/models/triton-model-repo nvcr.io/nvidia/tritonserver:22.08-py3 \
tritonserver --model-repository=/models/triton-model-repo \
Expand All @@ -42,21 +46,30 @@ docker run --rm -ti --gpus=all -p8000:8000 -p8001:8001 -p8002:8002 -v $PWD/model
--load-model ransomw-model-short-rf
```

##### Verify Model Deployment
Once Triton server finishes starting up, it will display the status of all loaded models. Successful deployment of the model will show the following:

```
+----------------------------+---------+--------+
| Model | Version | Status |
+----------------------------+---------+--------+
| ransomw-model-short-rf | 1 | READY |
+----------------------------+---------+--------+
```

> **Note**: If this is not present in the output, check the Triton log for any error messages related to loading the model.
## Requirements
**Note**: Make sure `dask` and `distributed` are installed in your Conda environment before running the ransomware detection pipeline. Run the installation command specified below if not.
> **Note**: Make sure `dask` and `distributed` are installed in your Conda environment before running the ransomware detection pipeline. Run the installation command specified below if not.
```bash
conda install dask==2022.7.0 distributed==2022.7.0
```

## Run Pipeline
Launch the example using the following

Input features for a short model can be taken from every three snapshots sequence, such as (1, 2, 3), or (2, 3, 4). The sliding window represents the number of subsequent snapshots that need to be taken into consideration when generating the input for a model. Sliding window for the medium model is `5` and for the long model it is `10`.
## Run Ransomware Detection Pipeline
Run the following from the `examples/ransomware_detection` directory to start the ransomware detection pipeline:

```bash
cd ${MORPHEUS_ROOT}/examples/ransomware_detection

python run.py --server_url=localhost:8001 \
--sliding_window=3 \
--model_name=ransomw-model-short-rf \
Expand All @@ -65,6 +78,8 @@ python run.py --server_url=localhost:8001 \
--output_file=./ransomware_detection_output.jsonlines
```

Input features for a short model can be taken from every three snapshots sequence, such as (1, 2, 3), or (2, 3, 4). The sliding window represents the number of subsequent snapshots that need to be taken into consideration when generating the input for a model. Sliding window for the medium model is `5` and for the long model it is `10`.

The configuration options for this example can be queried with:

```bash
Expand Down
5 changes: 3 additions & 2 deletions examples/root_cause_analysis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ These examples illustrate how to use Morpheus to build a binary sequence classif

Like any other Linux based machine, DGX's generate a vast amount of logs. Analysts spend hours trying to identify the root causes of each failure. There could be infinitely many types of root causes of the failures. Some patterns might help to narrow it down; however, regular expressions can only help to identify previously known patterns. Moreover, this creates another manual task of maintaining a search script.

In this example, we show how we can use Morpheus to accelerate the analysis of the enormous amount of logs using machine learning. Another benefit of analyzing in a probabilistic way is that we can pin down previously undetected root causes. To achieve this, we will fine-tune a pre-trained BERT[^1] model with a classification layer using HuggingFace library.
In this example, we demonstrate how using Morpheus can accelerate the analysis of the enormous amount of logs using machine learning. Another benefit of analyzing in a probabilistic way is that we can pin down previously undetected root causes. To achieve this, we will fine-tune a pre-trained BERT[^1] model with a classification layer using HuggingFace library.

Once the model is capable of identifying even the new root causes, it can also be deployed as a process running in the machines to predict failures before they happen.

Expand Down Expand Up @@ -63,10 +63,11 @@ Once Triton has loaded the model, the following should be displayed:
+------------------------+---------+--------+
```
> **Note**: If this is not present in the output, check the Triton log for any error messages related to loading the model.
#### ONNX->TensorRT Model Conversion

To achieve optimized inference performance, Triton Inference Server provides option to convert our ONNX model to TensorRT. Simply add the following to the end of your `config.pbtxt`:
To achieve optimized inference performance, Triton Inference Server provides an option to convert our ONNX model to TensorRT. Simply add the following to the end of your `config.pbtxt`:
```
dynamic_batching {
preferred_batch_size: [ 1, 4, 8, 16, 32 ]
Expand Down

0 comments on commit 426b2ee

Please sign in to comment.