Skip to content

Commit

Permalink
Updating versions for v24.06.00
Browse files Browse the repository at this point in the history
  • Loading branch information
dagardner-nv committed Apr 5, 2024
1 parent f4169a9 commit 752e85f
Show file tree
Hide file tree
Showing 22 changed files with 53 additions and 53 deletions.
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
[submodule "external/morpheus-visualizations"]
path = external/morpheus-visualizations
url = https://github.com/nv-morpheus/morpheus-visualizations.git
branch = branch-24.03
branch = branch-24.06
[submodule "external/utilities"]
path = external/utilities
url = https://github.com/nv-morpheus/utilities.git
branch = branch-24.03
branch = branch-24.06
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ morpheus_utils_initialize_cuda_arch(morpheus)
# Note intentionally excluding CUDA from the LANGUAGES list allowing us to set some clang specific settings later when
# we call morpheus_utils_enable_cuda()
project(morpheus
VERSION 24.03.00
VERSION 24.06.00
LANGUAGES C CXX
)

Expand Down
2 changes: 1 addition & 1 deletion conda/environments/all_cuda-121_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ dependencies:
- libtool
- libwebp=1.3.2
- mlflow=2.9.2
- mrc=24.03
- mrc=24.06
- myst-parser=0.18.1
- nbsphinx
- networkx=2.8.8
Expand Down
2 changes: 1 addition & 1 deletion conda/environments/dev_cuda-121_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ dependencies:
- isort
- librdkafka>=1.9.2,<1.10.0a0
- mlflow=2.9.2
- mrc=24.03
- mrc=24.06
- myst-parser=0.18.1
- nbsphinx
- networkx=2.8.8
Expand Down
2 changes: 1 addition & 1 deletion dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ dependencies:
- glog=0.6
- gxx_linux-64=11.2
- librdkafka>=1.9.2,<1.10.0a0
- mrc=24.03
- mrc=24.06
- ninja=1.11
- nlohmann_json=3.11
- pkg-config=0.29 # for mrc cmake
Expand Down
12 changes: 6 additions & 6 deletions docs/source/cloud_deployment_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ The Helm chart (`morpheus-ai-engine`) that offers the auxiliary components requi
Follow the below steps to install Morpheus AI Engine:

```bash
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-ai-engine-24.03.tgz --username='$oauthtoken' --password=$API_KEY --untar
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-ai-engine-24.06.tgz --username='$oauthtoken' --password=$API_KEY --untar
```
```bash
helm install --set ngc.apiKey="$API_KEY" \
Expand Down Expand Up @@ -146,7 +146,7 @@ replicaset.apps/zookeeper-87f9f4dd 1 1 1 54s
Run the following command to pull the Morpheus SDK Client (referred to as Helm chart `morpheus-sdk-client`) on to your instance:

```bash
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-sdk-client-24.03.tgz --username='$oauthtoken' --password=$API_KEY --untar
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-sdk-client-24.06.tgz --username='$oauthtoken' --password=$API_KEY --untar
```

#### Morpheus SDK Client in Sleep Mode
Expand Down Expand Up @@ -184,7 +184,7 @@ kubectl -n $NAMESPACE exec sdk-cli-helper -- cp -RL /workspace/models /common
The Morpheus MLflow Helm chart offers MLflow server with Triton plugin to deploy, update, and remove models from the Morpheus AI Engine. The MLflow server UI can be accessed using NodePort `30500`. Follow the below steps to install the Morpheus MLflow:

```bash
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-mlflow-24.03.tgz --username='$oauthtoken' --password=$API_KEY --untar
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-mlflow-24.06.tgz --username='$oauthtoken' --password=$API_KEY --untar
```
```bash
helm install --set ngc.apiKey="$API_KEY" \
Expand Down Expand Up @@ -402,7 +402,7 @@ To publish messages to a Kafka topic, we need to copy datasets to locations wher
kubectl -n $NAMESPACE exec sdk-cli-helper -- cp -R /workspace/examples/data /common
```

Refer to the [Morpheus CLI Overview](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/docs/source/basics/overview.rst) and [Building a Pipeline](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/docs/source/basics/building_a_pipeline.md) documentation for more information regarding the commands.
Refer to the [Morpheus CLI Overview](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/basics/overview.rst) and [Building a Pipeline](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/basics/building_a_pipeline.md) documentation for more information regarding the commands.

> **Note**: Before running the example pipelines, ensure the criteria below are met:
- Ensure models specific to the pipeline are deployed.
Expand Down Expand Up @@ -782,8 +782,8 @@ kubectl -n $NAMESPACE exec deploy/broker -c broker -- kafka-topics.sh \

## Additional Documentation
For more information on how to use the Morpheus Python API to customize and run your own optimized AI pipelines, Refer to below documentation.
- [Morpheus Developer Guides](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/docs/source/developer_guide/guides.md)
- [Morpheus Pipeline Examples](https://github.com/nv-morpheus/Morpheus/tree/branch-24.03/examples)
- [Morpheus Developer Guides](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/developer_guide/guides.md)
- [Morpheus Pipeline Examples](https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/examples)


## Troubleshooting
Expand Down
12 changes: 6 additions & 6 deletions docs/source/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,22 +41,22 @@ More advanced users, or those who are interested in using the latest pre-release
### Pull the Morpheus Image
1. Go to [https://catalog.ngc.nvidia.com/orgs/nvidia/teams/morpheus/containers/morpheus/tags](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/morpheus/containers/morpheus/tags)
1. Choose a version
1. Download the selected version, for example for `24.03`:
1. Download the selected version, for example for `24.06`:
```bash
docker pull nvcr.io/nvidia/morpheus/morpheus:24.03-runtime
docker pull nvcr.io/nvidia/morpheus/morpheus:24.06-runtime
```

> **Note about Morpheus versions:**
>
> Morpheus uses Calendar Versioning ([CalVer](https://calver.org/)). For each Morpheus release there will be an image tagged in the form of `YY.MM-runtime` this tag will always refer to the latest point release for that version. In addition to this there will also be at least one point release version tagged in the form of `vYY.MM.00-runtime` this will be the initial point release for that version (ex. `v24.03.00-runtime`). In the event of a major bug, we may release additional point releases (ex. `v24.03.01-runtime`, `v24.03.02-runtime` etc...), and the `YY.MM-runtime` tag will be updated to reference that point release.
> Morpheus uses Calendar Versioning ([CalVer](https://calver.org/)). For each Morpheus release there will be an image tagged in the form of `YY.MM-runtime` this tag will always refer to the latest point release for that version. In addition to this there will also be at least one point release version tagged in the form of `vYY.MM.00-runtime` this will be the initial point release for that version (ex. `v24.06.00-runtime`). In the event of a major bug, we may release additional point releases (ex. `v24.06.01-runtime`, `v24.06.02-runtime` etc...), and the `YY.MM-runtime` tag will be updated to reference that point release.
>
> Users who want to ensure they are running with the latest bug fixes should use a release image tag (`YY.MM-runtime`). Users who need to deploy a specific version into production should use a point release image tag (`vYY.MM.00-runtime`).
### Starting the Morpheus Container
1. Ensure that [The NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker) is installed.
1. Start the container downloaded from the previous section:
```bash
docker run --rm -ti --runtime=nvidia --gpus=all --net=host -v /var/run/docker.sock:/var/run/docker.sock nvcr.io/nvidia/morpheus/morpheus:24.03-runtime bash
docker run --rm -ti --runtime=nvidia --gpus=all --net=host -v /var/run/docker.sock:/var/run/docker.sock nvcr.io/nvidia/morpheus/morpheus:24.06-runtime bash
```

Note about some of the flags above:
Expand Down Expand Up @@ -133,10 +133,10 @@ To run the built "release" container, use the following:
./docker/run_container_release.sh
```

The `./docker/run_container_release.sh` script accepts the same `DOCKER_IMAGE_NAME`, and `DOCKER_IMAGE_TAG` environment variables that the `./docker/build_container_release.sh` script does. For example, to run version `v24.03.00` use the following:
The `./docker/run_container_release.sh` script accepts the same `DOCKER_IMAGE_NAME`, and `DOCKER_IMAGE_TAG` environment variables that the `./docker/build_container_release.sh` script does. For example, to run version `v24.06.00` use the following:

```bash
DOCKER_IMAGE_TAG="v24.03.00-runtime" ./docker/run_container_release.sh
DOCKER_IMAGE_TAG="v24.06.00-runtime" ./docker/run_container_release.sh
```

## Launching Triton Server
Expand Down
2 changes: 1 addition & 1 deletion examples/digital_fingerprinting/production/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# limitations under the License.

ARG MORPHEUS_CONTAINER=nvcr.io/nvidia/morpheus/morpheus
ARG MORPHEUS_CONTAINER_VERSION=v24.03.00-runtime
ARG MORPHEUS_CONTAINER_VERSION=v24.06.00-runtime

FROM ${MORPHEUS_CONTAINER}:${MORPHEUS_CONTAINER_VERSION} as base

Expand Down
4 changes: 2 additions & 2 deletions examples/digital_fingerprinting/production/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ services:
target: jupyter
args:
- MORPHEUS_CONTAINER=${MORPHEUS_CONTAINER:-nvcr.io/nvidia/morpheus/morpheus}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.03.00-runtime}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.06.00-runtime}
deploy:
resources:
reservations:
Expand Down Expand Up @@ -109,7 +109,7 @@ services:
target: runtime
args:
- MORPHEUS_CONTAINER=${MORPHEUS_CONTAINER:-nvcr.io/nvidia/morpheus/morpheus}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.03.00-runtime}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.06.00-runtime}
image: dfp_morpheus
container_name: morpheus_pipeline
deploy:
Expand Down
2 changes: 1 addition & 1 deletion examples/digital_fingerprinting/starter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ We show here how to set up and run the DFP pipeline for three log types: CloudTr

## Environment Setup

Follow the instructions [here](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/docs/source/developer_guide/contributing.md) to set up your development environment in either a Docker container or conda environment.
Follow the instructions [here](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/developer_guide/contributing.md) to set up your development environment in either a Docker container or conda environment.

## Morpheus CLI

Expand Down
2 changes: 1 addition & 1 deletion external/utilities
6 changes: 3 additions & 3 deletions models/model-cards/abp-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ limitations under the License.

**Link:**

* https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/training-data/abp-sample-nvsmi-training-data.json <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/abp-sample-nvsmi-training-data.json <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**

Expand All @@ -98,7 +98,7 @@ limitations under the License.

**Link:**

* https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/validation-data/abp-validation-data.jsonlines <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/abp-validation-data.jsonlines <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**

Expand Down Expand Up @@ -180,7 +180,7 @@ NVIDIA believes Trustworthy AI is a shared responsibility and we have establishe

### Link the location of the training dataset's repository.

* https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/training-data/abp-sample-nvsmi-training-data.json
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/abp-sample-nvsmi-training-data.json

### Is the model used in an application with physical safety impact?

Expand Down
32 changes: 16 additions & 16 deletions models/model-cards/dfp-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,36 +64,36 @@ The model architecture consists of an Autoencoder, where the reconstruction loss
* Linux<br>

## Model Version(s):
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/dfp-models/hammah-role-g-20211017-dill.pkl
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/dfp-models/hammah-user123-20211017-dill.pkl
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/dfp-models/hammah-role-g-20211017-dill.pkl
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/dfp-models/hammah-user123-20211017-dill.pkl

# Training & Evaluation:

## Training Dataset:

**Link:**
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.03/models/datasets/training-data/cloudtrail
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/training-data/cloudtrail

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**

The training dataset consists of AWS CloudTrail logs. It contains logs from two entities, providing information about their activities within the AWS environment.
* [hammah-role-g-training-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/training-data/cloudtrail/hammah-role-g-training-part1.json): 700 records <br>
* [hammah-role-g-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/training-data/cloudtrail/hammah-role-g-training-part2.json): 1187 records <br>
* [hammah-user123-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/training-data/cloudtrail/hammah-user123-training-part2.json): 1000 records <br>
* [hammah-user123-training-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/training-data/cloudtrail/hammah-user123-training-part3.json): 1000 records <br>
* [hammah-user123-training-part4.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/training-data/cloudtrail/hammah-user123-training-part4.json): 387 records <br>
* [hammah-role-g-training-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-role-g-training-part1.json): 700 records <br>
* [hammah-role-g-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-role-g-training-part2.json): 1187 records <br>
* [hammah-user123-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-user123-training-part2.json): 1000 records <br>
* [hammah-user123-training-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-user123-training-part3.json): 1000 records <br>
* [hammah-user123-training-part4.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-user123-training-part4.json): 387 records <br>

## Evaluation Dataset:
**Link:**
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.03/models/datasets/validation-data/cloudtrail <br>
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/validation-data/cloudtrail <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**

The evaluation dataset consists of AWS CloudTrail logs. It contains logs from two entities, providing information about their activities within the AWS environment.
* [hammah-role-g-validation.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/validation-data/cloudtrail/hammah-role-g-validation.json): 314 records
* [hammah-user123-validation-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part1.json): 300 records
* [hammah-user123-validation-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part2.json): 300 records
* [hammah-user123-validation-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part3.json): 247 records
* [hammah-role-g-validation.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-role-g-validation.json): 314 records
* [hammah-user123-validation-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part1.json): 300 records
* [hammah-user123-validation-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part2.json): 300 records
* [hammah-user123-validation-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part3.json): 247 records

## Inference:
**Engine:**
Expand Down Expand Up @@ -159,7 +159,7 @@ The evaluation dataset consists of AWS CloudTrail logs. It contains logs from tw

### List the steps explaining how this model works.
* The model works by training on baseline behaviors and subsequently detecting deviations from the established baseline, triggering alerts accordingly.
* [Training notebook](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/training-tuning-scripts/dfp-models/hammah-20211017.ipynb)
* [Training notebook](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/training-tuning-scripts/dfp-models/hammah-20211017.ipynb)

### Name the adversely impacted groups (protected classes) this has been tested to deliver comparable outcomes regardless of:
* Not Applicable
Expand All @@ -179,7 +179,7 @@ The evaluation dataset consists of AWS CloudTrail logs. It contains logs from tw
## Model Card ++ Saftey & Security Subcard

### Link the location of the training dataset's repository (if able to share).
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.03/models/datasets/training-data/cloudtrail
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/training-data/cloudtrail

### Is the model used in an application with physical safety impact?
* No
Expand Down Expand Up @@ -240,7 +240,7 @@ The evaluation dataset consists of AWS CloudTrail logs. It contains logs from tw
* No

### Are we able to identify and trace source of dataset?
* Yes ([fully synthetic dataset](https://github.com/nv-morpheus/Morpheus/tree/branch-24.03/models/datasets/training-data/cloudtrail))
* Yes ([fully synthetic dataset](https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/training-data/cloudtrail))

### Does data labeling (annotation, metadata) comply with privacy laws?
* Not applicable (as the dataset is fully synthetic)
Expand Down
2 changes: 1 addition & 1 deletion models/model-cards/gnn-fsi-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ NVIDIA believes Trustworthy AI is a shared responsibility and we have establishe
* None

### Link the relevant end user license agreement
* [Apache 2.0](https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/LICENSE)
* [Apache 2.0](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/LICENSE)

## Model Card ++ Saftey & Security Subcard

Expand Down
2 changes: 1 addition & 1 deletion models/model-cards/phishing-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ limitations under the License.

**Link:**

* https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/models/datasets/validation-data/phishing-email-validation-data.jsonlines <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/phishing-email-validation-data.jsonlines <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**

Expand Down
Loading

0 comments on commit 752e85f

Please sign in to comment.