Skip to content

Add Dockerfiles for Neuron DLC with SDK 2.20.2 #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 22, 2024
Merged

Add Dockerfiles for Neuron DLC with SDK 2.20.2 #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker/pytorch/inference/1.13.1/Dockerfile.neuron
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
FROM public.ecr.aws/docker/library/ubuntu:20.04

LABEL dlc_major_version="1"

Check failure on line 3 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3048 style: Invalid label key.
LABEL maintainer="Amazon AI"
LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port=true

# Neuron SDK components version numbers
ARG NEURON_FRAMEWORK_VERSION=1.13.1.2.11.7.0
ARG NEURON_FRAMEWORK_VERSION=1.13.1.2.11.13.0
ARG NEURON_CC_VERSION=1.24.0.0
ARG NEURONX_TOOLS_VERSION=2.19.0.0

Expand All @@ -22,7 +22,7 @@
ENV SAGEMAKER_SERVING_MODULE=sagemaker_pytorch_serving_container.serving:main
ENV TEMP=/home/model-server/tmp

RUN apt-get update \

Check failure on line 25 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends software-properties-common \
&& add-apt-repository ppa:openjdk-r/ppa \
Expand Down Expand Up @@ -54,9 +54,9 @@
&& apt-get clean

RUN echo "deb https://apt.repos.neuron.amazonaws.com focal main" > /etc/apt/sources.list.d/neuron.list
RUN wget -qO - https://apt.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | apt-key add -

Check failure on line 57 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check

RUN apt-get update \

Check failure on line 59 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3015 info: Avoid additional packages by specifying `--no-install-recommends`
&& apt-get install -y aws-neuronx-tools=$NEURONX_TOOLS_VERSION \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /tmp/tmp* \
Expand All @@ -67,7 +67,7 @@
mv /etc/ssl/certs/java/cacerts.jks /etc/ssl/certs/java/cacerts; \
/var/lib/dpkg/info/ca-certificates-java.postinst configure;

RUN curl -L -o ~/mambaforge.sh https://github.com/conda-forge/miniforge/releases/download/${MAMBA_VERSION}/Mambaforge-${MAMBA_VERSION}-Linux-x86_64.sh \

Check failure on line 70 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL4001 warning: Either use Wget or Curl but not both
&& chmod +x ~/mambaforge.sh \
&& ~/mambaforge.sh -b -p /opt/conda \
&& rm ~/mambaforge.sh \
Expand All @@ -86,7 +86,7 @@

&& /opt/conda/bin/conda clean -ya

RUN conda install -c conda-forge \

Check failure on line 89 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`

Check failure on line 89 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3042 warning: Avoid use of cache directory with pip. Use `pip install --no-cache-dir <package>`
scikit-learn \
h5py \
requests \
Expand All @@ -97,7 +97,7 @@
enum-compat \
ipython

RUN pip install --no-cache-dir -U \

Check failure on line 100 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`
opencv-python>=4.8.1.78 \
"numpy<1.24,>1.21" \
"scipy>=1.8.0" \
Expand All @@ -108,7 +108,7 @@
boto3 \
cryptography

RUN pip install neuron-cc==$NEURON_CC_VERSION --extra-index-url https://pip.repos.neuron.amazonaws.com \

Check failure on line 111 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3042 warning: Avoid use of cache directory with pip. Use `pip install --no-cache-dir <package>`
torch-neuron==$NEURON_FRAMEWORK_VERSION \
&& pip install -U protobuf==3.19.5 \
torchserve==${TORCHSERVE_VERSION} \
Expand All @@ -132,7 +132,7 @@

RUN chmod +x /usr/local/bin/deep_learning_container.py

RUN pip install --no-cache-dir "sagemaker-pytorch-inference==${SM_TOOLKIT_VERSION}"

Check failure on line 135 in docker/pytorch/inference/1.13.1/Dockerfile.neuron

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.

RUN HOME_DIR=/root \
&& curl -o ${HOME_DIR}/oss_compliance.zip https://aws-dlinfra-utilities.s3.amazonaws.com/oss_compliance.zip \
Expand Down
6 changes: 3 additions & 3 deletions docker/pytorch/inference/1.13.1/Dockerfile.neuronx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
FROM public.ecr.aws/docker/library/ubuntu:20.04

LABEL dlc_major_version="1"

Check failure on line 3 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3048 style: Invalid label key.
LABEL maintainer="Amazon AI"
LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port=true

# Neuron SDK components version numbers
ARG NEURONX_FRAMEWORK_VERSION=1.13.1.1.16.0
ARG NEURONX_DISTRIBUTED_VERSION=0.9.0
ARG NEURONX_CC_VERSION=2.15.141.0
ARG NEURONX_CC_VERSION=2.15.143.0
ARG NEURONX_TRANSFORMERS_VERSION=0.12.313
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.26.0-17a033bc8
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.14.0-6e27b8d5b
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.33.0-d2128d1aa
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.19.0-5856c0b42
ARG NEURONX_TOOLS_VERSION=2.19.0.0

ARG PYTHON=python3.10
Expand All @@ -26,7 +26,7 @@
ENV SAGEMAKER_SERVING_MODULE=sagemaker_pytorch_serving_container.serving:main
ENV TEMP=/home/model-server/tmp

RUN apt-get update \

Check failure on line 29 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends software-properties-common \
&& add-apt-repository ppa:openjdk-r/ppa \
Expand Down Expand Up @@ -58,9 +58,9 @@
&& apt-get clean

RUN echo "deb https://apt.repos.neuron.amazonaws.com focal main" > /etc/apt/sources.list.d/neuron.list
RUN wget -qO - https://apt.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | apt-key add -

Check failure on line 61 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check

RUN apt-get update \

Check failure on line 63 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3015 info: Avoid additional packages by specifying `--no-install-recommends`
&& apt-get install -y \
aws-neuronx-tools=$NEURONX_TOOLS_VERSION \
aws-neuronx-collectives=$NEURONX_COLLECTIVES_LIB_VERSION \
Expand All @@ -74,7 +74,7 @@
mv /etc/ssl/certs/java/cacerts.jks /etc/ssl/certs/java/cacerts; \
/var/lib/dpkg/info/ca-certificates-java.postinst configure;

RUN curl -L -o ~/mambaforge.sh https://github.com/conda-forge/miniforge/releases/download/${MAMBA_VERSION}/Mambaforge-${MAMBA_VERSION}-Linux-x86_64.sh \

Check failure on line 77 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL4001 warning: Either use Wget or Curl but not both
&& chmod +x ~/mambaforge.sh \
&& ~/mambaforge.sh -b -p /opt/conda \
&& rm ~/mambaforge.sh \
Expand All @@ -93,7 +93,7 @@

&& /opt/conda/bin/conda clean -ya

RUN conda install -c conda-forge \

Check failure on line 96 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`

Check failure on line 96 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3042 warning: Avoid use of cache directory with pip. Use `pip install --no-cache-dir <package>`
scikit-learn \
h5py \
requests \
Expand All @@ -104,7 +104,7 @@
enum-compat \
ipython

RUN pip install --no-cache-dir -U \

Check failure on line 107 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`
opencv-python>=4.8.1.78 \
"numpy<1.24,>1.21" \
"scipy>=1.8.0" \
Expand All @@ -115,7 +115,7 @@
boto3 \
cryptography

RUN pip install -U --extra-index-url https://pip.repos.neuron.amazonaws.com \

Check failure on line 118 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3042 warning: Avoid use of cache directory with pip. Use `pip install --no-cache-dir <package>`
neuronx-cc==$NEURONX_CC_VERSION \
torch-neuronx==$NEURONX_FRAMEWORK_VERSION \
transformers-neuronx==$NEURONX_TRANSFORMERS_VERSION \
Expand All @@ -142,7 +142,7 @@

RUN chmod +x /usr/local/bin/deep_learning_container.py

RUN pip install --no-cache-dir "sagemaker-pytorch-inference==${SM_TOOLKIT_VERSION}"

Check failure on line 145 in docker/pytorch/inference/1.13.1/Dockerfile.neuronx

View workflow job for this annotation

GitHub Actions / dockerfile-linter 

DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.

# patch default_pytorch_inference_handler.py to import torch_neuronx
RUN DEST_DIR=$(python -c "import os.path, sagemaker_pytorch_serving_container; print(os.path.dirname(sagemaker_pytorch_serving_container.__file__))") \
Expand Down
8 changes: 4 additions & 4 deletions docker/pytorch/inference/2.1.2/Dockerfile.neuronx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port=true

# Neuron SDK components version numbers
ARG NEURONX_DISTRIBUTED_VERSION=0.9.0
ARG NEURONX_CC_VERSION=2.15.141.0
ARG NEURONX_FRAMEWORK_VERSION=2.1.2.2.3.1
ARG NEURONX_CC_VERSION=2.15.143.0
ARG NEURONX_FRAMEWORK_VERSION=2.1.2.2.3.2
ARG NEURONX_TRANSFORMERS_VERSION=0.12.313
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.26.0-17a033bc8
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.14.0-6e27b8d5b
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.33.0-d2128d1aa
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.19.0-5856c0b42
ARG NEURONX_TOOLS_VERSION=2.19.0.0

ARG PYTHON=python3.10
Expand Down
23 changes: 10 additions & 13 deletions docker/pytorch/training/1.13.1/Dockerfile.neuronx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ LABEL dlc_major_version="1"
# Neuron SDK components version numbers
ARG NEURONX_FRAMEWORK_VERSION=1.13.1.1.16.0
ARG NEURONX_DISTRIBUTED_VERSION=0.9.0
ARG NEURONX_DISTRIBUTED_TRAINING_VERSION=1.0.0
ARG NEURONX_CC_VERSION=2.15.141.0
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.26.0-17a033bc8
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.14.0-6e27b8d5b
ARG NEURONX_DISTRIBUTED_TRAINING_VERSION=1.0.1
ARG NEURONX_CC_VERSION=2.15.143.0
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.33.0-d2128d1aa
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.19.0-5856c0b42
ARG NEURONX_TOOLS_VERSION=2.19.0.0

ARG PYTHON=python3.10
Expand Down Expand Up @@ -162,12 +162,11 @@ RUN git clone https://github.com/NVIDIA/apex.git /root/apex \

#Install dependencies from requirements and extras for SageMaker usecase
RUN wget https://raw.githubusercontent.com/aws-neuron/neuronx-distributed-training/master/requirements.txt \
&& pip install --no-deps --no-cache-dir --no-build-isolation -r requirements.txt /root/apex/dist/apex-0.1-py3-none-any.whl \
&& pip install --force-reinstall "numba==0.57.1" \
"multiprocess==0.70.16" \
"numpy>=1.24.3,<=1.25.2" \
"dill==0.3.8"

&& pip install --no-cache-dir -r requirements.txt /root/apex/dist/apex-0.1-py3-none-any.whl \
&& pip install --force-reinstall "multiprocess==0.70.16" \
"dill==0.3.8" \
"torch==1.13.1"


RUN ${PIP} install --force-reinstall --no-deps neuronx_distributed_training==$NEURONX_DISTRIBUTED_TRAINING_VERSION --extra-index-url https://pip.repos.neuron.amazonaws.com

Expand All @@ -192,9 +191,7 @@ RUN pip install --no-cache-dir -U \
"opencv-python>=4.8.1.78" \
"plotly>=5.11,<6" \
"seaborn>=0.12,<1" \
"numba>=0.56.4,<0.57" \
"shap>=0.41,<1" \
"numpy<1.24,>1.21"
"shap>=0.41,<1"

# EFA Installer does apt get. Make sure to run apt update before that
RUN apt-get update
Expand Down
86 changes: 3 additions & 83 deletions docker/pytorch/training/1.13.1/Dockerfile.neuronx.cve_allowlist.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,84 +1,4 @@
{
"CVE-2023-6730": {
"description": "Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.",
"remediation": {
"recommendation": {
"text": "None Provided"
}
},
"score": 8.8,
"score_details": {
"cvss": {
"adjustments": [],
"score": 8.8,
"scoreSource": "NVD",
"scoringVector": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H",
"version": "3.1"
}
},
"severity": "HIGH",
"source": "NVD",
"source_url": "https://nvd.nist.gov/vuln/detail/CVE-2023-6730",
"status": "ACTIVE",
"title": "CVE-2023-6730 - transformers, transformers",
"vulnerability_id": "CVE-2023-6730",
"vulnerable_packages": [
{
"epoch": 0,
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.31.0.dist-info/METADATA",
"name": "transformers",
"packageManager": "PYTHONPKG",
"version": "4.31.0"
},
{
"epoch": 0,
"filePath": "requirements.txt",
"name": "transformers",
"packageManager": "PIP",
"version": "4.31.0"
}
]
},
"CVE-2023-7018": {
"description": "Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.",
"remediation": {
"recommendation": {
"text": "None Provided"
}
},
"score": 7.8,
"score_details": {
"cvss": {
"adjustments": [],
"score": 7.8,
"scoreSource": "NVD",
"scoringVector": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H",
"version": "3.1"
}
},
"severity": "HIGH",
"source": "NVD",
"source_url": "https://nvd.nist.gov/vuln/detail/CVE-2023-7018",
"status": "ACTIVE",
"title": "CVE-2023-7018 - transformers, transformers",
"vulnerability_id": "CVE-2023-7018",
"vulnerable_packages": [
{
"epoch": 0,
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.31.0.dist-info/METADATA",
"name": "transformers",
"packageManager": "PYTHONPKG",
"version": "4.31.0"
},
{
"epoch": 0,
"filePath": "requirements.txt",
"name": "transformers",
"packageManager": "PIP",
"version": "4.31.0"
}
]
},
"CVE-2024-31580": {
"description": "PyTorch before v2.2.0 was discovered to contain a heap buffer overflow vulnerability in the component /runtime/vararg_functions.cpp. This vulnerability allows attackers to cause a Denial of Service (DoS) via a crafted input.",
"remediation": {
Expand Down Expand Up @@ -197,17 +117,17 @@
"vulnerable_packages": [
{
"epoch": 0,
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.31.0.dist-info/METADATA",
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.36.0.dist-info/METADATA",
"name": "transformers",
"packageManager": "PYTHONPKG",
"version": "4.31.0"
"version": "4.36.0"
},
{
"epoch": 0,
"filePath": "requirements.txt",
"name": "transformers",
"packageManager": "PIP",
"version": "4.31.0"
"version": "4.36.0"
}
]
},
Expand Down
28 changes: 13 additions & 15 deletions docker/pytorch/training/2.1.2/Dockerfile.neuronx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ LABEL dlc_major_version="1"

# Neuron SDK components version numbers
ARG NEURONX_DISTRIBUTED_VERSION=0.9.0
ARG NEURONX_DISTRIBUTED_TRAINING_VERSION=1.0.0
ARG NEURONX_CC_VERSION=2.15.141.0
ARG NEURONX_FRAMEWORK_VERSION=2.1.2.2.3.1
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.26.0-17a033bc8
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.14.0-6e27b8d5b
ARG NEURONX_DISTRIBUTED_TRAINING_VERSION=1.0.1
ARG NEURONX_CC_VERSION=2.15.143.0
ARG NEURONX_FRAMEWORK_VERSION=2.1.2.2.3.2
ARG NEURONX_COLLECTIVES_LIB_VERSION=2.22.33.0-d2128d1aa
ARG NEURONX_RUNTIME_LIB_VERSION=2.22.19.0-5856c0b42
ARG NEURONX_TOOLS_VERSION=2.19.0.0

ARG PYTHON=python3.10
Expand Down Expand Up @@ -146,8 +146,9 @@ RUN ${PIP} config set global.extra-index-url https://pip.repos.neuron.amazonaws.
RUN ${PIP} install --force-reinstall --no-deps neuronx_distributed==$NEURONX_DISTRIBUTED_VERSION --extra-index-url https://pip.repos.neuron.amazonaws.com

## Installation for Neuronx Distributed Training framework
# Install Cython
RUN pip install --no-cache-dir Cython
# Install Cython & wheel
RUN ${PIP} install --no-cache-dir Cython \
&& ${PIP} install --no-cache-dir wheel

# Copy the apex_setup.py file
COPY apex_setup.py /root/apex_setup.py
Expand All @@ -161,11 +162,10 @@ RUN git clone https://github.com/NVIDIA/apex.git /root/apex \

#Install dependencies from requirements and extras for SageMaker usecase
RUN wget https://raw.githubusercontent.com/aws-neuron/neuronx-distributed-training/master/requirements.txt \
&& pip install --no-deps --no-cache-dir --no-build-isolation -r requirements.txt /root/apex/dist/apex-0.1-py3-none-any.whl \
&& pip install --force-reinstall "numba==0.57.1" \
"multiprocess==0.70.16" \
"numpy>=1.24.3,<=1.25.2" \
"dill==0.3.8"
&& ${PIP} install --no-cache-dir -r requirements.txt /root/apex/dist/apex-0.1-py3-none-any.whl \
&& ${PIP} install --force-reinstall "multiprocess==0.70.16" \
"dill==0.3.8" \
"torch==2.1.2"


RUN ${PIP} install --force-reinstall --no-deps neuronx_distributed_training==$NEURONX_DISTRIBUTED_TRAINING_VERSION --extra-index-url https://pip.repos.neuron.amazonaws.com
Expand All @@ -191,9 +191,7 @@ RUN pip install --no-cache-dir -U \
"opencv-python>=4.8.1.78" \
"plotly>=5.11,<6" \
"seaborn>=0.12,<1" \
"numba>=0.56.4,<0.57" \
"shap>=0.41,<1" \
"numpy<1.24,>1.21"
"shap>=0.41,<1"

# EFA Installer does apt get. Make sure to run apt update before that
RUN apt-get update
Expand Down
102 changes: 40 additions & 62 deletions docker/pytorch/training/2.1.2/Dockerfile.neuronx.cve_allowlist.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,81 +1,34 @@
{
"CVE-2023-6730": {
"description": "Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.",
"CVE-2022-40897": {
"description": "Python Packaging Authority (PyPA) setuptools before 65.5.1 allows remote attackers to cause a denial of service via HTML in a crafted package or custom PackageIndex page. There is a Regular Expression Denial of Service (ReDoS) in package_index.py.",
"remediation": {
"recommendation": {
"text": "None Provided"
}
},
"score": 8.8,
"score": 5.9,
"score_details": {
"cvss": {
"adjustments": [],
"score": 8.8,
"score": 5.9,
"scoreSource": "NVD",
"scoringVector": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H",
"scoringVector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:H",
"version": "3.1"
}
},
"severity": "HIGH",
"severity": "MEDIUM",
"source": "NVD",
"source_url": "https://nvd.nist.gov/vuln/detail/CVE-2023-6730",
"source_url": "https://nvd.nist.gov/vuln/detail/CVE-2022-40897",
"status": "ACTIVE",
"title": "CVE-2023-6730 - transformers, transformers",
"vulnerability_id": "CVE-2023-6730",
"title": "CVE-2022-40897 - setuptools",
"vulnerability_id": "CVE-2022-40897",
"vulnerable_packages": [
{
"epoch": 0,
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.31.0.dist-info/METADATA",
"name": "transformers",
"packageManager": "PYTHONPKG",
"version": "4.31.0"
},
{
"epoch": 0,
"filePath": "requirements.txt",
"name": "transformers",
"packageManager": "PIP",
"version": "4.31.0"
}
]
},
"CVE-2023-7018": {
"description": "Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.",
"remediation": {
"recommendation": {
"text": "None Provided"
}
},
"score": 7.8,
"score_details": {
"cvss": {
"adjustments": [],
"score": 7.8,
"scoreSource": "NVD",
"scoringVector": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H",
"version": "3.1"
}
},
"severity": "HIGH",
"source": "NVD",
"source_url": "https://nvd.nist.gov/vuln/detail/CVE-2023-7018",
"status": "ACTIVE",
"title": "CVE-2023-7018 - transformers, transformers",
"vulnerability_id": "CVE-2023-7018",
"vulnerable_packages": [
{
"epoch": 0,
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.31.0.dist-info/METADATA",
"name": "transformers",
"filePath": "usr/local/lib/python3.10/site-packages/setuptools-59.5.0.dist-info/METADATA",
"name": "setuptools",
"packageManager": "PYTHONPKG",
"version": "4.31.0"
},
{
"epoch": 0,
"filePath": "requirements.txt",
"name": "transformers",
"packageManager": "PIP",
"version": "4.31.0"
"version": "59.5.0"
}
]
},
Expand Down Expand Up @@ -197,17 +150,17 @@
"vulnerable_packages": [
{
"epoch": 0,
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.31.0.dist-info/METADATA",
"filePath": "usr/local/lib/python3.10/site-packages/transformers-4.36.0.dist-info/METADATA",
"name": "transformers",
"packageManager": "PYTHONPKG",
"version": "4.31.0"
"version": "4.36.0"
},
{
"epoch": 0,
"filePath": "requirements.txt",
"name": "transformers",
"packageManager": "PIP",
"version": "4.31.0"
"version": "4.36.0"
}
]
},
Expand Down Expand Up @@ -282,5 +235,30 @@
"version": "1.8.6"
}
]
},
"CVE-2024-6345": {
"description": "A vulnerability in the package_index module of pypa/setuptools versions up to 69.1.1 allows for remote code execution via its download functions. These functions, which are used to download packages from URLs provided by users or retrieved from package index servers, are susceptible to code injection. If these functions are exposed to user-controlled inputs, such as package URLs, they can execute arbitrary commands on the system. The issue is fixed in version 70.0.",
"remediation": {
"recommendation": {
"text": "None Provided"
}
},
"score": 0.0,
"score_details": {},
"severity": "UNTRIAGED",
"source": "NVD",
"source_url": "https://nvd.nist.gov/vuln/detail/CVE-2024-6345",
"status": "ACTIVE",
"title": "CVE-2024-6345 - setuptools",
"vulnerability_id": "CVE-2024-6345",
"vulnerable_packages": [
{
"epoch": 0,
"filePath": "usr/local/lib/python3.10/site-packages/setuptools-59.5.0.dist-info/METADATA",
"name": "setuptools",
"packageManager": "PYTHONPKG",
"version": "59.5.0"
}
]
}
}