Skip to content

Commit

Permalink
[documentation] Add Container overview page. (#481)
Browse files Browse the repository at this point in the history
* add new overview

* latest only

* fix code intents

* correct versions
  • Loading branch information
philschmid authored Feb 15, 2024
1 parent 49b14d0 commit ee0c1f4
Show file tree
Hide file tree
Showing 2 changed files with 31 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/source/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
title: Installation
- local: quickstart
title: Quickstart
- local: containers
title: Optimum Containers
- sections:
- local: tutorials/overview
title: Overview
Expand Down
29 changes: 29 additions & 0 deletions docs/source/containers.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
<!--Copyright 2023 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Optimum Neuron Container

We provide pre-built Optimum Neuron containers for Amazon SageMaker. These containers come with all of the Hugging Face libraries and dependencies pre-installed, so you can start using them right away.
We have containers for training and inference, and optimized text generation containers with TGI. The table is up to date and only includes the latest versions of each container. You can find older versions in the [Deep Learning Container Release Notes](https://github.com/aws/deep-learning-containers/releases?q=hf-neuronx&expanded=true)

We recommend using the `sagemaker` Python SDK to retrieve the image URI for the container you want to use.

## Available Optimum Neuron Containers

| Type | Optimum Version | Image URI |
|-----------------------------|-----------------|---------------------------------------------|
| Training | 0.0.13 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training-neuronx:1.13.1-neuronx-py310-sdk2.15.0-ubuntu20.04` |
| Inference | 0.0.13 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-inference-neuronx:1.13.1-neuronx-py310-sdk2.15.0-ubuntu20.04` |
| Text Generation Inference | 0.0.17 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-tgi-inference:1.13.1-optimum0.0.17-neuronx-py310-ubuntu22.04` |


Please replace `763104351884` with the correct [AWS account ID](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/image_uri_config/huggingface-neuronx.json) and `region` with the AWS region you are working in.

0 comments on commit ee0c1f4

Please sign in to comment.