This document is relevant for: Inf1, Inf2, Trn1, Trn2

Neuron Containers#

In this section, you’ll find resources to help you use containers for accelerating your deep learning models on Inferentia and Trainium instances.

Quickstart: Configure and deploy a DLC with vLLM

Quickstart: Configure and deploy a Deep Learning Container (DLC) with vLLM

Get started by configuring and deploying a Deep Learning Container (DLC) with the AWS Neuron SDK and vLLM.

Supports: Inf1, Inf2, Trn1 / Trn1n, Trn2

Getting started with Neuron DLCs using Docker#

AWS Neuron Deep Learning Containers (DLCs) are a set of Docker images for training and serving models on AWS Trainium and Inferentia instances using AWS Neuron SDK. To build a Neuron container using Docker, please refer to Getting started with Neuron DLC using Docker.

Neuron Deep Learning Containers#

In most cases, it is recommended to use a preconfigured Deep Learning Container (DLC) from AWS. Each DLC is pre-configured to have all of the Neuron components installed and is specific to the chosen ML Framework. For more details on Neuron Deep Learning Containers, please refer to Neuron Deep Learning Containers.

Customize Neuron DLC#

Neuron DLC can be customized as needed. To learn more about how to customize the Neuron Deep Learning Container (DLC) to fit your specific project needs, please refer to Customize Neuron DLC.

Neuron Plugins for Containerized Environments#

Neuron provides plugins for better observability and fault tolerance. For more information on the plugins, please refer to Neuron Plugins for Containerized Environments.