How to Build a Neuron Container

Introduction

This document explains how to build a Neuron Container using an existing Dockerfile.

Pre-requisites

  1. Docker version 18 or newer is configured according to Docker environment setup

  2. Inf1 instance with available Neuron Devices

  3. If running a serving application such as tensorflow-model-server, torchserve or multi-model-server, make sure the appropriate ports that the server listens to are exposed using EXPOSE in the Dockerfile or the arguments -p 80:8080 on the docker run command.

Build and Run the Application Container

Follow the steps below for creating neuron application containers. If there were already existing containers that are packaged as per Packaging Container Applications using Neuron Runtime 1.x refer the Migration to Neuron Runtime 2.x (libnrt.so)

  1. Build the container using Dockerfile for Application Container

  2. Run the container locally:

docker run -it --name pt17 -p 80:8080 -e "AWS_NEURON_VISIBLE_DEVICES=ALL"  neuron-container:pytorch neuron-top

Important to know

Devices

There are currently two ways to specify Neuron Devices to a container.

  1. The docker native way is to use –device /dev/neuron# for each of the Neuron Devices intended to be passed. When using –device option ALL/all is not supported.

    docker run --device=/dev/neuron0 --device=/dev/neuron1

  2. If you install the aws-neuron-runtime-base package, you will have an OCI hook that also supports use of a container environment variable AWS_NEURON_VISIBLE_DEVICES=<ALL | csv of devices>, which intends to make things easier for multi device scenarios. Following are some examples

    docker run -e “AWS_NEURON_VISIBLE_DEVICES=0,1”
docker run -e “AWS_NEURON_VISIBLE_DEVICES=ALL”

  3. Multiple container applications running in the same host can share the devices but the cores cannot be shared. This is similar to running multiple applications in the host.