This document is relevant for: Inf1, Trn1

Get Started with TensorFlow Neuron

This page provide links that will assist you to quickly start with TensorFlow Neuron.

Get Started with Inference

Launch Inf1 Instance

• Please follow the instructions at launch an Amazon EC2 Instance to Launch an Inf1 instance, when choosing the instance type at the EC2 console. Please make sure to select the correct instance type. To get more information about Inf1 instances sizes and pricing see Inf1 web page.

• Select your Amazon Machine Image (AMI) of choice, please note that Neuron support Amazon Linux 2 AMI(HVM) - Kernel 5.10

• After launching the instance, follow the instructions in Connect to your instance to connect to the instance

Install Drivers and Tools

# Configure Linux for Neuron repository updates
sudo tee /etc/yum.repos.d/neuron.repo > /dev/null <<EOF
[neuron]
name=Neuron YUM Repository
baseurl=https://yum.repos.neuron.amazonaws.com
enabled=1
metadata_expire=0
EOF
sudo rpm --import https://yum.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB

# Update OS packages
sudo yum update -y

################################################################################################################
# Remove older versions of Neuron
################################################################################################################
sudo yum remove aws-neuron-dkms -y
sudo yum remove aws-neuronx-dkms -y
sudo yum remove aws-neuron-tools -y
sudo yum remove aws-neuronx-tools -y

################################################################################################################
# To install or update to Neuron versions 2.5 and newer from previous releases:
# - DO NOT skip 'aws-neuronx-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Install OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Install Neuron Driver
sudo yum install aws-neuronx-dkms-2.* -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Install Neuron Tools
sudo yum install aws-neuronx-tools=2.* -y

export PATH=/opt/aws/neuron/bin:$PATH

Install TensorFlow Neuron (tensorflow-neuron)

# Install Python venv and activate Python virtual environment to install
# Neuron pip packages.
sudo yum install -y python3.7-venv gcc-c++
python3.7 -m venv tensorflow_venv
source tensorflow_venv/bin/activate
pip install -U pip


# Install Jupyter notebook kernel
pip install ipykernel
python3.7 -m ipykernel install --user --name tensorflow_venv --display-name "Python (Neuron TensorFlow)"
pip install jupyter notebook
pip install environment_kernels


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Install Neuron TensorFlow
pip install tensorflow-neuron[cc] "protobuf==3.20.1"

# Optional: Install Neuron TensorFlow model server
sudo yum install tensorflow-model-server-neuron -y

# Install Neuron TensorBoard
pip install tensorboard-plugin-neuron

Run Tutorial

ResNet-50

Visit TensorFlow Neuron section for more

Get Started with Training

Note

TensorFlow Neuron support for training workloads is coming soon.

This document is relevant for: Inf1, Trn1