This document is relevant for: Inf1

Select a Different Framework or Platform for Setup

TensorFlow Neuron (“tensorflow-neuron”) Setup on Ubuntu 20#

Get Started with Latest Release of TensorFlow Neuron (tensorflow-neuron)#

This section provide links that will assist you to quickly start with a fresh installation of Tensorflow Neuron (tensorflow-neuron) Setup for Inference.

Launch the Instance

  • Please follow the instructions at launch an Amazon EC2 Instance to launch an instance. When choosing the instance type at the EC2 console. please make sure to select the correct instance type.

  • To get more information about instances sizes and pricing see: Inf1 web page

  • Select Ubuntu Server 20 AMI

  • After launching the instance, follow the instructions in Connect to your instance to connect to the instance

Install Drivers and Tools
# Configure Linux for Neuron repository updates
. /etc/os-release
sudo tee /etc/apt/sources.list.d/neuron.list > /dev/null <<EOF
deb https://apt.repos.neuron.amazonaws.com ${VERSION_CODENAME} main
EOF
wget -qO - https://apt.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | sudo apt-key add -

# Update OS packages 
sudo apt-get update -y

# Install OS headers 
sudo apt-get install linux-headers-$(uname -r) -y

# Install git 
sudo apt-get install git -y

# install Neuron Driver
sudo apt-get install aws-neuronx-dkms=2.* -y

# Install Neuron Tools 
sudo apt-get install aws-neuronx-tools=2.* -y

# Add PATH
export PATH=/opt/aws/neuron/bin:$PATH
Install TensorFlow Neuron (tensorflow-neuron)
# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc] "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.10.1.2.10.19.0 -y
Visit TensorFlow Neuron(tensorflow-neuron) for Inference section
Visit TensorFlow Neuron section for more

Update to latest TensorFlow Neuron (tensorflow-neuron)#

If you already have a previous Neuron release installed, this section provide links that will assist you to update to latest Neuron release.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Update TensorFlow Neuron
python -m pip install --upgrade tensorflow-neuron[cc] "protobuf"

# Update Neuron TensorBoard
python -m pip install --upgrade tensorboard-plugin-neuron

# Optional: Update Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.10.1.2.10.19.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Update TensorFlow Neuron
python -m pip install --upgrade tensorflow-neuron[cc]==2.9.3.* "protobuf"

# Update Neuron TensorBoard
python -m pip install --upgrade tensorboard-plugin-neuron

# Optional: Update Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.9.3.2.10.19.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Update TensorFlow Neuron
python -m pip install --upgrade tensorflow-neuron[cc]==2.8.4.* "protobuf"

# Update Neuron TensorBoard
python -m pip install --upgrade tensorboard-plugin-neuron

# Optional: Update Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.8.4.2.10.19.0 -y

Install Previous Tensorflow Neuron Releases for Ubuntu (tensorflow-neuron)#

This section will assist you to install previous Neuron releases.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.10.1.2.10.8.0 "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.10.1.2.10.8.0 -y

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.10.1.2.10.8.0 "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.10.1.2.10.8.0 -y

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.10.1.2.10.2.0 "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.10.1.2.10.2.0 -y

This document is relevant for: Inf1