This document is relevant for: Inf1

Update to latest TensorFlow Neuron#

Note

  • Instructions in this page only apply to setting up Neuron components on Linux host running Ubuntu or Amazon Linux AMI.

  • For an example of how to install Neuron components in a container, see Tutorial Docker environment setup and our neuron-containers documentation for more details.

Develop on AWS ML accelerator instance#

The simplest environment setup for model development installs all Neuron SDK components directly on an AWS ML accelerator instance: the Neuron framework extensions, compiler, runtime, and tools. This will allow you to compile, execute, and performance tune your model, all in the same instance. This is the recommended workflow when first starting to work with Neuron device or when optimizing a model.

Important

For successful installation or update to next releases (Neuron 1.20.0 and newer):
  • Uninstall aws-neuron-dkms by running: sudo apt remove aws-neuron-dkms or sudo yum remove aws-neuron-dkms

  • Install or upgrade to latest Neuron driver (aws-neuron-dkms) by following the “Setup Guide” instructions.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo apt-get install aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo yum update aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum update tensorflow-model-server-neuronx -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms --allow-change-held-packages -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo apt-get install aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx --allow-change-held-packages -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum versionlock delete aws-neuronx-dkms
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo yum update aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum versionlock delete tensorflow-model-server-neuronx
sudo yum update tensorflow-model-server-neuronx -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo apt-get install aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=2.5.4.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo yum update aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum update tensorflow-model-server-neuronx-2.5.4.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms --allow-change-held-packages -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo apt-get install aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=2.5.4.2.5.6.0 --allow-change-held-packages -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum versionlock delete aws-neuronx-dkms
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo yum update aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum versionlock delete tensorflow-model-server-neuronx
sudo yum update tensorflow-model-server-neuronx-2.5.4.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo apt-get install aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=1.15.0.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo yum update aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum update tensorflow-model-server-neuronx-1.15.0.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms --allow-change-held-packages -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo apt-get install aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate TensorFlow
source activate aws_neuron_tensorflow_p36

# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=1.15.0.2.5.6.0 --allow-change-held-packages -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum versionlock delete aws-neuronx-dkms
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Update Neuron Tools
sudo yum update aws-neuronx-tools -y

# Update Neuron TensorBoard
pip install --upgrade tensorboard-plugin-neuron

# Activate TensorFlow
source activate aws_neuron_tensorflow_p36

# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum versionlock delete tensorflow-model-server-neuronx
sudo yum update tensorflow-model-server-neuronx-1.15.0.2.5.6.0 -y

Compile on compute instance#

If model compilation occurs outside the model deployment environment, you can install only the Neuron framework extensions and the compiler on any compute instance. This setup is helpful when compiling large complex models that require large amount of memory or during a CICD process where models are compiled in a separate step, prior to deployment.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc] "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron[cc]==2.5.3.* "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate TensorFlow
source activate aws_neuron_tensorflow_p36

# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Activate TensorFlow
source activate aws_neuron_tensorflow_p36

# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* neuron-cc "protobuf"

Deploy on AWS ML accelerator instance#

During deployment it can be beneficial to reduce the number of components installed in the system. For use-cases where only inference is necessary (compilation is already complete), only the framework and runtime should be installed.

Important

For successful installation or update to next releases (Neuron 1.20.0 and newer):
  • Uninstall aws-neuron-dkms by running: sudo apt remove aws-neuron-dkms or sudo yum remove aws-neuron-dkms

  • Install or upgrade to latest Neuron driver (aws-neuron-dkms) by following the “Setup Guide” instructions.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum update tensorflow-model-server-neuronx -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms --allow-change-held-packages -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx --allow-change-held-packages -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum versionlock delete aws-neuronx-dkms
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum versionlock delete tensorflow-model-server-neuronx
sudo yum update tensorflow-model-server-neuronx -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.7.1 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

__main__ : error:   tensorflow-2.6.3 is not a supported framework

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=2.5.4.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum update tensorflow-model-server-neuronx-2.5.4.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms --allow-change-held-packages -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=2.5.4.2.5.6.0 --allow-change-held-packages -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum versionlock delete aws-neuronx-dkms
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==2.5.3.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum versionlock delete tensorflow-model-server-neuronx
sudo yum update tensorflow-model-server-neuronx-2.5.4.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=1.15.0.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate a Python 3.7 virtual environment where Neuron pip packages were installed 
source tensorflow_venv/bin/activate


# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum update tensorflow-model-server-neuronx-1.15.0.2.5.6.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo apt-get update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo apt-get install linux-headers-$(uname -r) -y

# Update Neuron Driver
sudo apt-get install aws-neuronx-dkms --allow-change-held-packages -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate TensorFlow
source activate aws_neuron_tensorflow_p36

# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo apt-get install tensorflow-model-server-neuronx=1.15.0.2.5.6.0 --allow-change-held-packages -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Update OS packages
sudo yum update -y

################################################################################################################
# To install or update to Neuron versions 1.19.1 and newer from previous releases:
# - DO NOT skip 'aws-neuron-dkms' install or upgrade step, you MUST install or upgrade to latest Neuron driver
################################################################################################################

# Update OS headers
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y

# Update Neuron Driver
sudo yum versionlock delete aws-neuronx-dkms
sudo yum update aws-neuronx-dkms -y

####################################################################################
# Warning: If Linux kernel is updated as a result of OS package update
#          Neuron driver (aws-neuron-dkms) should be re-installed after reboot
####################################################################################

# Activate TensorFlow
source activate aws_neuron_tensorflow_p36

# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Update Neuron TensorFlow
pip install --upgrade tensorflow-neuron==1.15.5.* "protobuf"

# Optional: Update Neuron TensorFlow model server
sudo yum versionlock delete tensorflow-model-server-neuronx
sudo yum update tensorflow-model-server-neuronx-1.15.0.2.5.6.0 -y

This document is relevant for: Inf1