This document is relevant for: Inf1

Install TensorFlow Neuron#

Note

  • Instructions in this page only apply to setting up Neuron components on Linux host running Ubuntu or Amazon Linux AMI.

  • For an example of how to install Neuron components in a container, see Tutorial Docker environment setup and our neuron-containers documentation for more details.

Develop on AWS ML accelerator instance#

The simplest environment setup for model development installs all Neuron SDK components directly on an AWS ML accelerator instance: the Neuron framework extensions, compiler, runtime, and tools. This will allow you to compile, execute, and performance tune your model, all in the same instance. This is the recommended workflow when first starting to work with Neuron device or when optimizing a model.

Note: If you are using a regular U18, U20, or AL2 AMI, follow the same setup instructions as the Base DLAMIs respectively.

Important

For successful installation or update to next releases (Neuron 1.20.0 and newer):
  • Uninstall aws-neuron-dkms by running: sudo apt remove aws-neuron-dkms or sudo yum remove aws-neuron-dkms

  • Install or upgrade to latest Neuron driver (aws-neuron-dkms) by following the “Setup Guide” instructions.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc] "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.10.1.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc] "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.10.1.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.9.3.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.9.3.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.9.3.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.9.3.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.8.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.8.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.8.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.8.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.7.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.7.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.7.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.7.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "src/helperscripts/n2-helper.py", line 1091, in <module>
    print(n2_manifest.generate_script(args))
  File "src/helperscripts/n2-helper.py", line 135, in generate_script
    str_python = self.set_python_venv(args)
  File "src/helperscripts/n2-helper.py", line 534, in set_python_venv
    packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
  File "src/helperscripts/n2-helper.py", line 84, in get_pip_packages_supporting_python_versions
    'supported_python_versions'].values[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "src/helperscripts/n2-helper.py", line 1091, in <module>
    print(n2_manifest.generate_script(args))
  File "src/helperscripts/n2-helper.py", line 135, in generate_script
    str_python = self.set_python_venv(args)
  File "src/helperscripts/n2-helper.py", line 534, in set_python_venv
    packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
  File "src/helperscripts/n2-helper.py", line 84, in get_pip_packages_supporting_python_versions
    'supported_python_versions'].values[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

Compile on compute instance#

If model compilation occurs outside the model deployment environment, you can install only the Neuron framework extensions and the compiler on any compute instance. This setup is helpful when compiling large complex models that require large amount of memory or during a CICD process where models are compiled in a separate step, prior to deployment.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc] "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc] "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.9.3.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.9.3.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.8.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.8.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.7.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.7.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "src/helperscripts/n2-helper.py", line 1091, in <module>
    print(n2_manifest.generate_script(args))
  File "src/helperscripts/n2-helper.py", line 135, in generate_script
    str_python = self.set_python_venv(args)
  File "src/helperscripts/n2-helper.py", line 534, in set_python_venv
    packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
  File "src/helperscripts/n2-helper.py", line 84, in get_pip_packages_supporting_python_versions
    'supported_python_versions'].values[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "src/helperscripts/n2-helper.py", line 1091, in <module>
    print(n2_manifest.generate_script(args))
  File "src/helperscripts/n2-helper.py", line 135, in generate_script
    str_python = self.set_python_venv(args)
  File "src/helperscripts/n2-helper.py", line 534, in set_python_venv
    packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
  File "src/helperscripts/n2-helper.py", line 84, in get_pip_packages_supporting_python_versions
    'supported_python_versions'].values[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

Deploy on AWS ML accelerator instance#

During deployment it can be beneficial to reduce the number of components installed in the system. For use-cases where only inference is necessary (compilation is already complete), only the framework and runtime should be installed.

Note: If you are using a regular U18, U20, or AL2 AMI, follow the same setup instructions as the Base DLAMIs respectively.

Important

For successful installation or update to next releases (Neuron 1.20.0 and newer):
  • Uninstall aws-neuron-dkms by running: sudo apt remove aws-neuron-dkms or sudo yum remove aws-neuron-dkms

  • Install or upgrade to latest Neuron driver (aws-neuron-dkms) by following the “Setup Guide” instructions.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc] "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.10.1.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc] "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.10.1.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.9.3.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.9.3.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.9.3.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.9.3.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.8.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.8.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.8.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.8.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo apt-get install -y python3.8-venv g++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.7.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo apt-get install tensorflow-model-server-neuronx=2.7.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

# Install Python venv 
sudo yum install -y python3.8-venv gcc-c++ 

# Create Python venv
python3.8 -m venv aws_neuron_venv_tensorflow_inf1 

# Activate Python venv 
source aws_neuron_venv_tensorflow_inf1/bin/activate 
python -m pip install -U pip 

# Install Jupyter notebook kernel
pip install ipykernel 
python3.8 -m ipykernel install --user --name aws_neuron_venv_tensorflow_inf1 --display-name "Python (tensorflow-neuron)"
pip install jupyter notebook
pip install environment_kernels

# Set pip repository pointing to the Neuron repository 
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

# Install TensorFlow Neuron
python -m pip install tensorflow-neuron[cc]==2.7.4.* "protobuf"

# Install Neuron TensorBoard
python -m pip install tensorboard-plugin-neuron

# Optional: Install Tensorflow Neuron model server
sudo yum install tensorflow-model-server-neuronx=2.7.4.2.10.8.0 -y

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "src/helperscripts/n2-helper.py", line 1091, in <module>
    print(n2_manifest.generate_script(args))
  File "src/helperscripts/n2-helper.py", line 135, in generate_script
    str_python = self.set_python_venv(args)
  File "src/helperscripts/n2-helper.py", line 534, in set_python_venv
    packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
  File "src/helperscripts/n2-helper.py", line 84, in get_pip_packages_supporting_python_versions
    'supported_python_versions'].values[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "src/helperscripts/n2-helper.py", line 1091, in <module>
    print(n2_manifest.generate_script(args))
  File "src/helperscripts/n2-helper.py", line 135, in generate_script
    str_python = self.set_python_venv(args)
  File "src/helperscripts/n2-helper.py", line 534, in set_python_venv
    packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
  File "src/helperscripts/n2-helper.py", line 84, in get_pip_packages_supporting_python_versions
    'supported_python_versions'].values[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

This document is relevant for: Inf1