This document is relevant for: Inf2, Trn1, Trn2
PyTorch Neuron (“torch-neuronx”) Setup on Amazon Linux 2#
Note
As of 2.20.0, Neuron Runtime no longer supports AL2. Upgrade to AL2023 following the AL2 Migration guide
Note
Please make sure to upgrade from python 3.7 to python 3.8 to use Neuron SDK on Amazon Linux 2. Starting from Neuron Release 2.13, python 3.7 is no longer supported as mentioned here.
Also, we do not have support for torch-neuronx 2.1.2 on Amaznon Linux 2.
Get Started with Latest Release of PyTorch Neuron (torch-neuronx)#
This section provide links that will assist you to quickly start with a fresh installation of PyTorch Neuron for both Inference and Training.
Launch the Instance
Please follow the instructions at launch an Amazon EC2 Instance to launch an instance. When choosing the instance type at the EC2 console, please make sure to select the correct instance type.
To get more information about instances sizes and pricing see: Trn1 web page, Inf2 web page
Select Amazon Linux 2 AMI(HVM) - Kernel 5.10
When launching a Trn1, please adjust your primary EBS volume size to a minimum of 512GB.
After launching the instance, follow the instructions in Connect to your instance to connect to the instance
Install Drivers and Tools
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1137, in <module>
print(n2_manifest.generate_script(args))
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 138, in generate_script
str_python = self.set_python_venv(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 575, in set_python_venv
packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 82, in get_pip_packages_supporting_python_versions
framework_python_versions = df_framework.loc[
IndexError: index 0 is out of bounds for axis 0 with size 0
Install EFA (Applicable only for Trn1\Trn1n)
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1137, in <module>
print(n2_manifest.generate_script(args))
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 138, in generate_script
str_python = self.set_python_venv(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 575, in set_python_venv
packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 82, in get_pip_packages_supporting_python_versions
framework_python_versions = df_framework.loc[
IndexError: index 0 is out of bounds for axis 0 with size 0
Install PyTorch Neuron (torch-neuronx)
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1137, in <module>
print(n2_manifest.generate_script(args))
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 138, in generate_script
str_python = self.set_python_venv(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 575, in set_python_venv
packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 82, in get_pip_packages_supporting_python_versions
framework_python_versions = df_framework.loc[
IndexError: index 0 is out of bounds for axis 0 with size 0
Update to latest PyTorch NeuronX#
If you already have a previous Neuron release installed, this section provide links that will assist you to update to latest Neuron release.
Note
Instructions in this page only apply to setting up Neuron components on Linux host running Ubuntu or Amazon Linux AMI.
When launching a Trn1/Trn2, please adjust your primary EBS volume size to a minimum of 512GB.
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1137, in <module>
print(n2_manifest.generate_script(args))
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 138, in generate_script
str_python = self.set_python_venv(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 575, in set_python_venv
packages_supporting_python_versions = self.get_pip_packages_supporting_python_versions(args)
File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 82, in get_pip_packages_supporting_python_versions
framework_python_versions = df_framework.loc[
IndexError: index 0 is out of bounds for axis 0 with size 0
Install Previous PyTorch Neuron Releases for Amazon Linux (torch-neuronx)#
This section will assist you to install previous Neuron releases.
# Install Python venv
sudo yum install -y python3.8-venv gcc-c++
# Create Python venv
python3.8 -m venv aws_neuron_venv_pytorch
# Activate Python venv
source aws_neuron_venv_pytorch/bin/activate
python -m pip install -U pip
# Install Jupyter notebook kernel
pip install ipykernel
python3.8 -m ipykernel install --user --name aws_neuron_venv_pytorch --display-name "Python (torch-neuronx)"
pip install jupyter notebook
pip install environment_kernels
# Set pip repository pointing to the Neuron repository
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
# Install wget, awscli
python -m pip install wget
python -m pip install awscli
# Install Neuron Compiler and Framework
python -m pip install neuronx-cc==2.13.66.0 torch-neuronx==1.13.1.1.14.0
# Install Python venv
sudo yum install -y python3.8-venv gcc-c++
# Create Python venv
python3.8 -m venv aws_neuron_venv_pytorch
# Activate Python venv
source aws_neuron_venv_pytorch/bin/activate
python -m pip install -U pip
# Install Jupyter notebook kernel
pip install ipykernel
python3.8 -m ipykernel install --user --name aws_neuron_venv_pytorch --display-name "Python (torch-neuronx)"
pip install jupyter notebook
pip install environment_kernels
# Set pip repository pointing to the Neuron repository
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
# Install wget, awscli
python -m pip install wget
python -m pip install awscli
# Install Neuron Compiler and Framework
python -m pip install neuronx-cc==2.12.68.0 torch-neuronx==1.13.1.1.13.1
# Install Python venv
sudo yum install -y python3.8-venv gcc-c++
# Create Python venv
python3.8 -m venv aws_neuron_venv_pytorch
# Activate Python venv
source aws_neuron_venv_pytorch/bin/activate
python -m pip install -U pip
# Install Jupyter notebook kernel
pip install ipykernel
python3.8 -m ipykernel install --user --name aws_neuron_venv_pytorch --display-name "Python (torch-neuronx)"
pip install jupyter notebook
pip install environment_kernels
# Set pip repository pointing to the Neuron repository
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
# Install wget, awscli
python -m pip install wget
python -m pip install awscli
# Install Neuron Compiler and Framework
python -m pip install neuronx-cc==2.12.54.0 torch-neuronx==1.13.1.1.13.0
This document is relevant for: Inf2, Trn1, Trn2