Install Neuron PyTorch

Ubuntu 18 AMI (via Pip)

Configuring Linux for repository updates

. /etc/os-release
sudo tee /etc/apt/sources.list.d/neuron.list > /dev/null <<EOF
deb https://apt.repos.neuron.amazonaws.com ${VERSION_CODENAME} main
EOF

wget -qO - https://apt.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | sudo apt-key add -

sudo apt-get update -y
sudo apt-get install linux-headers-$(uname -r) -y
sudo apt-get install aws-neuron-dkms -y
sudo apt-get install aws-neuron-runtime-base -y
sudo apt-get install aws-neuron-runtime -y
sudo apt-get install aws-neuron-tools -y

export PATH=/opt/aws/neuron/bin:$PATH

Note

If you see the following errors during apt-get install, please wait a minute or so for background updates to finish and retry apt-get install:

E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?

Note

aws-neuron-dkms is a special kernel module package that has a dependency on the linux kernel version. This means that when the package is installed through apt, it will only be compatible with the linux kernel version that was running on the instance during the installation.

You have to re-install this package if you change the linux kernel version. The current kernel version can be checked using uname -r, and a list of the kernels that have dkms installed can be checked with dkms status | grep aws-neuron. Refer to the NRT Troubleshooting Guide for steps on how to re-install aws-neuron-dkms on a new kernel.

Install Neuron Pip Packages

It is recommended to use a virtual environment when installing Neuron pip packages. The following steps show how to setup the virtual environment on Ubuntu or Amazon Linux:

# Ubuntu
sudo apt-get update
sudo apt-get install -y python3-venv g++
# Amazon Linux
sudo yum update
sudo yum install -y python3 gcc-c++

Setup a new Python virtual environment:

python3 -m venv test_venv
source test_venv/bin/activate
pip install -U pip

Modify Pip repository configurations to point to the Neuron repository:

tee $VIRTUAL_ENV/pip.conf > /dev/null <<EOF
[global]
extra-index-url = https://pip.repos.neuron.amazonaws.com
EOF

Note

curl https://pip.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | gpg --import
pip download --no-deps neuron-cc
# The above shows you the name of the package downloaded
# Use it in the following command
wget https://pip.repos.neuron.amazonaws.com/neuron-cc/neuron_cc-<VERSION FROM FILE>.whl.asc
gpg --verify neuron_cc-<VERSION FROM FILE>.whl.asc neuron_cc-<VERSION FROM FILE>.whl

The following Pip installation commands assume you are using a virtual Python environment (see above for instructions on how to setup a virtual Python environment). If not using virtual Python environment, please switch ‘pip’ with ‘pip3’ as appropriate for your Python environment.

Install PyTorch

# NOTE: Make sure [tensorflow] option is provided during installation of neuron-cc for PyTorch-Neuron compilation; this is not necessary for PyTorch-Neuron inference.
pip install neuron-cc[tensorflow]
pip install torch-neuron

Previous Versions

Installing the default version of torch-neuron will use the latest supported torch version. Previous versions of torch may be supported using specific torch-neuron versions.

# Example: Install latest torch-neuron compatible with PyTorch 1.5
pip install "torch-neuron==1.5.*"

# Example: Install latest torch-neuron compatible with PyTorch 1.7
pip install "torch-neuron==1.7.*"

Amazon Linux2 AMI (via Pip)

Configuring Linux for repository updates

Verify the instance has kernel version 4.14 or latest and kernel headers are installed.

sudo tee /etc/yum.repos.d/neuron.repo > /dev/null <<EOF
[neuron]
name=Neuron YUM Repository
baseurl=https://yum.repos.neuron.amazonaws.com
enabled=1
metadata_expire=0
EOF

sudo rpm --import https://yum.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r) -y
sudo yum install aws-neuron-dkms -y
sudo yum install aws-neuron-runtime-base -y
sudo yum install aws-neuron-runtime -y
sudo yum install aws-neuron-tools -y

export PATH=/opt/aws/neuron/bin:$PATH

Note

aws-neuron-dkms is a special kernel module package that has a dependency on the linux kernel version. This means that when the package is installed through yum, it will only be compatible with the linux kernel version that was running on the instance during the installation.

You have to re-install this package if you change the linux kernel version. The current kernel version can be checked using uname -r, and a list of the kernels that have dkms installed can be checked with dkms status | grep aws-neuron. Refer to the NRT Troubleshooting Guide for steps on how to re-install aws-neuron-dkms on a new kernel.

Install Neuron Pip Packages

It is recommended to use a virtual environment when installing Neuron pip packages. The following steps show how to setup the virtual environment on Ubuntu or Amazon Linux:

# Ubuntu
sudo apt-get update
sudo apt-get install -y python3-venv g++
# Amazon Linux
sudo yum update
sudo yum install -y python3 gcc-c++

Setup a new Python virtual environment:

python3 -m venv test_venv
source test_venv/bin/activate
pip install -U pip

Modify Pip repository configurations to point to the Neuron repository:

tee $VIRTUAL_ENV/pip.conf > /dev/null <<EOF
[global]
extra-index-url = https://pip.repos.neuron.amazonaws.com
EOF

Note

curl https://pip.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | gpg --import
pip download --no-deps neuron-cc
# The above shows you the name of the package downloaded
# Use it in the following command
wget https://pip.repos.neuron.amazonaws.com/neuron-cc/neuron_cc-<VERSION FROM FILE>.whl.asc
gpg --verify neuron_cc-<VERSION FROM FILE>.whl.asc neuron_cc-<VERSION FROM FILE>.whl

The following Pip installation commands assume you are using a virtual Python environment (see above for instructions on how to setup a virtual Python environment). If not using virtual Python environment, please switch ‘pip’ with ‘pip3’ as appropriate for your Python environment.

Install PyTorch

# NOTE: Make sure [tensorflow] option is provided during installation of neuron-cc for PyTorch-Neuron compilation; this is not necessary for PyTorch-Neuron inference.
pip install neuron-cc[tensorflow]
pip install torch-neuron

Previous Versions

Installing the default version of torch-neuron will use the latest supported torch version. Previous versions of torch may be supported using specific torch-neuron versions.

# Example: Install latest torch-neuron compatible with PyTorch 1.5
pip install "torch-neuron==1.5.*"

# Example: Install latest torch-neuron compatible with PyTorch 1.7
pip install "torch-neuron==1.7.*"

Ubuntu or Amazon Linux2 AMI (via Conda)

The following commands assumes you are using a Conda environment and have already activated it. Please see https://docs.conda.io/projects/conda/en/latest/user-guide/install/ for installation instruction if Conda is not installed. The following steps are example steps to install and activate Conda environment:

curl -O https://repo.anaconda.com/miniconda/Miniconda3-4.7.12.1-Linux-x86_64.sh
echo "bfe34e1fa28d6d75a7ad05fd02fa5472275673d5f5621b77380898dee1be15d2 Miniconda3-4.7.12.1-Linux-x86_64.sh" | sha256sum --check
bash Miniconda3-4.7.12.1-Linux-x86_64.sh
source ~/.bashrc
conda create -q -y -n test_conda_env python=3.6
source activate test_conda_env
# Add Neuron Conda channel to Conda environment
conda config --env --add channels https://conda.repos.neuron.amazonaws.com
# If you are installing Torch-Neuron plus Neuron-Compiler
conda install torch-neuron

Previous Versions

Installing the default version of torch-neuron will use the latest supported torch version. Previous versions of torch may be supported using specific torch-neuron versions.

To view the supported versions of the given package, use:

# Show python/framework versions supported by torch-neuron
conda search torch-neuron

To install a specific version of a package use a version specifier:

# Example: Install latest torch-neuron compatible with PyTorch 1.5
conda install torch-neuron=1.5

# Example: Install latest torch-neuron compatible with PyTorch 1.7
conda install torch-neuron=1.7

Note

The framework Conda packages already include neuron-cc packages for compilation so there’s no need to install them separately.