Install Neuron Apache MXNet (Incubating)

Ubuntu 18 AMI (via Pip)

Configuring Linux for repository updates

. /etc/os-release
sudo tee /etc/apt/sources.list.d/neuron.list > /dev/null <<EOF
deb https://apt.repos.neuron.amazonaws.com ${VERSION_CODENAME} main
EOF

wget -qO - https://apt.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | sudo apt-key add -

sudo apt-get update
sudo apt-get install linux-headers-$(uname -r)
sudo apt-get install aws-neuron-dkms
sudo apt-get install aws-neuron-runtime-base
sudo apt-get install aws-neuron-runtime
sudo apt-get install aws-neuron-tools

Note

If you see the following errors during apt-get install, please wait a minute or so for background updates to finish and retry apt-get install:

E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?

Note

aws-neuron-dkms is a special kernel module package that has a dependency on the linux kernel version. This means that when the package is installed through apt, it will only be compatible with the linux kernel version that was running on the instance during the installation.

You have to re-install this package if you change the linux kernel version. The current kernel version can be checked using uname -r, and a list of the kernels that have dkms installed can be checked with dkms status | grep aws-neuron. Refer to the NRT Troubleshooting Guide for steps on how to re-install aws-neuron-dkms on a new kernel.

Install Neuron Pip Packages

It is recommended to use a virtual environment when installing Neuron pip packages. The following steps show how to setup the virtual environment on Ubuntu or Amazon Linux:

# Ubuntu
sudo apt-get update
sudo apt-get install -y python3-venv g++
# Amazon Linux
sudo yum update
sudo yum install -y python3 gcc-c++

Setup a new Python virtual environment:

python3 -m venv test_venv
source test_venv/bin/activate
pip install -U pip

Modify Pip repository configurations to point to the Neuron repository:

tee $VIRTUAL_ENV/pip.conf > /dev/null <<EOF
[global]
extra-index-url = https://pip.repos.neuron.amazonaws.com
EOF

Note

curl https://pip.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | gpg --import
pip download --no-deps neuron-cc
# The above shows you the name of the package downloaded
# Use it in the following command
wget https://pip.repos.neuron.amazonaws.com/neuron-cc/neuron_cc-<VERSION FROM FILE>.whl.asc
gpg --verify neuron_cc-<VERSION FROM FILE>.whl.asc neuron_cc-<VERSION FROM FILE>.whl

The following Pip installation commands assume you are using a virtual Python environment (see above for instructions on how to setup a virtual Python environment). If not using virtual Python environment, please switch ‘pip’ with ‘pip3’ as appropriate for your Python environment.

Install MXNet

For MXNet 1.5:

pip install neuron-cc
pip install mxnet-neuron

For MXNet 1.8:

pip install neuron-cc
wget https://aws-mx-pypi.s3-us-west-2.amazonaws.com/1.8.0/aws_mx_cu110-1.8.0-py2.py3-none-manylinux2014_x86_64.whl .
pip install aws_mx_cu110-1.8.0-py2.py3-none-manylinux2014_x86_64.whl
pip install mx_neuron

Amazon Linux2 AMI (via Pip)

Configuring Linux for repository updates

Verify the instance has kernel version 4.14 or latest and kernel headers are installed.

sudo tee /etc/yum.repos.d/neuron.repo > /dev/null <<EOF
[neuron]
name=Neuron YUM Repository
baseurl=https://yum.repos.neuron.amazonaws.com
enabled=1
metadata_expire=0
EOF

sudo rpm --import https://yum.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB
sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
sudo yum install aws-neuron-dkms
sudo yum install aws-neuron-runtime-base
sudo yum install aws-neuron-runtime
sudo yum install aws-neuron-tools

Note

aws-neuron-dkms is a special kernel module package that has a dependency on the linux kernel version. This means that when the package is installed through yum, it will only be compatible with the linux kernel version that was running on the instance during the installation.

You have to re-install this package if you change the linux kernel version. The current kernel version can be checked using uname -r, and a list of the kernels that have dkms installed can be checked with dkms status | grep aws-neuron. Refer to the NRT Troubleshooting Guide for steps on how to re-install aws-neuron-dkms on a new kernel.

Install Neuron Pip Packages

It is recommended to use a virtual environment when installing Neuron pip packages. The following steps show how to setup the virtual environment on Ubuntu or Amazon Linux:

# Ubuntu
sudo apt-get update
sudo apt-get install -y python3-venv g++
# Amazon Linux
sudo yum update
sudo yum install -y python3 gcc-c++

Setup a new Python virtual environment:

python3 -m venv test_venv
source test_venv/bin/activate
pip install -U pip

Modify Pip repository configurations to point to the Neuron repository:

tee $VIRTUAL_ENV/pip.conf > /dev/null <<EOF
[global]
extra-index-url = https://pip.repos.neuron.amazonaws.com
EOF

Note

curl https://pip.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | gpg --import
pip download --no-deps neuron-cc
# The above shows you the name of the package downloaded
# Use it in the following command
wget https://pip.repos.neuron.amazonaws.com/neuron-cc/neuron_cc-<VERSION FROM FILE>.whl.asc
gpg --verify neuron_cc-<VERSION FROM FILE>.whl.asc neuron_cc-<VERSION FROM FILE>.whl

The following Pip installation commands assume you are using a virtual Python environment (see above for instructions on how to setup a virtual Python environment). If not using virtual Python environment, please switch ‘pip’ with ‘pip3’ as appropriate for your Python environment.

Install MXNet

For MXNet 1.5:

pip install neuron-cc
pip install mxnet-neuron

For MXNet 1.8:

pip install neuron-cc
wget https://aws-mx-pypi.s3-us-west-2.amazonaws.com/1.8.0/aws_mx_cu110-1.8.0-py2.py3-none-manylinux2014_x86_64.whl .
pip install aws_mx_cu110-1.8.0-py2.py3-none-manylinux2014_x86_64.whl
pip install mx_neuron

Ubuntu or Amazon Linux2 AMI (via Conda) (only for MXNet 1.5)

The following commands assumes you are using a Conda environment and have already activated it. Please see https://docs.conda.io/projects/conda/en/latest/user-guide/install/ for installation instruction if Conda is not installed. The following steps are example steps to install and activate Conda environment:

curl -O https://repo.anaconda.com/miniconda/Miniconda3-4.7.12.1-Linux-x86_64.sh
echo "bfe34e1fa28d6d75a7ad05fd02fa5472275673d5f5621b77380898dee1be15d2 Miniconda3-4.7.12.1-Linux-x86_64.sh" | sha256sum --check
bash Miniconda3-4.7.12.1-Linux-x86_64.sh
source ~/.bashrc
conda create -q -y -n test_conda_env python=3.6
source activate test_conda_env
# Add Neuron Conda channel to Conda environment
conda config --env --add channels https://conda.repos.neuron.amazonaws.com
# If you are installing MXNet-Neuron plus Neuron-Compiler
conda install mxnet-neuron

Note

The framework Conda packages already include neuron-cc packages for compilation so there’s no need to install them separately.