DLAMI with Neuron Release Notes

For more information about using Neuron with Conda and Base DLAMI, please see https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-inferentia.html.

[DLAMI v43.0]

Resolved Issues

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.3.8.0 (includes neuron_cc-1.2.7.0)

conda package tensorflow-neuron-1.15.5.1.2.9.0 (includes neuron_cc-1.2.7.0)

conda package torch-neuron-1.7.1.1.2.16.0 (includes neuron_cc-1.2.7.0)

aws-neuron-dkms-1.4.5.0.

aws-neuron-runtime-base-1.4.8.0

aws-neuron-runtime-1.4.12.0

aws-neuron-tools-1.4.12.0

tensorflow-model-server-neuron-1.15.0.1.2.9.0

[DLAMI v42.2]

Resolved Issues

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.3.8.0 (includes neuron_cc-1.2.7.0)

conda package tensorflow-neuron-1.15.5.1.2.9.0 (includes neuron_cc-1.2.7.0)

conda package torch-neuron-1.7.1.1.2.16.0 (includes neuron_cc-1.2.7.0)

aws-neuron-dkms-1.4.5.0.

aws-neuron-runtime-base-1.4.8.0

aws-neuron-runtime-1.4.12.0

aws-neuron-tools-1.4.12.0

tensorflow-model-server-neuron-1.15.0.1.2.9.0

[DLAMI v41]

Resolved Issues

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.3.7.0 (includes neuron_cc-1.2.7.0)

conda package tensorflow-neuron-1.15.5.1.2.8.0 (includes neuron_cc-1.2.7.0)

conda package torch-neuron-1.7.1.1.2.15.0 (includes neuron_cc-1.2.7.0)

aws-neuron-dkms-1.4.1.0.

aws-neuron-runtime-base-1.4.2.0

aws-neuron-runtime-1.4.3.0

aws-neuron-tools-1.4.2.0

tensorflow-model-server-neuron-1.15.0.1.2.8.0

[DLAMI v40]

Resolved Issues

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.2.1.0 (includes neuron_cc-1.1.7.0)

conda package tensorflow-neuron-1.15.4.1.1.3.0 (includes neuron_cc-1.1.7.0)

conda package torch-neuron-1.7.1.1.1.7.0 (includes neuron_cc-1.1.7.0)

aws-neuron-dkms-1.3.2.0.

aws-neuron-runtime-base-1.3.2.0

aws-neuron-runtime-1.3.1.0

aws-neuron-tools-1.3.1.0

tensorflow-model-server-neuron-1.15.0.1.1.3.0

[DLAMI v39]

Resolved Issues

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.2.1.0 (includes neuron_cc-1.1.7.0)

conda package tensorflow-neuron-1.15.4.1.1.3.0 (includes neuron_cc-1.1.7.0)

conda package torch-neuron-1.7.1.1.1.7.0 (includes neuron_cc-1.1.7.0)

aws-neuron-dkms-1.3.2.0.

aws-neuron-runtime-base-1.3.2.0

aws-neuron-runtime-1.3.1.0

aws-neuron-tools-1.3.1.0

tensorflow-model-server-neuron-1.15.0.1.1.3.0

[DLAMI v38]

Resolved Issues

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.1.88.0 (includes neuron_cc-1.0.24045.0)

conda package tensorflow-neuron-1.15.4.1.0.2168.0 (includes neuron_cc-1.0.24045.0)

conda package torch-neuron-1.5.1.1.0.1978.0 (includes neuron_cc-1.0.24045.0)

aws-neuron-dkms-1.2

aws-neuron-runtime-base-1.2.0.0

aws-neuron-runtime-1.2.5.0

aws-neuron-tools-1.2.7.0

tensorflow-model-server-neuron-1.15.0.1.0.2168.0

[DLAMI v37]

Resolved Issues

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.1.88.0 (includes neuron_cc-1.0.24045.0)

conda package tensorflow-neuron-1.15.4.1.0.2168.0 (includes neuron_cc-1.0.24045.0)

conda package torch-neuron-1.5.1.1.0.1978.0 (includes neuron_cc-1.0.24045.0)

aws-neuron-dkms-1.2

aws-neuron-runtime-base-1.2.0.0

aws-neuron-runtime-1.2.5.0

aws-neuron-tools-1.2.7.0

tensorflow-model-server-neuron-1.15.0.1.0.2168.0

[DLAMI v36]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.1.52.0_2.0.757.0 (includes neuron_cc-1.0.20600.0)

conda package tensorflow-neuron-1.15.3.1.0.2043.0_2.0.894.0 (includes neuron_cc-1.0.20600.0)

conda package torch-neuron-1.5.1.1.0.298.0_2.0.880.0 (includes neuron-cc-1.0.18001.0)

aws-neuron-dkms-1.1

aws-neuron-runtime-base-1.1.1.0

aws-neuron-runtime-1.1.1402.0

aws-neuron-tools-1.1.228.0

tensorflow-model-server-neuron-1.15.0.1.0.2043.0

[DLAMI v35]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.1.52.0_2.0.757.0

conda package tensorflow-neuron-1.15.3.1.0.2043.0_2.0.894.0

conda package torch-neuron-1.5.1.1.0.298.0_2.0.880.0

aws-neuron-runtime-base-1.0.8126.0

aws-neuron-runtime-1.0.9592.0

aws-neuron-tools-1.0.11054.0

tensorflow-model-server-neuron-1.15.0.1.0.2043.0

[DLAMI v34]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.1.1.0_2.0.651.0

conda package tensorflow-neuron-1.15.3.1.0.1965.0_2.0.778.0

conda package torch-neuron-1.5.1.1.0.298.0_2.0.880.0

aws-neuron-runtime-base-1.0.7803.0

aws-neuron-runtime-1.0.9197.0

aws-neuron-tools-1.0.10616.0

tensorflow-model-server-neuron-1.15.0.1.0.1965.0

[DLAMI v33]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.1.1.0_2.0.651.0

conda package tensorflow-neuron-1.15.3.1.0.1965.0_2.0.778.0

conda package torch-neuron-1.5.1.1.0.298.0_2.0.880.0

aws-neuron-runtime-base-1.0.7803.0

aws-neuron-runtime-1.0.8896.0

aws-neuron-tools-1.0.10272.0

tensorflow-model-server-neuron-1.15.0.1.0.1965.0

[DLAMI v32]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.2101.0_2.0.631.0

conda package tensorflow-neuron-1.15.3.1.0.1953.0_2.0.769.0

conda package torch-neuron-1.5.1.1.0.258.0_2.0.871.0

aws-neuron-runtime-base-1.0.7618.0

aws-neuron-runtime-1.0.8813.0

aws-neuron-tools-1.0.10182.0

tensorflow-model-server-neuron-1.15.0.1.0.1953.0

[DLAMI v31]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.2093.0_2.0.579.0

conda package tensorflow-neuron-1.15.3.1.0.1891.0_2.0.706.0

conda package torch-neuron-1.5.1.1.0.251.0_2.0.783.0

aws-neuron-runtime-base-1.0.7395.0

aws-neuron-runtime-1.0.8032.0

aws-neuron-tools-1.0.9171.0

tensorflow-model-server-neuron-1.15.0.1.0.1796.0

[DLAMI v30.1 (Amazon Linux 2)]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.2033.0_2.0.489.0

conda package tensorflow-neuron-1.15.2.1.0.1796.0_2.0.593.0

conda package torch-neuron-1.3.0.1.0.215.0_2.0.633.0

aws-neuron-k8-plugin-1.0.9171.0

aws-neuron-runtime-base-1.0.7395.0

aws-neuron-runtime-1.0.8032.0

aws-neuron-tools-1.0.9171.0

tensorflow-model-server-neuron-1.15.0.1.0.1796.0

[DLAMI v30.0 (Ubuntu 16 and Ubuntu 18)]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.2033.0_2.0.489.0

conda package tensorflow-neuron-1.15.2.1.0.1796.0_2.0.593.0

conda package torch-neuron-1.3.0.1.0.215.0_2.0.633.0

aws-neuron-k8-plugin-1.0.9171.0

aws-neuron-runtime-base-1.0.7295.0

aws-neuron-runtime-1.0.7865.0

aws-neuron-tools-1.0.9043.0

tensorflow-model-server-neuron-1.15.0.1.0.1796.0

[DLAMI v29.0]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.1498.0_1.0.918.0

conda package tensorflow-neuron-1.15.0.1.0.1240.0_1.0.918.0

conda package torch-neuron-1.3.0.1.0.170.0_2.0.349.0

aws-neuron-runtime-base-1.0.7173.0

aws-neuron-runtime-1.0.6905.0

aws-neuron-tools-1.0.8550.0

tensorflow-model-server-neuron-1.15.0.1.0.1572.0

[DLAMI v28.0]

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.1498.0_1.0.918.0

conda package tensorflow-neuron-1.15.0.1.0.1240.0_1.0.918.0

conda package torch-neuron-1.3.0.1.0.90.0_1.0.918.0

aws-neuron-runtime-base-1.0.6554.0

aws-neuron-runtime-1.0.6222.0

aws-neuron-tools-1.0.6554.0

tensorflow-model-server-neuron-1.15.0.1.0.1333.0

[DLAMI v27.0]

This DLAMI release incorporates all content in the releases for Neuron up to and including the Feb 27, 2020 SDK release set.

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.1498.0_1.0.918.0

conda package tensorflow-neuron-1.15.0.1.0.1240.0_1.0.918.0

conda package torch-neuron-1.3.0.1.0.90.0_1.0.918.0

aws-neuron-runtime-base-1.0.5832.0

aws-neuron-runtime-1.0.5795.0

aws-neuron-tools-1.0.5832.0

tensorflow-model-server-neuron-1.15.0.1.0.1240.0

Resolved issues

  • To update Conda package in Conda DLAMI v27.0 and up, simply do “conda update tensorflow-neuron” within Conda environment aws_neuron_tensorflow_p36. There’s no need to install Numpy version 1.17.2 as in DLAMI v26.0.

Updating

  • It is strongly encouraged to update all packages to most recent release. If using Conda environments, please use “conda update” instead of “pip install” within the respective environment:

Base and Conda DLAMI on Ubuntu:

sudo apt-get update
sudo apt-get install aws-neuron-runtime-base
sudo apt-get install aws-neuron-runtime
sudo apt-get install aws-neuron-tools
sudo apt-get install tensorflow-model-server-neuron

Base and Conda DLAMI on Amazon Linux:

sudo yum install aws-neuron-runtime-base
sudo yum install aws-neuron-runtime
sudo yum install aws-neuron-tools
sudo yum install tensorflow-model-server-neuron

Conda DLAMI:

# MXNet-Neuron Conda environment
source activate aws_neuron_mxnet_p36
conda update mxnet-neuron
# TensorFlow-Neuron Conda environment
source activate aws_neuron_tensorflow_p36
conda update tensorflow-neuron
# PyTorch-Neuron Conda environment
source activate aws_neuron_pytorch_p36
conda update torch-neuron

[DLAMI v26.0]

NOTE: It is strongly encouraged to update all packages to most recent release. If using Conda environments, please use “conda update” instead of “pip install” within the respective environment:

Supported Operating Systems:

Amazon Linux 2

Ubuntu 16

Ubuntu 18

Versions of Neuron packages included:

conda package mxnet-neuron-1.5.1.1.0.1260.0_1.0.298.0

conda package tensorflow-neuron-1.15.0.1.0.663.0_1.0.298.0

aws-neuron-runtime-base-1.0.3657.0

aws-neuron-runtime-1.0.4109.0

aws-neuron-tools-1.0.3657.0

tensorflow-model-server-neuron-1.15.0.1.0.663.0

Known Issues

Installation Guidelines

Base and Conda DLAMI on Ubuntu:

sudo apt-get update
sudo apt-get install aws-neuron-runtime-base
sudo apt-get install aws-neuron-runtime
sudo apt-get install aws-neuron-tools
sudo apt-get install tensorflow-model-server-neuron

Base and Conda DLAMI on Amazon Linux:

sudo yum install aws-neuron-runtime-base
sudo yum install aws-neuron-runtime
sudo yum install aws-neuron-tools
sudo yum install tensorflow-model-server-neuron

Conda DLAMI:

# MXNet-Neuron Conda environment
source activate aws_neuron_mxnet_p36
conda update mxnet-neuron
# TensorFlow-Neuron Conda environment (DLAMI v26)
source activate aws_neuron_tensorflow_p36
conda install numpy=1.17.2 --yes --quiet
conda update tensorflow-neuron
  • In TensorFlow-Neuron conda environment (aws_neuron_tensorflow_p36), the installed numpy version prevents update to latest conda package version. Please do “conda install numpy=1.17.2 –yes –quiet” before “conda update tensorflow-neuron”.

  • When using the Conda DLAMI, use the above conda commands to update packages, not pip.

  • When doing conda update aws_neuron_tensorflow in the aws_neuron_tensorflow_p36 environment or when using pip install, you will see the following warning which can be ignored: “neuron-cc has requirement numpy<=1.17.2,>=1.13.3, but you’ll have numpy 1.17.4 which is incompatible.””

  • Customers experiencing 404 errors from https://yum.repos.neuron.amazonaws.com during yum updates will need to remake their yum HTTP caches as shown in the code below this bullet. It’s also encouraged to configure the Neuron repository for immediate metadata expiration to avoid the 404 errors in the future as shown here: Setup Guide

# refresh yum HTTP cache:
sudo yum makecache
  • If using Base DLAMI and installing tensorflow-neuron outside of Conda or virtual environment, the package ‘wrapt’ may cause an error during installation using Pip. In this case an error like this will occur:

ERROR: Cannot uninstall 'wrapt'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
    • To resolve this, execute:

python3 -m pip install wrapt --ignore-installed
python3 -m pip install tensorflow-neuron
  • The tensorflow-neuron conda package comes with:
    • TensorBoard-Neuron for Neuron v1.12.2 release and earlier (tensorflow-neuron<=1.15.5.1.2.9.0)

    • the Neuron plugin for TensorBoard for Neuron v1.13.0 release and later

  • There is no standalone tensorboard-neuron or tensorboard-plugin-neuron package at this time.

For more information, please see Known Issues and Limitations - updated 08/12/2021.