This document is relevant for: Inf2, Trn1, Trn2, Trn3

Get started with nrtpy#

This page walks you through setting up a Python environment and installing nrtpy on a Neuron instance. When you are done, you will have nrtpy imported and ready to load NEFFs.

Before you begin#

You need a Trainium or Inferentia2 EC2 instance with the Neuron SDK installed. The simplest way to get started is to launch an instance using the latest Neuron Deep Learning AMI (DLAMI). For detailed instructions on launching and connecting to a Neuron instance, see Quickstart: Train a Model on Trainium.

Set up your environment#

Create a dedicated virtual environment for nrtpy:

python3 -m venv ~/nrtpy_venv
source ~/nrtpy_venv/bin/activate

Set the Neuron Runtime library path so nrtpy can find libnrt:

export LD_LIBRARY_PATH=/opt/aws/neuron/lib:$LD_LIBRARY_PATH

Tip

Add the export LD_LIBRARY_PATH line to ~/nrtpy_venv/bin/activate so it is set automatically each time you activate the environment.

Install nrtpy#

Install nrtpy from the Neuron pip repository:

pip install nrtpy --extra-index-url=https://pip.repos.neuron.amazonaws.com

For general information about installing Neuron packages, see Quickstart: Train a Model on Trainium.

Using nrtpy with NKI#

Warning

If you plan to use nrtpy to execute NEFFs compiled from NKI kernels, you will also need a separate NKI environment for compilation. The standalone nrtpy wheel and the nki wheel cannot be installed in the same Python environment due to a namespace conflict. In a future release, this limitation will be resolved.

Recommended setup:

Next steps#

Further reading#

This document is relevant for: Inf2, Trn1, Trn2, Trn3