This document is relevant for: Inf1, Inf2, Trn1, Trn2, Trn3

PyTorch on Inf1 (legacy)

Warning

Legacy hardware: Inf1 instances use NeuronCore v1 with PyTorch 1.x (torch-neuron).

For new projects, use Inf2, Trn1, Trn2, or Trn3 with PyTorch 2.9+ (torch-neuronx). See Install PyTorch for Neuron for current PyTorch setup.

Key differences from current PyTorch

Feature	Inf1 (torch-neuron)	Inf2, Trn1, Trn2, Trn3 (torch-neuronx)
PyTorch version	1.x	2.9+
Backend	PyTorch/XLA (torch_neuron)	Native Neuron (torch_neuronx)
Compilation	torch_neuron.trace()	torch.compile(backend='neuronx')
Training support	No	Yes
NeuronCore version	v1	v2

Setup instructions

Ubuntu 20.04

Launch Instance

• Launch an Amazon EC2 Instance and select an Inf1 instance type.

• Select Ubuntu Server 20 AMI.

• Connect to your instance.

Install Drivers and Tools

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Install PyTorch Neuron (torch-neuron)

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Visit PyTorch Neuron(torch-neuron) for Inference section

Visit PyTorch Neuron section for more

Ubuntu 22.04

Launch Instance

• Launch an Amazon EC2 Instance and select an Inf1 instance type.

• Select Ubuntu Server 22 AMI.

• Connect to your instance.

Install Drivers and Tools

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Install PyTorch Neuron (torch-neuron)

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Visit PyTorch Neuron(torch-neuron) for Inference section

Visit PyTorch Neuron section for more

Amazon Linux 2023

Launch Instance

• Launch an Amazon EC2 Instance and select an Inf1 instance type.

• Select Amazon Linux 2023 AMI.

• Connect to your instance.

Install Drivers and Tools

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Install PyTorch Neuron (torch-neuron)

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Visit PyTorch Neuron(torch-neuron) for Inference section

Visit PyTorch Neuron section for more

Update an Existing Installation

Ubuntu 20.04

If you already have a previous Neuron release installed, select the PyTorch version tab below to get the update commands.

PyTorch 1.13.1

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Ubuntu 22.04

If you already have a previous Neuron release installed, select the PyTorch version tab below to get the update commands.

PyTorch 1.13.1

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Amazon Linux 2023

If you already have a previous Neuron release installed, select the PyTorch version tab below to get the update commands.

PyTorch 1.13.1

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Previous Versions

Ubuntu 20.04

Use the tabs below to install a specific previous Neuron SDK release. Select the Neuron version you need.

Neuron 2.21.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Neuron 2.20.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Neuron 2.19.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Ubuntu 22.04

Use the tabs below to install a specific previous Neuron SDK release. Select the Neuron version you need.

Neuron 2.21.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Neuron 2.20.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Neuron 2.19.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Amazon Linux 2023

Use the tabs below to install a specific previous Neuron SDK release. Select the Neuron version you need.

Neuron 2.21.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Neuron 2.20.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Neuron 2.19.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/latest/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range

Verification

After installation, verify with:

import torch
import torch_neuron

print(f"torch-neuron version: {torch_neuron.__version__}")

neuron-ls

Next steps

• API Reference Guide (torch-neuron) - torch-neuron API reference

• Inference with torch-neuronx (Inf2 & Trn1/Trn2) - Inference guides

• Install AWS Neuron SDK - Current setup options (Inf2, Trn1, Trn2, Trn3)

This document is relevant for: Inf1, Inf2, Trn1, Trn2, Trn3