This document is relevant for: Inf1, Inf2, Trn1, Trn2, Trn3

PyTorch on Inf1 (legacy)#

Warning

Legacy hardware: Inf1 instances use NeuronCore v1 with PyTorch 1.x (torch-neuron).

For new projects, use Inf2, Trn1, Trn2, or Trn3 with PyTorch 2.9+ (torch-neuronx). See Install PyTorch for Neuron for current PyTorch setup.

Key differences from current PyTorch#

Feature

Inf1 (torch-neuron)

Inf2, Trn1, Trn2, Trn3 (torch-neuronx)

PyTorch version

1.x

2.9+

Backend

PyTorch/XLA (torch_neuron)

Native Neuron (torch_neuronx)

Compilation

torch_neuron.trace()

torch.compile(backend='neuronx')

Training support

No

Yes

NeuronCore version

v1

v2

Setup instructions#

Launch Instance

Install Drivers and Tools

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Install PyTorch Neuron (torch-neuron)
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range
Visit PyTorch Neuron(torch-neuron) for Inference section
Visit PyTorch Neuron section for more

Launch Instance

Install Drivers and Tools

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Install PyTorch Neuron (torch-neuron)
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range
Visit PyTorch Neuron(torch-neuron) for Inference section
Visit PyTorch Neuron section for more

Launch Instance

Install Drivers and Tools

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Install PyTorch Neuron (torch-neuron)
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 1151, in <module>
    print(n2_manifest.generate_script(args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 121, in generate_script
    str_driver = self.install_neuron_driver(args)
  File "/home/docs/checkouts/readthedocs.org/user_builds/awsdocs-neuron/checkouts/v2.31.1/src/helperscripts/n2-helper.py", line 373, in install_neuron_driver
    driver_package = self.get_package_names(category='driver', instance=args.instance)[0]
IndexError: list index out of range
Visit PyTorch Neuron(torch-neuron) for Inference section
Visit PyTorch Neuron section for more

Update an Existing Installation#

If you already have a previous Neuron release installed, select the PyTorch version tab below to get the update commands.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

If you already have a previous Neuron release installed, select the PyTorch version tab below to get the update commands.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

If you already have a previous Neuron release installed, select the PyTorch version tab below to get the update commands.

Note

For a successful installation or update, execute each line of the instructions below separately or copy the contents of the code block into a script file and source its contents.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Previous Versions#

Use the tabs below to install a specific previous Neuron SDK release. Select the Neuron version you need.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Use the tabs below to install a specific previous Neuron SDK release. Select the Neuron version you need.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Use the tabs below to install a specific previous Neuron SDK release. Select the Neuron version you need.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Note

Inf1 instances are no longer supported. The installation and upgrade commands that were previously generated on this page have been removed. This document is retained for archival reference only.

Verification#

After installation, verify with:

import torch
import torch_neuron

print(f"torch-neuron version: {torch_neuron.__version__}")

neuron-ls

Next steps#

This document is relevant for: Inf1, Inf2, Trn1, Trn2, Trn3