This document is relevant for: Inf1, Trn1

What’s New#

Neuron 2.6.0 (12/12/2022)#

What’s New#

This release introduces the support of PyTorch 1.12 version, and introduces PyTorch Neuron (torch-neuronx) profiling through Neuron Plugin for TensorBoard. Pytorch Neuron (torch-neuronx) users can now profile their models through the following TensorBoard views:

  • Operator Framework View

  • Operator HLO View

  • Operator Trace View

This release introduces the support of LAMB optimizer for FP32 mode, and adds support for capturing snapshots of inputs, outputs and graph HLO for debugging.

In addition, this release introduces the support of new operators and resolves issues that improve stability for Trn1 customers.

For more detailed release notes of the new features and resolved issues, see Neuron Components Release Notes.

Neuron Components Release Notes#

Inf1 and Trn1 common packages#

Component

Instance/s

Package/s

Details

Neuron Runtime

Trn1 , Inf1

  • Trn1: aws-neuronx-runtime-lib (.deb, .rpm)

  • Inf1: Runtime is linked into the ML frameworks packages

Neuron Runtime Driver

Trn1, Inf1

  • aws-neuronx-dkms (.deb, .rpm)

Neuron System Tools

Trn1, Inf1

  • aws-neuronx-tools (.deb, .rpm)

Containers

Trn1, Inf1

  • aws-neuronx-k8-plugin (.deb, .rpm)

  • aws-neuronx-k8-scheduler (.deb, .rpm)

  • aws-neuronx-oci-hooks (.deb, .rpm)

NeuronPerf (Inference only)

Trn1, Inf1

  • neuronperf (.whl)

Trn1 only packages#

Component

Instance/s

Package/s

Details

PyTorch Neuron

Trn1

  • torch-neuronx (.whl)

Neuron Compiler (Trn1 only)

Trn1

  • neuronx-cc (.whl)

Collective Communication library

Trn1

  • aws-neuronx-collective (.deb, .rpm)

Note

In next releases aws-neuronx-tools and aws-neuronx-runtime-lib will add support for Inf1.

Inf1 only packages#

Component

Instance/s

Package/s

Details

PyTorch Neuron

Inf1

  • torch-neuron (.whl)

TensorFlow Neuron

Inf1

  • tensorflow-neuron (.whl)

TensorFlow Model Server Neuron

Inf1

  • tensorflow-model-server-neuronx (.deb, .rpm)

Apache MXNet (Incubating)

Inf1

  • mx_neuron (.whl)

Neuron Compiler (Inf1 only)

Inf1

  • neuron-cc (.whl)