This document is relevant for: Trn1, Trn2

Training (torch-neuronx)

Setup (torch-neuronx)

Tutorials

• Hugging Face BERT Pretraining Tutorial (Data-Parallel)

• Multi-Layer Perceptron Training Tutorial

• PyTorch Neuron for Trainium Hugging Face BERT MRPC task finetuning using Hugging Face Trainer API

• Fine-tune T5 model on Trn1

• ZeRO-1 Tutorial

• Analyze for Training Tutorial

• Neuron Custom C++ Operators in MLP Training

• Neuron Custom C++ Operators Performance Optimization

Note

To use Jupyter Notebook see:

• Jupyter Notebook QuickStart

• Running Jupyter Notebook as script

Additional Examples

• AWS Neuron Reference for Nemo Megatron GitHub Repository

• AWS Neuron Samples for EKS

• AWS Neuron Samples for AWS ParallelCluster

• AWS Neuron Samples GitHub Repository

API Reference Guide

• PyTorch NeuronX neuron_parallel_compile CLI

• Neuron Persistent Cache

• PyTorch NeuronX Environment Variables

• PyTorch NeuronX Profiling API

Developer Guide

• Developer Guide for Training with PyTorch NeuronX

• How to debug models in PyTorch NeuronX

• Developer Guide for Profiling with PyTorch NeuronX

Misc

• PyTorch Neuron (torch-neuronx) - Supported Operators

• How to prepare trn1.32xlarge for multi-node execution

• PyTorch Neuron (torch-neuronx) for Training Troubleshooting Guide

• PyTorch Neuron (torch-neuronx) release notes

This document is relevant for: Trn1, Trn2