This document is relevant for: Inf2, Trn1, Trn1n

Tutorials for NeuronX Distributed (neuronx-distributed )

• Training with Tensor Parallelism (neuronx-distributed )

• Training GPT-NeoX 6.9B with Tensor Parallelism and ZeRO-1 Optimizer (neuronx-distributed )

• Training GPT-NeoX 20B with Tensor Parallelism and ZeRO-1 Optimizer (neuronx-distributed )

• Training Llama3.1-8B, Llama3-8B and Llama2-7B with Tensor Parallelism and ZeRO-1 Optimizer (neuronx-distributed )

• Training Llama-3.1-70B, Llama-3-70B or Llama-2-13B/70B with Tensor Parallelism and Pipeline Parallelism (neuronx-distributed )

• Training Llama-2-7B/13B/70B using Tensor Parallelism and Pipeline Parallelism with Neuron PyTorch-Lightning (neuronx-distributed )

• Fine-tuning Llama2 7B with tensor parallelism and ZeRO-1 optimizer using Neuron PyTorch-Lightning (neuronx-distributed )

• T5 inference tutorial [html] [notebook]

• Llama inference tutorial [html] [notebook]

• CodeLlama-13b Inference with 16k sequence length [notebook]

This document is relevant for: Inf2, Trn1, Trn1n