Tutorials for NeuronX Distributed (neuronx-distributed )
This document is relevant for: Inf2
, Trn1
, Trn1n
Tutorials for NeuronX Distributed (neuronx-distributed
)#
Training GPT-NeoX 6.9B with Tensor Parallelism and ZeRO-1 Optimizer (neuronx-distributed )
Training GPT-NeoX 20B with Tensor Parallelism and ZeRO-1 Optimizer (neuronx-distributed )
Training Llama2 7B with Tensor Parallelism and ZeRO-1 Optimizer (neuronx-distributed )
Training Llama-2-13B/70B with Tensor Parallelism and Pipeline Parallelism (neuronx-distributed )
T5 inference tutorial [html] [notebook]
Llama inference tutorial [html] [notebook]
This document is relevant for: Inf2
, Trn1
, Trn1n