Tutorials

This section includes tutorials that you can follow to get started with the NeuronX Distributed (NxD) Inference library.

• Tutorial: Deploying Llama3.1 405B (Trn2)

• Tutorial: Deploying Llama3.2 Multimodal Models

• Tutorial: Using Speculative Decoding to improve Llama-3.3-70B inference performance on Trn2 instances

• Tutorial: Scaling LLM Inference with Data Parallelism on Trn2

• Tutorial: Multi-LoRA serving for Llama-3.1-8B on Trn2 instances

• Tutorial: Using Speculative Decoding and Quantization to improve Llama-3.1-405B inference performance on Trn2 instances

• Tutorial: Evaluating Accuracy of Llama-3.1-70B on Neuron using open source datasets

• Tutorial: Disaggregated Inference [BETA]

• Tutorial: Static 1P1D Disaggregated Inference on Trn2 [BETA]

• Tutorial: Evaluating Performance of Llama-3.3-70B on Neuron using Performance CLI