This document is relevant for: Trn2, Trn3

Model development

Onboard new model architectures, debug accuracy issues, and develop without hardware. For developers adding or validating models on vLLM Neuron.

Onboard a new model

Implement and register a new architecture with vLLM.

Onboard a vision-language model

Add a vision encoder tower on top of the text-decoder flow.

Optimizing a vision-language model

Roofline, sharding, and profiling to optimize a vision-language model.

CPU development workflow

Develop and test without Neuron hardware.

NKI CPU simulator

Validate NKI kernel correctness on CPU.

Debugging model code

Use pdb and print statements to inspect model execution.

Debugging accuracy issues

Methodology for isolating where accuracy drift is introduced.

Accuracy debugger tools

Run the automated debugger pipeline and interpret results.

This document is relevant for: Trn2, Trn3