This document is relevant for: Inf2, Trn1, Trn2, Trn3
NeuronX Compiler for Trn* & Inf2
The NeuronX Compiler (neuronx-cc) is the XLA-based Neuron Graph Compiler for
NeuronCores v2 to v4 (Trn1, Trn1n, Inf2, and Trn2). Use the guides below to learn what it
does, look up its command line, and work through common compilation tasks.
Get started
About neuronx-cc
What the Neuron Graph Compiler is, how it fits into the Neuron workflow, and
the files and features that make it up.
CLI reference guide
The complete neuronx-cc command line: the compile and
list-operators commands, every option, and their valid values.
How-to guides
How to generate a NEFF file
Compile an XLA HLO graph into a NEFF file with neuronx-cc, the artifact the
Neuron Runtime loads to run your model.
How to use convolution kernels in UNet models
Modify UNet training models to use custom convolution kernels with NKI, to
avoid out-of-memory errors on convolution-heavy models.
Guides and reference
FAQ
Common questions about compiling to NEFF, neuron-cc vs. neuronx-cc,
supported operators, precision, and recompilation.
This document is relevant for: Inf2, Trn1, Trn2, Trn3
