Announcing end of support for the NxD Inference library and NxD Core Inference APIs on Trn2 and Trn3 starting in Neuron 2.34

As announced in Neuron 2.30 and reaffirmed in Neuron 2.31, the NxD Inference library and NxD Core Inference APIs have been in maintenance mode, receiving critical bug fixes and security patches only.

Starting with Neuron Release 2.34, Neuron will end support for the NxD Inference library and the NxD Core Inference APIs on Trn2 and Trn3 instances. No further releases, bug fixes, or security patches will be provided after this release to support Trn2 and Trn3 instances.

Current users on Trn2 and Trn3 should migrate to the vLLM Neuron plugin, which does not depend on the NxD Inference libraries and is the recommended inference solution for text generation and vision understanding LLMs on Trn2 and Trn3. For migration guidance, see Migrating from NxD Inference to vLLM Neuron.

On Inf2 and Trn1 instances, the NxD Inference library and NxD Core Inference APIs remain in maintenance mode at Neuron SDK 2.31, as described in Neuron 2.31. These instances continue to receive critical bug fixes and security patches only.

NxD Core model build and trace APIs

The NxD Core model build and trace APIs (for example neuronx_distributed.trace.parallel_model_trace and the ModelBuilder / ModelBuilderV2 APIs) are superseded by native PyTorch support on Trainium through TorchNeuron, which provides torch.compile-based compilation for Neuron.

Current users of these APIs should migrate to the native PyTorch torch.compile(backend="neuron") workflow. For details and migration guidance, see Native PyTorch for AWS Trainium.