This document is relevant for: Inf1
, Inf2
, Trn1
, Trn2
Announcing migration of NxD Core examples from NxD Core repository to NxD Inference repository in next release#
Neuron Release 2.21 will be the last release to include NxD Core repository inference examples under the NxD Core repository: aws-neuron/neuronx-distributed. Starting with Neuron Release 2.21, the models and modules in NxD Core inference examples are now available through NxD Inference package. We recommend customers to update their applications to use examples from the NxD Inference repository. See NxD Inference Overview
In Neuron Release 2.22, the NxD Core inference samples will only reside under the NxD Inference repository. Current users are advised to start using samples/tutorials under the NxD Inference repository: aws-neuron/neuronx-distributed-inference.
I currently utilize an inference sample from the NxD Core repository in my model code. What do I do?#
If your applications depend on the inference examples from NxD Core, we recommend that you update your code to use the new NxD Inference package. With NxD Inference, you can import and use these models and modules in your applications. Any models compiled with inference code from the NxD Core repository will need to be re-compiled. Please refer to the Migrating from NxD Core inference examples to NxD Inference for guidance.
This document is relevant for: Inf1
, Inf2
, Trn1
, Trn2