AWS Neuron SDK Documentation#

AWS Neuron is a software development kit (SDK) that enables high-performance deep learning and generative AI workloads on AWS Inferentia and AWS Trainium instances. Neuron provides a complete machine learning development experience with compiler optimization, runtime efficiency, and comprehensive tooling.

Key Features:

  • Native Framework Integration - Seamlessly integrated with PyTorch and JAX, with distributed training libraries for large-scale workloads

  • Frontier Model Support - Optimized for large language models including Llama 3.3-70B and Llama 3.1-405B

  • Performance Optimization - Advanced compiler, profiling tools, and custom kernel support for maximum efficiency

  • Enterprise Ready - Full integration with AWS services including SageMaker, EKS, ECS, and third-party platforms

Supported Instance Types: Inf1, Inf2, Trn1, Trn2, and Trn2 UltraServer

About the AWS Neuron SDK

Learn about the AWS Neuron SDK, its components, and supported hardware

Install the AWS Neuron SDK

Step-by-step guides for installing the AWS Neuron SDK

Get started with the Neuron SDK

Start building with step-by-step tutorials

Release notes

Latest updates and changes to the AWS Neuron SDK

Contents#

Learn about AWS Neuron
Get started with AWS Neuron
Develop with AWS Neuron
AWS Neuron-supported ML frameworks
NeuronX Distributed (NxD) libraries
Additional ML libraries
Developer workloads
Runtime & Collectives
Neuron Kernel Interface (NKI)
Neuron Compiler

AWS and the AWS logo are trademarks of Amazon Web Services, Inc. or its affiliates. All rights reserved.