Welcome to AWS Neuron

AWS Neuron is the SDK used to run deep learning workloads on AWS Inferentia and AWS Trainium based instances. It supports customers in their end-to-end ML development lifecycle to build new models, train and optimize these models, and then deploy them for production. To learn about the model architectures currently supported on Inf1, Inf2, Trn1 and Trn1n instances, please see Model Samples and Tutorials. To learn about upcoming capabilities, please view the Roadmap.

AWS Neuron includes a deep learning compiler, runtime, and tools that are natively integrated into TensorFlow, PyTorch and Apache MXNet. The EC2 Trn1/Trn1n instances are optimized for the highest performance and best price-performance training in AWS. The EC2 Inf2 instances are designed for high-performance deep learning inference applications. With Neuron, customers can quickly start using Inf/Trn instances through services like Amazon Sagemaker, Amazon Elastic Container Service (ECS), Amazon Elastic Kubernetes Service (EKS), AWS Batch, and AWS Parallel Cluster.

Check Announcements and check Neuron 2.20.1 (10/25/2024) for latest release.

Get Started with Neuron

Neuron Quick Links

Overview

• Quick Links
• Get Started with Neuron
• Samples and Tutorials
• Performance
• What’s New
• Announcements

ML Frameworks

• PyTorch Neuron
• JAX Neuron
• TensorFlow Neuron
• Apache MXNet

NeuronX Distributed (NxD)

• NxD Training (Beta)
• NxD Core

Additional Libraries

• Transformers Neuron
• AWS Neuron Reference for NeMo Megatron

Developer Flows

• Neuron DLAMI
• Neuron Containers
• Workload Orchestration
• Setup Guide

Runtime & Tools

• Neuron Runtime
• Neuron Tools
• Neuron Calculator

Compiler

• Neuron Compiler
• Neuron Kernel Interface (Beta)
• Neuron C++ Custom Operators

Learning Neuron

• Architecture
• Features
• Application Notes
• FAQ
• Troubleshooting

About Neuron

• Release Details
• Roadmap
• Support