This document is relevant for: Inf1, Inf2, Trn1, Trn2, Trn3

Get Started with AWS Neuron#

Get up and running with AWS Neuron SDK in minutes. These quickstarts guide you through your first training or inference workload on Inferentia and Trainium instances.

Note

First time using AWS Neuron? These quickstarts assume you have:

An active AWS account with EC2 access
Basic familiarity with your chosen ML framework (PyTorch, JAX, or TensorFlow)
SSH access to launch and connect to EC2 instances

For detailed installation instructions, see the Setup Guide.

Choose Your Path#

Select the quickstart that matches your use case:

🚀 Training Quickstart

Train your first model on Trainium

Launch a Trn1 instance
Run a PyTorch training script
Monitor training progress

Time: ~15 minutes

Trn1 Trn2 Trn3

🎯 Inference Quickstart

Run your first inference on Inferentia

Launch an Inf2 instance
Load a pre-compiled model
Run predictions

Time: ~10 minutes

Inf2 Trn1

Specialized Quickstarts#

💬 LLM Serving with vLLM

Deploy large language models for production inference

Online serving (OpenAI-compatible API)
Offline batch inference

Time: ~20 minutes

Inf2 Trn1

🤖 Amazon Q Developer

Use AI-powered code assistance for Neuron development

Get code suggestions
Debug Neuron applications
Optimize performance

Time: ~5 minutes

Framework-Specific Guides#

Need framework-specific setup instructions?

PyTorch

PyTorch 2.9+ setup

JAX

JAX 0.7+ setup

TensorFlow

TensorFlow 2.x setup

Additional Resources#

Model samples and tutorials - Pre-tested model samples and tutorials
Amazon EC2 - Detailed EC2 deployment workflows
Neuron Containers - Use Deep Learning Containers
Neuron Quick Links - Quick links to all Neuron documentation
Neuron GitHub Samples - GitHub sample repositories

Legacy Quick-Start Pages (Inf1)#

Warning

The following pages are for legacy Inf1 instances only. For new projects, use the quickstarts above for Inf2, Trn1, Trn2, or Trn3.

Get Started with PyTorch Neuron - PyTorch on Inf1
Get Started with TensorFlow Neuron - TensorFlow on Inf1
Get Started with Apache MXNet Neuron - MXNet on Inf1