Neuron 2.6 is released! check What's New and Announcements
logo

AWS Neuron Documentation

Overview

  • Quick Links
  • Get Started with PyTorch
  • Get Started with TensorFlow
  • Get Started with MXNet
  • Performance
  • What’s New
  • Announcements

ML Frameworks

  • PyTorch Neuron
    • Fresh install
    • Update to latest release
    • Install previous releases
      • Neuron 2.4.0
      • Neuron 2.3.0
    • Hugging Face BERT Pretraining Tutorial
    • Multi-Layer Perceptron Training Tutorial
    • PyTorch Neuron for Trainium Hugging Face BERT MRPC task finetuning using Hugging Face Trainer API
    • Megatron-LM GPT Pretraining Tutorial
    • PyTorch Neuron neuron_parallel_compile CLI
    • PyTorch Neuron Environment Variable
    • PyTorch Neuron Profiling API
    • Developer Guide for Training with PyTorch Neuron ( torch-neuronx )
    • How to debug models in PyTorch Neuron ( torch-neuronx )
    • Developer Guide for Profiling with PyTorch Neuron ( torch-neuronx )
    • PyTorch Neuron ( torch-neuronx ) - Training Supported Operators
    • PyTorch Neuron ( torch-neuronx ) for Training Troubleshooting Guide
    • PyTorch Neuron ( torch-neuronx ) release notes
    • Fresh install
    • Update to latest release
    • Install previous releases
      • Neuron 2.5.0
      • Neuron 2.4.0
      • Neuron 2.3.0
      • Neuron 1.19.0
      • Neuron 1.18.0
      • Neuron 1.17.2
      • Neuron 1.16.3
      • Neuron 1.16.2
      • Neuron 1.16.1
      • Neuron 1.15.2
      • Neuron 1.15.1
      • Neuron 1.15.0
      • Neuron 1.14.2
    • Install with support for cxx11 ABI
    • ResNet50 model for Inferentia
    • Evaluate YOLO v4 on Inferentia
    • Compiling and Deploying HuggingFace Pretrained BERT
    • Deploy a pretrained PyTorch BERT model from HuggingFace on Amazon SageMaker with Neuron container
    • Transformers MarianMT Tutorial
    • Using NeuronCore Pipeline with PyTorch
    • PyTorch Neuron trace Python API
    • torch.neuron.DataParallel API
    • Experimental: NeuronCore Placement APIs
    • Running Inference on Variable Input Shapes with Bucketing
    • Data Parallel Inference on PyTorch Neuron
    • Developer Guide - PyTorch Neuron ( torch-neuron ) LSTM Support
    • Developer Guide - PyTorch Neuron ( torch-neuron ) Core Placement
    • PyTorch Neuron ( torch-neuron ) Supported operators
    • Troubleshooting Guide for PyTorch Neuron ( torch-neuron )
    • PyTorch Neuron ( torch-neuron ) release notes
  • TensorFlow Neuron
    • Fresh Install
    • Update to Latest Release
    • Install Previous Releases
      • Neuron 1.19.0
      • Neuron 1.18.0
      • Neuron 1.17.2
      • Neuron 1.17.1
      • Neuron 1.17.0
      • Neuron 1.16.3
      • Neuron 1.15.2
      • Neuron 1.15.1
      • Neuron 1.15.0
      • Neuron 1.14.2
    • Running OpenPose on Inferentia
    • Running ResNet50 on Inferentia
    • Working with YOLO v4 using AWS Neuron SDK
    • Evaluate YOLO v3 on Inferentia
    • Running SSD300 with AWS Neuron
    • Tensorflow ResNet 50 Optimization Tutorial
    • Running TensorFlow BERT-Large with AWS Neuron
    • Compiling and Deploying Pretrained HuggingFace Pipelines distilBERT with Tensorflow2 Neuron
    • Using NEURON_RT_VISIBLE_CORES with TensorFlow Serving
    • TensorFlow 2.x ( tensorflow-neuron ) Tracing API
    • TensorFlow 1.x ( tensorflow-neuron ) Compilation API
    • TensorFlow Neuron ( tensorflow-nueron ) Auto Multicore Replication (Experimental)
    • TensorFlow Neuron ( tensorflow-neuron (TF1.x) ) Release Notes
    • TensorFlow Neuron ( tensorflow-neuron (TF2.x) ) Release Notes
    • TensorFlow Neuron ( tensorflow-neuron (TF2.x) ) Accelerated (torch-neuron) Python APIs and Graph Ops
    • TensorFlow Neuron ( tensorflow-neuron (TF1.x) ) Supported operators
  • Apache MXNet (Incubating)
    • Fresh install
    • Update to latest release
    • Install previous releases
      • Neuron 1.19.0
      • Neuron 1.18.0
      • Neuron 1.17.2
      • Neuron 1.16.3
      • Neuron 1.15.2
      • Neuron 1.15.1
      • Neuron 1.15.0
      • Neuron 1.14.2
    • Running Neuron Apache MXNet (Incubating) ResNet50 on Inferentia
    • Tutorial: Neuron Apache MXNet (Incubating) Model Serving
    • MXNet 1.8: Getting Started with Gluon Tutorial
    • Using Data Parallel Mode with Gluon MXNet
    • Neuron Apache MXNet (Incubating) - Configurations for NeuronCore Groups Using Resnet50
    • Neuron Apache MXNet (Incubating) Compilation Python API
    • Flexible Execution Group (FlexEG) in Neuron-MXNet
    • Troubleshooting Guide for Neuron Apache MXNet (Incubating)
    • What's New
    • Neuron Apache MXNet (Incubating) Supported operators

User Guide

  • Neuron Runtime
    • Runtime Configuration
    • Troubleshooting on Trn1
    • Troubleshooting on Inf1
    • FAQ
    • Neuron Runtime Release Notes
    • Neuron Driver Release Notes
    • Neuron Collectives Release Notes
  • Neuron Compiler
    • Neuron Compiler CLI Reference Guide
    • Mixed Precision and Performance-accuracy Tuning ( neuronx-cc )
    • FAQ
    • What's New
    • Neuron compiler CLI Reference Guide ( neuron-cc )
    • Mixed precision and performance-accuracy tuning ( neuron-cc )
    • FAQ
    • What's New
    • Neuron Supported operators
      • TensorFlow Neuron ( tensorflow-neuron (TF1.x) ) Supported operators
      • PyTorch Neuron ( torch-neuron ) Supported operators
      • Neuron Apache MXNet (Incubating) Supported operators
  • Neuron Tools
    • System Tools
      • Neuron-Monitor User Guide
      • Neuron-Top User Guide
      • Neuron-LS User Guide
      • What's New
    • TensorBoard
      • Track Training Progress in TensorBoard using PyTorch Neuron
      • TensorBoard Plugin for Neuron (Trn1)
      • What's New
      • TensorBoard Plugin for Neuron (Inf1)
    • Helper Tools
      • Check Model
      • GatherInfo
    • NeuronPerf (Beta)
      • Overview
      • Terminology
      • Examples
      • Benchmark Guide
      • Evaluate Guide
      • Compile Guide
      • Model Index Guide
      • API
      • Framework Notes
      • FAQ
      • Troubleshooting
      • What’s New
  • Setup Guide
    • Fresh install
    • Update to latest release
    • Install previous releases
      • Neuron 2.5.0
      • Neuron 2.4.0
      • Neuron 2.3.0
      • Neuron 1.19.0
      • Neuron 1.18.0
      • Neuron 1.17.2
      • Neuron 1.16.3
      • Neuron 1.16.2
      • Neuron 1.16.1
      • Neuron 1.15.2
      • Neuron 1.15.1
      • Neuron 1.15.0
      • Neuron 1.14.2
    • Install with support for cxx11 ABI
    • Fresh Install
    • Update to Latest Release
    • Install Previous Releases
      • Neuron 1.19.0
      • Neuron 1.18.0
      • Neuron 1.17.2
      • Neuron 1.17.1
      • Neuron 1.17.0
      • Neuron 1.16.3
      • Neuron 1.15.2
      • Neuron 1.15.1
      • Neuron 1.15.0
      • Neuron 1.14.2
    • Fresh install
    • Update to latest release
    • Install previous releases
      • Neuron 1.19.0
      • Neuron 1.18.0
      • Neuron 1.17.2
      • Neuron 1.16.3
      • Neuron 1.15.2
      • Neuron 1.15.1
      • Neuron 1.15.0
      • Neuron 1.14.2
    • Fresh install
    • Update to latest release
    • Install previous releases
      • Neuron 2.4.0
      • Neuron 2.3.0
  • Containers Deployment
    • Run training in Pytorch Neuron container
    • Deploy a simple mlp training script as a Kubernetes job
    • Run inference in pytorch neuron container
    • Deploy a TensorFlow Resnet50 model as a Kubernetes service
    • Deploy Neuron Container on EC2
    • Deploy Neuron Container on Elastic Container Service (ECS)
    • Deploy Neuron Container on Elastic Kubernetes Service (EKS)
    • Bring Your Own Neuron Container to Sagemaker Hosting
    • FAQ
    • Troubleshooting Neuron Containers
    • Neuron Containers Release Notes
    • Neuron K8 Release Notes
  • Developer Flows
    • Deploy Containers with Neuron
      • Run training in Pytorch Neuron container
      • Deploy a simple mlp training script as a Kubernetes job
      • Run inference in pytorch neuron container
      • Deploy a TensorFlow Resnet50 model as a Kubernetes service
      • Deploy Neuron Container on EC2
      • Deploy Neuron Container on Elastic Container Service (ECS)
      • Deploy Neuron Container on Elastic Kubernetes Service (EKS)
      • Bring Your Own Neuron Container to Sagemaker Hosting
      • FAQ
      • Troubleshooting Neuron Containers
      • Neuron Containers Release Notes
      • Neuron K8 Release Notes
    • Compile with Framework API and Deploy on EC2 Inf1
    • Train your model on EC2
    • Deploy Neuron Container on Elastic Kubernetes Service (EKS)
    • Deploy Neuron Container on Elastic Container Service (ECS)
    • Compile with Sagemaker Neo and Deploy on Sagemaker Hosting
    • Bring Your Own Neuron Container to Sagemaker Hosting
    • Train your model on SageMaker
    • Train your model on ParallelCluster

Learning Neuron

  • Architecture
    • AWS Inf1 Architecture
    • AWS Trn1 Architecture
    • AWS NeuronCore Architecture
    • Neuron Model Architecture Fit Guidelines
    • Neuron Glossary
  • Features
    • Data Types
    • Rounding Modes
    • Neuron Batching
    • NeuronCore Pipeline
    • Neuron Persistent Cache
    • Collective Communication
    • Neuron Control Flow
    • Neuron Custom C++ Operators
    • Neuron Dynamic Shapes
  • Application Notes
    • Introducing first release of Neuron 2.x enabling EC2 Trn1 general availability (GA)
    • Introducing Neuron Runtime 2.x (libnrt.so)
    • Performance Tuning
    • Parallel Execution using NEURON_RT_NUM_CORES
    • Running R-CNNs on Inf1
  • FAQ
  • Troubleshooting

About Neuron

  • Release Details
  • Roadmap
    • Neuron Public Roadmap
  • Support
    • SDK Maintenance Policy
    • Security Disclosures
    • Contact Us
Theme by the Executable Book Project
  • repository
  • open issue
  • suggest edit
  • .rst
Contents
  • Neuron 2.x FAQ
  • Training Only FAQ
  • Inference Only FAQ
  • Runtime FAQ
  • Compiler FAQ
  • Neuron Containers
  • Support

Neuron FAQ

Contents

  • Neuron 2.x FAQ
  • Training Only FAQ
  • Inference Only FAQ
  • Runtime FAQ
  • Compiler FAQ
  • Neuron Containers
  • Support

This document is relevant for: Inf1, Trn1

Neuron FAQ#

Table of contents

  • Neuron 2.x FAQ

  • Training Only FAQ

  • Inference Only FAQ

  • Runtime FAQ

  • Compiler FAQ

  • Neuron Containers

  • Support

Neuron 2.x FAQ#

  • Neuron 2.x Introduction at Trn1 GA - FAQ

Training Only FAQ#

  • Training with Neuron - FAQ

Inference Only FAQ#

  • Inference with Neuron - FAQ

  • Troubleshooting for Inf1 - FAQ

  • TensorFlow 1.x FAQ

  • TensorFlow 2.x FAQ

  • NeuronPerf

Runtime FAQ#

  • Neuron Runtime FAQ

Compiler FAQ#

  • Neuron Compiler FAQ (neuronx-cc)

  • Neuron Compiler FAQ (neuron-cc)

Neuron Containers#

  • Neuron Containers FAQ

Support#

  • Roadmap FAQ

  • Contributing Guidelines FAQs

This document is relevant for: Inf1, Trn1

previous

Running R-CNNs on Inf1

next

Troubleshooting Guide

By AWS
© Copyright 2022, Amazon.com.