This document is relevant for: Inf1

Inference with torch-neuron (Inf1)

Warning

This document is archived. torch-neuron (Inf1) is no longer actively developed. For new workloads, use TorchNeuron Native or torch-neuronx. See PyTorch Support on Neuron for current PyTorch support.

Setup (torch-neuron)

Tutorials (torch-neuron)

Computer Vision Tutorials

• ResNet-50 tutorial [html] [notebook]

• PyTorch YOLOv4 tutorial [html] [notebook]

Natural Language Processing (NLP) Tutorials

• HuggingFace pretrained BERT tutorial [html] [notebook]

• HuggingFace pretrained BERT tutorial with shared weights [html] [notebook]

• Bring your own HuggingFace pretrained BERT container to Sagemaker Tutorial [html] [notebook]

• LibTorch C++ tutorial [html]

• TorchServe tutorial [html]

• HuggingFace MarianMT tutorial [html] [notebook]

Utilizing Neuron Capabilities Tutorials

• BERT TorchServe tutorial [html]

• NeuronCore Pipeline tutorial [html] [notebook]

Note

To use Jupyter Notebook see:

• Jupyter Notebook QuickStart

• Running Jupyter Notebook as script

Additional Examples (torch-neuron)

• AWS Neuron Samples GitHub Repository

API Reference Guide (torch-neuron)

• PyTorch Neuron trace Python API

• torch.neuron.DataParallel API

• PyTorch Neuron (torch-neuron) Core Placement API

Developer Guide (torch-neuron)

• Running Inference on Variable Input Shapes with Bucketing

• Data Parallel Inference on PyTorch Neuron

• Developer Guide - PyTorch Neuron (torch-neuron) LSTM Support

• PyTorch Neuron (torch-neuron) Core Placement

Misc (torch-neuron)

• PyTorch Neuron (torch-neuron) Supported operators

• Troubleshooting Guide for PyTorch Neuron (torch-neuron)

• PyTorch Neuron (torch-neuron) release notes

This document is relevant for: Inf1