This document is relevant for: Inf1

nosearch:

Inference Samples/Tutorials (Inf1) (archived)

Warning

This page is archived and is no longer maintained. Inf1 instances and the torch-neuron, TensorFlow Neuron, and MXNet Neuron frameworks are no longer supported. For current inference, see Inference on Neuron.

Important

The samples linked on this page have been archived and are provided for historical reference only. They are not tested with recent versions of the Neuron SDK.

Encoders

Model	Frameworks/Libraries	Samples and Tutorials
bert-base-cased-finetuned-mrpc	torch-neuron	HuggingFace pretrained BERT tutorial [html] [notebook] BertBaseCased Inference on Inf1 instances Bert TorchServe tutorial [html] Bring your own HuggingFace pretrained BERT container to Sagemaker Tutorial [html] [notebook]
bert-base-uncased	torch-neuron	NeuronCore Pipeline tutorial [html] [notebook]
bert-large-uncased	torch-neuron	BertLargeUncased Inference on Inf1 instances
roberta-base	torch-neuron	Roberta-Base inference on Inf1 instances
distilbert-base-uncased-finetuned-sst-2-english	tensorflow-neuron	Tensorflow 2.x - HuggingFace Pipelines distilBERT with Tensorflow2 Neuron [html] [notebook]
gluon bert	mxnet-neuron	MXNet 1.8: Using data parallel mode tutorial [html] [notebook]

Vision Transformers

Model	Frameworks/Libraries	Samples and Tutorials
ssd	torch-neuron	Inference of SSD model on inf1 instances
TrOCR	torch-neuron	TrOCR inference on Inf1 instances
vgg	torch-neuron	VGG inference on Inf1 instances
google/vit-base-patch16-224	torch-neuron	ViT model inference on Inf1

Convolutional Neural Networks(CNN)

Model	Frameworks/Libraries	Samples and Tutorials
EfficientNet	torch-neuron	EfficientNet model inference on Inf1 instances
GFL (MMDetection)	torch-neuron	GFL (MMDetection) inference on Inf1 instances
HRNet	torch-neuron	HRNET - Pose Estimation
MarianMT	torch-neuron	HuggingFace MarianMT tutorial [html] [notebook] Inference of Pre-trained MarianMT model on Inf1
Detectron2 R-CNN	torch-neuron	R-CNN inference on Inf1
resnet	torch-neuron	Inference of Pre-trained Resnet model (18,34,50,101,152) on Inf1 ResNet-50 tutorial [html] [notebook]
resnet	tensorflow-neuron	Tensorflow 2.x - Using NEURON_RT_VISIBLE_CORES with TensorFlow Serving [html]
resnet	mxnet-neuron	ResNet-50 tutorial [html] [notebook] Getting started with Gluon tutorial [html] [notebook] NeuronCore Groups tutorial [html] [notebook]
Resnext	torch-neuron	Inference of Resnext model on Inf1
Yolov4	torch-neuron	PyTorch YOLOv4 tutorial [html] [notebook]
Yolov5	torch-neuron	Inference of Yolov5 on Inf1
Yolov6	torch-neuron	Inference of Yolov6 on Inf1 instances
Yolov7	torch-neuron	Inference of Yolov7 model on Inf1
Yolof	torch-neuron	Inference of Yolof model on Inf1
fairseq	torch-neuron	Inference of fairseq model on Inf1
unet	tensorflow-neuron	Unet - Tensorflow 2.x tutorial

Vision

Model	Frameworks/Libraries	Samples and Tutorials
craft-pytorch	torch-neuron	CRAFT model inference on Inf1

This document is relevant for: Inf1