This document is relevant for: Inf1

Inference Samples/Tutorials (Inf1)

Encoders

Model	Frameworks/Libraries	Samples and Tutorials
bert-base-cased-finetuned-mrpc	torch-neuron	HuggingFace pretrained BERT tutorial [html] [notebook] BertBaseCased Inference on Inf1 instances Bert TorchServe tutorial [html] Bring your own HuggingFace pretrained BERT container to Sagemaker Tutorial [html] [notebook]
bert-base-uncased	torch-neuron	NeuronCore Pipeline tutorial [html] [notebook]
bert-large-uncased	torch-neuron	BertLargeUncased Inference on Inf1 instances
bert-large	tensorflow-neuron	Tensorflow 1.x - Running TensorFlow BERT-Large with AWS Neuron [html]
roberta-base	torch-neuron	Roberta-Base inference on Inf1 instances
distilbert-base-uncased-finetuned-sst-2-english	tensorflow-neuron	Tensorflow 2.x - HuggingFace Pipelines distilBERT with Tensorflow2 Neuron [html] [notebook]
gluon bert	mxnet-neuron	MXNet 1.8: Using data parallel mode tutorial [html] [notebook]

Vision Transformers

Model	Frameworks/Libraries	Samples and Tutorials
ssd	torch-neuron	Inference of SSD model on inf1 instances
ssd	tensorflow-neuron	Tensorflow 1.x - SSD300 tutorial [html]
TrOCR	torch-neuron	TrOCR inference on Inf1 instances
vgg	torch-neuron	VGG inference on Inf1 instances
google/vit-base-patch16-224	torch-neuron	ViT model inference on Inf1

Convolutional Neural Networks(CNN)

Model	Frameworks/Libraries	Samples and Tutorials
EfficientNet	torch-neuron	EfficientNet model inference on Inf1 instances
GFL (MMDetection)	torch-neuron	GFL (MMDetection) inference on Inf1 instances
HRNet	torch-neuron	HRNET - Pose Estimation
MarianMT	torch-neuron	HuggingFace MarianMT tutorial [html] [notebook] Inference of Pre-trained MarianMT model on Inf1
Detectron2 R-CNN	torch-neuron	R-CNN inference on Inf1
resnet	torch-neuron	Inference of Pre-trained Resnet model (18,34,50,101,152) on Inf1 ResNet-50 tutorial [html] [notebook]
resnet	tensorflow-neuron	Tensorflow 1.x - ResNet-50 tutorial [html] [notebook] Tensorflow 1.x - Keras ResNet-50 optimization tutorial [html] [notebook] Tensorflow 1.x & 2.x - Using NEURON_RT_VISIBLE_CORES with TensorFlow Serving [html]
resnet	mxnet-neuron	ResNet-50 tutorial [html] [notebook] Getting started with Gluon tutorial [html] [notebook] NeuronCore Groups tutorial [html] [notebook]
Resnext	torch-neuron	Inference of Resnext model on Inf1
Yolov3	tensorflow-neuron	Tensorflow 1.x - YOLOv3 tutorial [html] [notebook]
Yolov4	torch-neuron	PyTorch YOLOv4 tutorial [html] [notebook]
Yolov4	tensorflow-neuron	Tensorflow 1.x - YOLOv4 tutorial [html] [notebook]
Yolov5	torch-neuron	Inference of Yolov5 on Inf1
Yolov6	torch-neuron	Inference of Yolov6 on Inf1 instances
Yolov7	torch-neuron	Inference of Yolov7 model on Inf1
Yolof	torch-neuron	Inference of Yolof model on Inf1
fairseq	torch-neuron	Inference of fairseq model on Inf1
openpose	tensorflow-neuron	Tensorflow 1.x - OpenPose tutorial [html] [notebook]
unet	tensorflow-neuron	Unet - Tensorflow 2.x tutorial

Vision

Model	Frameworks/Libraries	Samples and Tutorials
craft-pytorch	torch-neuron	CRAFT model inference on Inf1

This document is relevant for: Inf1