This document is relevant for: Inf2
, Trn1
Inference Samples/Tutorials (Inf2/Trn1/Trn2)#
Encoders#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
bert-base-cased-finetuned-mrpc |
torch-neuronx |
|
bert-base-cased-finetuned-mrpc |
neuronx-distributed |
|
bert-base-uncased |
torch-neuronx |
|
distilbert-base-uncased |
torch-neuronx |
|
roberta-base |
tensorflow-neuronx |
|
roberta-large |
torch-neuronx |
Decoders#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
gpt2 |
torch-neuronx |
|
meta-llama/Llama-3.3-70B |
neuronx-distributed-inference |
|
meta-llama/Llama-3.2-90B |
neuronx-distributed-inference |
|
meta-llama/Llama-3.1-8b |
transformers-neuronx |
|
meta-llama/Llama-3.1-70b |
transformers-neuronx |
|
meta-llama/Llama-3.1-70b-Instruct |
transformers-neuronx |
|
meta-llama/Llama-3.1-405b |
neuronx-distributed-inference |
|
meta-llama/Llama-3.1-405b |
transformers-neuronx |
|
meta-llama/Llama-3-8b |
transformers-neuronx |
|
meta-llama/Llama-3-70b |
transformers-neuronx |
|
meta-llama/Llama-2-13b |
transformers-neuronx |
|
meta-llama/Llama-2-70b |
transformers-neuronx |
|
meta-llama/Llama-2-7b |
neuronx-distributed |
|
meta-llama/codellama-13b |
neuronx-distributed |
|
mistralai/Mistral-7B-Instruct-v0.1 |
transformers-neuronx |
|
mistralai/Mistral-7B-Instruct-v0.2 |
transformers-neuronx |
|
Mixtral-8x7B-v0.1 |
transformers-neuronx |
|
Mixtral-8x7B |
neuronx-distributed |
|
DBRX |
neuronx-distributed |
|
codellama/CodeLlama-13b-hf |
transformers-neuronx |
Encoder-Decoders#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
t5-large |
|
|
t5-3b |
neuronx-distributed |
|
google/flan-t5-xl |
neuronx-distributed |
|
Vision Transformers#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
google/vit-base-patch16-224 |
torch-neuronx |
|
clip-vit-base-patch32 |
torch-neuronx |
|
clip-vit-large-patch14 |
torch-neuronx |
Convolutional Neural Networks(CNN)#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
resnet50 |
torch-neuronx |
|
resnet50 |
tensorflow-neuronx |
|
unet |
torch-neuronx |
|
vgg |
torch-neuronx |
Stable Diffusion#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
stable-diffusion-v1-5 |
torch-neuronx |
|
stable-diffusion-2-1-base |
torch-neuronx |
|
stable-diffusion-2-1 |
torch-neuronx |
|
stable-diffusion-xl-base-1.0 |
torch-neuronx |
|
stable-diffusion-2-inpainting |
torch-neuronx |
Diffusion Transformers#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
pixart-alpha |
torch-neuronx |
|
pixart-sigma |
torch-neuronx |
Audio#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
wav2vec2-conformer |
torch-neuronx |
Multi Modal#
Model |
Frameworks/Libraries |
Samples and Tutorials |
---|---|---|
multimodal-perceiver |
torch-neuronx |
|
language-perceiver |
torch-neuronx |
|
vision-perceiver-conv |
torch-neuronx |
This document is relevant for: Inf2
, Trn1