Deploy Neuron Container on Elastic Kubernetes Service (EKS)¶
You can use the Neuron version of the AWS Deep Learning Containers to run inference on Amazon Elastic Kubernetes Service (EKS). In this developer flow, you set up an EKS cluster with Inf1 instances, create a Kubernetes manifest for your inference service and deploy it to your cluster. This developer flow assumes:
- Follow the instructions in this EKS documentation link to set up AWS Inferentia on your EKS cluster.
Before deploying your task definition to your EKS cluster, make sure to push the image to ECR. Refer to Pushing a Docker image for more information.
Please refer to Tutorial: Kubernetes environment setup for Neuron. In Deploy a TensorFlow Resnet50 model as a Kubernetes service, the container image referenced in the YML manifest is created using How to Build a Neuron Container.