{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "wrapped-soccer",
   "metadata": {},
   "source": [
    "# Running Neuron Apache MXNet ResNet50 on Inferentia "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "appreciated-daily",
   "metadata": {},
   "source": [
    "## Introduction:\n",
    "In this tutorial we will compile and deploy ResNet50 model for Inferentia.\n",
    "In this tutorial we provide two main sections:\n",
    "\n",
    "1.Compile the ResNet50 model.\n",
    "\n",
    "2.Infer the compiled model.\n",
    "\n",
    "Before running the following verify this Jupyter notebook is running “conda_aws_neuron_mxnet_p36” kernel. You can select the Kernel from the “Kernel -> Change Kernel” option on the top of this Jupyter notebook page.\n",
    "Neuron supports Python module, Symbol APIs and the C predict API. The following quick start example uses the Symbol API.\n",
    "\n",
    "### Warning\n",
    "This tutorial was tested on MXNet-1.5\n",
    "\n",
    "MXNet-1.5 entered maintenance mode and require Neuron runtime 1.0, please see : [MXNet-1.5 enters maintainence mode](../../../../release-notes/maintenance.html)\n",
    "\n",
    "To setup development environment for MXNet-1.5 see installation instructions for Neuron 1.15.1 : [Neuron-1.15.1 MXNet install](../../../../frameworks/mxnet-neuron/setup/mxnet-install.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "advance-rebound",
   "metadata": {},
   "source": [
    "## Compile model on Neuron\n",
    "The following step will compile the resnet50 model. Compilation will take a few minutes on inf1.6xlarge. At the end of compilation, the files resnet-50_compiled-0000.params and resnet-50_compiled-symbol.json will be created in local directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "alpha-publication",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "import numpy as np\n",
    "\n",
    "path='http://data.mxnet.io/models/imagenet/'\n",
    "mx.test_utils.download(path+'resnet/50-layers/resnet-50-0000.params')\n",
    "mx.test_utils.download(path+'resnet/50-layers/resnet-50-symbol.json')\n",
    "sym, args, aux = mx.model.load_checkpoint('resnet-50', 0)\n",
    "\n",
    "# Compile for Inferentia using Neuron\n",
    "inputs = { \"data\" : mx.nd.ones([1,3,224,224], name='data', dtype='float32') }\n",
    "sym, args, aux = mx.contrib.neuron.compile(sym, args, aux, inputs)\n",
    "\n",
    "#save compiled model\n",
    "mx.model.save_checkpoint(\"resnet-50_compiled\", 0, sym, args, aux)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "technical-reason",
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "meaningful-substance",
   "metadata": {},
   "source": [
    "## Deploy on Inferentia\n",
    "Using same instance to deploy the model.        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cooked-jonathan",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "import numpy as np\n",
    "\n",
    "path='http://data.mxnet.io/models/imagenet/'\n",
    "mx.test_utils.download(path+'synset.txt')\n",
    "\n",
    "fname = mx.test_utils.download('https://raw.githubusercontent.com/awslabs/mxnet-model-server/master/docs/images/kitten_small.jpg?raw=true')\n",
    "img = mx.image.imread(fname)# convert into format (batch, RGB, width, height)\n",
    "img = mx.image.imresize(img, 224, 224) # resize\n",
    "img = img.transpose((2, 0, 1)) # Channel first\n",
    "img = img.expand_dims(axis=0) # batchify\n",
    "img = img.astype(dtype='float32')\n",
    "\n",
    "sym, args, aux = mx.model.load_checkpoint('resnet-50_compiled', 0)\n",
    "softmax = mx.nd.random_normal(shape=(1,))\n",
    "args['softmax_label'] = softmax\n",
    "args['data'] = img\n",
    "\n",
    "# Inferentia context\n",
    "ctx = mx.neuron()\n",
    "\n",
    "exe = sym.bind(ctx=ctx, args=args, aux_states=aux, grad_req='null')\n",
    "\n",
    "with open('synset.txt', 'r') as f:\n",
    "     labels = [l.rstrip() for l in f]\n",
    "\n",
    "exe.forward(data=img)\n",
    "prob = exe.outputs[0].asnumpy()# print the top-5\n",
    "prob = np.squeeze(prob)\n",
    "a = np.argsort(prob)[::-1]\n",
    "for i in a[0:5]:\n",
    "     print('probability=%f, class=%s' %(prob[i], labels[i]))\n",
    "        \n",
    "# Sample output will look like below:\n",
    "#probability=0.634792, class=n02123045 tabby, tabby cat\n",
    "#probability=0.193601, class=n02123159 tiger cat\n",
    "#probability=0.103627, class=n02124075 Egyptian cat\n",
    "#probability=0.031604, class=n02127052 lynx, catamount\n",
    "#probability=0.015892, class=n02129604 tiger, Panthera tigris"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Environment (conda_aws_neuron_mxnet_p36)",
   "language": "python",
   "name": "conda_aws_neuron_mxnet_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}