{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "a3bskVXPvchm"
   },
   "source": [
    "# Running ResNet50 on Inferentia\n",
    "## Note: this tutorial runs on tensorflow-neuron 1.x only"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Introduction:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Rb5rSpcZvYbX"
   },
   "source": [
    "In this tutorial we will compile and deploy ResNet50 model for Inferentia.\n",
    "In this tutorial we provide two main sections:\n",
    "1. Compile the ResNet50 model.\n",
    "2. Infer the same compiled model.\n",
    "\n",
    "Verify that this Jupyter notebook is running the Python kernel environment that was set up according to the [Tensorflow Installation Guide](../../../../frameworks/tensorflow/tensorflow-neuron/setup/tensorflow-install.html#install-neuron-tensorflow). You can select the Kernel from the “Kernel -> Change Kernel” option on the top of this Jupyter notebook page.\n",
    "\n",
    "Instructions of how to setup Neuron Tensorflow environment and run the tutorial as a Jupyter notebook are available in the [Tensorflow Quick Setup](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/tensorflow/tensorflow-neuron/tutorials/tensorflow-tutorial-setup.html#tensorflow-tutorial-setup)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "!pip install tensorflow_neuron==1.15.5.2.8.9.0 --extra-index-url=https://pip.repos.neuron.amazonaws.com/\n",
    "!pip install neuron_cc==1.13.5.0 --extra-index-url=https://pip.repos.neuron.amazonaws.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E8FhiMivhcYB"
   },
   "source": [
    "## Compile for Neuron\n",
    "\n",
    "A trained model must be compiled to Inferentia target before it can be deployed on Inferentia instances. In this step we compile the Keras ResNet50 model and export it as a SavedModel which is an interchange format for TensorFlow models.\n",
    "At the end of compilation, the compiled SavedModel is saved in resnet50_neuron local directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import shutil\n",
    "import tensorflow as tf\n",
    "import tensorflow.neuron as tfn\n",
    "import tensorflow.compat.v1.keras as keras\n",
    "from tensorflow.keras.applications.resnet50 import ResNet50\n",
    "from tensorflow.keras.applications.resnet50 import preprocess_input\n",
    "\n",
    "# Create a workspace\n",
    "WORKSPACE = './ws_resnet50'\n",
    "os.makedirs(WORKSPACE, exist_ok=True)\n",
    "\n",
    "# Prepare export directory (old one removed)\n",
    "model_dir = os.path.join(WORKSPACE, 'resnet50')\n",
    "compiled_model_dir = os.path.join(WORKSPACE, 'resnet50_neuron')\n",
    "shutil.rmtree(model_dir, ignore_errors=True)\n",
    "shutil.rmtree(compiled_model_dir, ignore_errors=True)\n",
    "\n",
    "# Instantiate Keras ResNet50 model\n",
    "keras.backend.set_learning_phase(0)\n",
    "keras.backend.set_image_data_format('channels_last')\n",
    "\n",
    "model = ResNet50(weights='imagenet')\n",
    "\n",
    "# Export SavedModel\n",
    "tf.saved_model.simple_save(\n",
    "    session            = keras.backend.get_session(),\n",
    "    export_dir         = model_dir,\n",
    "    inputs             = {'input': model.inputs[0]},\n",
    "    outputs            = {'output': model.outputs[0]})\n",
    "\n",
    "# Compile using Neuron\n",
    "tfn.saved_model.compile(model_dir, compiled_model_dir)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "I52jQOyO8vAn"
   },
   "source": [
    "## Deploy on Inferentia\n",
    "\n",
    "Using same instance to deploy the model.\n",
    "In case of different deployment instance, launch a deployment inf1 instance and copy compiled model to the deployment inf1 instance.\n",
    "\n",
    "Download the example image, and install pillow module for inference on deployement instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl -O https://raw.githubusercontent.com/awslabs/mxnet-model-server/master/docs/images/kitten_small.jpg\n",
    "!pip install pillow  # Necessary for loading images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### After downloading the example image, run the inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.preprocessing import image\n",
    "from tensorflow.keras.applications import resnet50\n",
    "\n",
    "tf.keras.backend.set_image_data_format('channels_last')\n",
    "\n",
    "# Create input from image\n",
    "img_sgl = image.load_img('kitten_small.jpg', target_size=(224, 224))\n",
    "img_arr = image.img_to_array(img_sgl)\n",
    "img_arr2 = np.expand_dims(img_arr, axis=0)\n",
    "img_arr3 = resnet50.preprocess_input(img_arr2)\n",
    "\n",
    "# Load model\n",
    "COMPILED_MODEL_DIR = './ws_resnet50/resnet50_neuron/'\n",
    "predictor_inferentia = tf.contrib.predictor.from_saved_model(COMPILED_MODEL_DIR)\n",
    "\n",
    "# Run inference\n",
    "model_feed_dict={'input': img_arr3}\n",
    "infa_rslts = predictor_inferentia(model_feed_dict);\n",
    "\n",
    "# Display results\n",
    "print(resnet50.decode_predictions(infa_rslts[\"output\"], top=5)[0])\n",
    "\n",
    "# Sample output will look like below:\n",
    "#[('n02123045', 'tabby', 0.68817204), ('n02127052', 'lynx', 0.12701613), ('n02123159', 'tiger_cat', 0.08736559), ('n02124075', 'Egyptian_cat', 0.063844085), ('n02128757', 'snow_leopard', 0.009240591)]"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "default_view": {},
   "name": "Untitled",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3.8.9 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}