versions/1.9.1/api/python/docs/_sources/tutorials/deploy/inference/image_classification_jetson.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "67e24148",
   "metadata": {},
   "source": [
    "<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n",
    "<!--- or more contributor license agreements.  See the NOTICE file -->\n",
    "<!--- distributed with this work for additional information -->\n",
    "<!--- regarding copyright ownership.  The ASF licenses this file -->\n",
    "<!--- to you under the Apache License, Version 2.0 (the -->\n",
    "<!--- \"License\"); you may not use this file except in compliance -->\n",
    "<!--- with the License.  You may obtain a copy of the License at -->\n",
    "\n",
    "<!---   http://www.apache.org/licenses/LICENSE-2.0 -->\n",
    "\n",
    "<!--- Unless required by applicable law or agreed to in writing, -->\n",
    "<!--- software distributed under the License is distributed on an -->\n",
    "<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n",
    "<!--- KIND, either express or implied.  See the License for the -->\n",
    "<!--- specific language governing permissions and limitations -->\n",
    "<!--- under the License. -->\n",
    "\n",
    "# Image Classication using pretrained ResNet-50 model on Jetson module\n",
    "\n",
    "This tutorial shows how to install MXNet v1.x with Jetson support and use it to deploy a pre-trained MXNet model for image classification on a Jetson module.\n",
    "\n",
    "## What's in this tutorial?\n",
    "\n",
    "This tutorial shows how to:\n",
    "\n",
    "1. Install MXNet v1.x along with its dependencies on a Jetson module (This tutorial has been tested on Jetson Xavier AGX and Jetson Nano modules)\n",
    "\n",
    "2. Deploy a pre-trained MXNet model for image classifcation on the module\n",
    "\n",
    "## Who's this tutorial for?\n",
    "\n",
    "This tutorial would benefit developers working on Jetson modules implementing deep learning applications. It assumes that readers have a Jetson module setup with Jetpack installed, are familiar with the Jetson working environment and are somewhat familiar with deep learning using MXNet.\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "To complete this tutorial, you need:\n",
    "\n",
    "* A [Jetson module](https://developer.nvidia.com/embedded/develop/hardware) setup with [Jetpack 4.4](https://docs.nvidia.com/jetson/jetpack/release-notes/) installed using NVIDIA [SDK Manager](https://developer.nvidia.com/nvidia-sdk-manager)\n",
    "\n",
    "* An SSH connection to the module OR display and keyboard setup to directly open shell on the module\n",
    "\n",
    "* [Swapfile](https://help.ubuntu.com/community/SwapFaq) installed, especially on Jetson Nano for additional memory (increase memory if the inference script terminates with a `Killed` message)\n",
    "\n",
    "## Installing MXNet v1.x with Jetson support\n",
    "\n",
    "To install MXNet with Jetson support, you can follow the [installation guide](https://mxnet.apache.org/get_started/jetson_setup) on MXNet official website.\n",
    "\n",
    "Alternatively, you can also directly install MXNet v1.6 wheel with Jetson support, hosted on a public s3 bucket. Here are the steps to install this wheel:\n",
    "\n",
    "*WARNING: this MXNet wheel is provided for your convenience but it contains packages that are not provided nor endorsed by the Apache Software Foundation.\n",
    "As such, they might contain software components with more restrictive licenses than the Apache License and you'll need to decide whether they are appropriate for your usage. Like all Apache Releases, the\n",
    "official Apache MXNet (incubating) releases consist of source code only and are found at https://mxnet.apache.org/get_started/download .*\n",
    "\n",
    "We start by installing MXNet dependencies"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7811f034",
   "metadata": {},
   "source": [
    "```bash\n",
    "sudo apt-get update\n",
    "sudo apt-get install -y git build-essential libopenblas-dev libopencv-dev python3-pip\n",
    "sudo pip3 install -U pip\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aa4fe35",
   "metadata": {},
   "source": [
    "Then we download and install MXNet v1.6 wheel with Jetson support"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1348e89f",
   "metadata": {},
   "source": [
    "```bash\n",
    "wget https://mxnet-public.s3.us-east-2.amazonaws.com/install/jetson/1.6.0/mxnet_cu102-1.6.0-py2.py3-none-linux_aarch64.whl\n",
    "sudo pip3 install mxnet_cu102-1.6.0-py2.py3-none-linux_aarch64.whl\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28e93a0d",
   "metadata": {},
   "source": [
    "And we are done. You can test the installation now by importing mxnet from python3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8142bf1",
   "metadata": {},
   "source": [
    "```bash\n",
    ">>> python3 -c 'import mxnet'\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4aafdcd",
   "metadata": {},
   "source": [
    "## Running a pre-trained ResNet-50 model on Jetson\n",
    "\n",
    "We are now ready to run a pre-trained model and run inference on a Jetson module. In this tutorial we are using ResNet-50 model trained on Imagenet dataset. We run the following classification script with either cpu/gpu context using python3."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92ed301c",
   "metadata": {},
   "source": [
    "```python\n",
    "from mxnet import gluon\n",
    "import mxnet as mx\n",
    "\n",
    "# set context\n",
    "ctx = mx.gpu()\n",
    "\n",
    "# load pre-trained model\n",
    "net = gluon.model_zoo.vision.resnet50_v1(pretrained=True, ctx=ctx)\n",
    "net.hybridize(static_alloc=True, static_shape=True)\n",
    "\n",
    "# load labels\n",
    "lbl_path = gluon.utils.download('http://data.mxnet.io/models/imagenet/synset.txt')\n",
    "with open(lbl_path, 'r') as f:\n",
    "    labels = [l.rstrip() for l in f]\n",
    "\n",
    "# download and format image as (batch, RGB, width, height)\n",
    "img_path = gluon.utils.download('https://github.com/dmlc/web-data/blob/master/mxnet/doc/tutorials/python/predict_image/cat.jpg?raw=true')\n",
    "img = mx.image.imread(img_path)\n",
    "img = mx.image.imresize(img, 224, 224) # resize\n",
    "img = mx.image.color_normalize(img.astype(dtype='float32')/255,\n",
    "                               mean=mx.nd.array([0.485, 0.456, 0.406]),\n",
    "                               std=mx.nd.array([0.229, 0.224, 0.225])) # normalize\n",
    "img = img.transpose((2, 0, 1)) # channel first\n",
    "img = img.expand_dims(axis=0) # batchify\n",
    "img = img.as_in_context(ctx)\n",
    "\n",
    "prob = net(img).softmax() # predict and normalize output\n",
    "idx = prob.topk(k=5)[0] # get top 5 result\n",
    "for i in idx:\n",
    "    i = int(i.asscalar())\n",
    "    print('With prob = %.5f, it contains %s' % (prob[0,i].asscalar(), labels[i]))\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1251a13",
   "metadata": {},
   "source": [
    "After running the above script, you should get the following output showing the five classes that the image most relates to with probability:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f62f66a6",
   "metadata": {},
   "source": [
    "```bash\n",
    "With prob = 0.41940, it contains n02119789 kit fox, Vulpes macrotis\n",
    "With prob = 0.28096, it contains n02119022 red fox, Vulpes vulpes\n",
    "With prob = 0.06857, it contains n02124075 Egyptian cat\n",
    "With prob = 0.03046, it contains n02120505 grey fox, gray fox, Urocyon cinereoargenteus\n",
    "With prob = 0.02770, it contains n02441942 weasel\n",
    "```\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}