versions/1.9.1/api/python/docs/_sources/tutorials/deploy/export/onnx.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "81fcd2de",
   "metadata": {},
   "source": [
    "<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n",
    "<!--- or more contributor license agreements.  See the NOTICE file -->\n",
    "<!--- distributed with this work for additional information -->\n",
    "<!--- regarding copyright ownership.  The ASF licenses this file -->\n",
    "<!--- to you under the Apache License, Version 2.0 (the -->\n",
    "<!--- \"License\"); you may not use this file except in compliance -->\n",
    "<!--- with the License.  You may obtain a copy of the License at -->\n",
    "\n",
    "<!---   http://www.apache.org/licenses/LICENSE-2.0 -->\n",
    "\n",
    "<!--- Unless required by applicable law or agreed to in writing, -->\n",
    "<!--- software distributed under the License is distributed on an -->\n",
    "<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n",
    "<!--- KIND, either express or implied.  See the License for the -->\n",
    "<!--- specific language governing permissions and limitations -->\n",
    "<!--- under the License. -->\n",
    "\n",
    "# Exporting to ONNX format\n",
    "\n",
    "[Open Neural Network Exchange (ONNX)](https://github.com/onnx/onnx) provides an open source format for AI models. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types. In the MXNet 1.9 release, the MXNet-to-ONNX export module (mx2onnx) has received a major update with new features such as dynamic input shapes and better operator and model coverages. Please visit the [ONNX Export Support for MXNet](https://github.com/apache/mxnet/tree/v1.x/python/mxnet/onnx#onnx-export-support-for-mxnet) page for more information.\n",
    "\n",
    "In this tutorial, we will learn how to use the mx2onnx exporter on pre-trained models.\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "To run the tutorial we will need to have installed the following python modules:\n",
    "- [MXNet >= 1.9.0](/get_started) _OR_ an earlier MXNet version + [the mx2onnx wheel](https://github.com/apache/mxnet/tree/v1.x/python/mxnet/onnx#installation)\n",
    "- [onnx >= 1.7.0](https://github.com/onnx/onnx#installation)\n",
    "\n",
    "*Note:* The latest mx2onnx exporting module is tested with ONNX op set version 12 or later, which corresponds to ONNX version 1.7 or later. Use of ealier ONNX versions may still work on some simple models, but again this is not tested."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efc5816d",
   "metadata": {},
   "source": [
    "```python\n",
    "import mxnet as mx\n",
    "import numpy as np\n",
    "import logging\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a21dae41",
   "metadata": {},
   "source": [
    "## Download a model from the MXNet model zoo\n",
    "\n",
    "We can download a pre-trained ResNet-18 [ImageNet](http://www.image-net.org/) model from the [MXNet Model Zoo](/api/python/docs/api/gluon/model_zoo/index.html).\n",
    "We will also download a synset file to match the labels."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e16e351",
   "metadata": {},
   "source": [
    "```python\n",
    "# Download pre-trained resnet model - json and params by running following code.\n",
    "path='http://data.mxnet.io/models/imagenet/'\n",
    "[mx.test_utils.download(path+'resnet/18-layers/resnet-18-0000.params'),\n",
    " mx.test_utils.download(path+'resnet/18-layers/resnet-18-symbol.json'),\n",
    " mx.test_utils.download(path+'synset.txt')]\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bb501eb",
   "metadata": {},
   "source": [
    "## MXNet to ONNX exporter (mx2onnx) API\n",
    "\n",
    "Now let's check MXNet's `export_model` API."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a41ec373",
   "metadata": {},
   "source": [
    "```python\n",
    "help(mx.onnx.export_model)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c95f99f8",
   "metadata": {},
   "source": [
    "Output:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee761632",
   "metadata": {},
   "source": [
    "```text\n",
    "Help on function export_model in module mxnet.onnx.mx2onnx._export_model:\n",
    "\n",
    "export_model(sym, params, in_shapes=None, in_types=<class 'numpy.float32'>, onnx_file_path='model.onnx', verbose=False, dynamic=False, dynamic_input_shapes=None, run_shape_inference=False, input_type=None, input_shape=None)\n",
    "    Exports the MXNet model file, passed as a parameter, into ONNX model.\n",
    "    Accepts both symbol,parameter objects as well as json and params filepaths as input.\n",
    "    Operator support and coverage -\n",
    "    https://github.com/apache/mxnet/tree/v1.x/python/mxnet/onnx#operator-support-matrix\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    sym : str or symbol object\n",
    "        Path to the json file or Symbol object\n",
    "    params : str or dict or list of dict\n",
    "        str - Path to the params file\n",
    "        dict - params dictionary (Including both arg_params and aux_params)\n",
    "        list - list of length 2 that contains arg_params and aux_params\n",
    "    in_shapes : List of tuple\n",
    "        Input shape of the model e.g [(1,3,224,224)]\n",
    "    in_types : data type or list of data types\n",
    "        Input data type e.g. np.float32, or [np.float32, np.int32]\n",
    "    onnx_file_path : str\n",
    "        Path where to save the generated onnx file\n",
    "    verbose : Boolean\n",
    "        If True will print logs of the model conversion\n",
    "    dynamic: Boolean\n",
    "        If True will allow for dynamic input shapes to the model\n",
    "    dynamic_input_shapes: list of tuple\n",
    "        Specifies the dynamic input_shapes. If None then all dimensions are set to None\n",
    "    run_shape_inference : Boolean\n",
    "        If True will run shape inference on the model\n",
    "    input_type : data type or list of data types\n",
    "        This is the old name of in_types. We keep this parameter name for backward compatibility\n",
    "    input_shape : List of tuple\n",
    "        This is the old name of in_shapes. We keep this parameter name for backward compatibility\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    onnx_file_path : str\n",
    "        Onnx file path\n",
    "    \n",
    "    Notes\n",
    "    -----\n",
    "    This method is available when you ``import mxnet.onnx``\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "509337c8",
   "metadata": {},
   "source": [
    "The `export_model` API can accept a MXNet model in one of the following ways.\n",
    "\n",
    "1. MXNet's exported json and params files:\n",
    "    * This is useful if we have pre-trained models and we want to convert them to ONNX format.\n",
    "2. MXNet sym, params objects:\n",
    "    * This is useful if we are training a model. At the end of training, we just need to invoke the `export_model` function and provide the sym and params objects as inputs to save the model in ONNX format. The params can be either a single object that contains both argument and auxiliary parameters, or a list that includes arg_parmas and aux_params objects\n",
    "\n",
    "Since we have downloaded pre-trained model files, we will use the `export_model` API by passing in the paths of the symbol and params files.\n",
    "\n",
    "## Use mx2onnx to export the model\n",
    "\n",
    "We will use the downloaded pre-trained model files (sym, params) and define a few more parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cca3cc90",
   "metadata": {},
   "source": [
    "```python\n",
    "# Downloaded input symbol and params files\n",
    "sym = './resnet-18-symbol.json'\n",
    "params = './resnet-18-0000.params'\n",
    "\n",
    "# Standard Imagenet input - 3 channels, 224 * 224\n",
    "in_shapes = [(1, 3, 224, 224)]\n",
    "in_types = [np.float32]\n",
    "\n",
    "# Path of the output file\n",
    "onnx_file = './mxnet_exported_resnet18.onnx'\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6b65c9b",
   "metadata": {},
   "source": [
    "We have defined the input parameters required for the `export_model` API. Now, we are ready to covert the MXNet model into ONNX format."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07336276",
   "metadata": {},
   "source": [
    "```python\n",
    "# Invoke export model API. It returns path of the converted onnx model\n",
    "converted_model_path = mx.onnx.export_model(sym, params, in_shapes, in_types, onnx_file)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2849b998",
   "metadata": {},
   "source": [
    "This API returns the path of the converted model which you can later use to run inference with or import the model into other frameworks. Please refer to [mx2onnx](https://github.com/apache/mxnet/tree/v1.x/python/mxnet/onnx#apis) for more details about the API.\n",
    "\n",
    "## Dynamic input shapes\n",
    "The mx2onnx module also supports dynamic input shapes. We can set `dynamic=True` to turn it on. Note that even with dynamic shapes, a set of static input shapes still need to be specified in `in_shapes`; on top of that, we'll also need to specify which dimensions of the input shapes are dynamic in `dynamic_input_shapes`. We can simply set the dynamic dimensions as `None`, e.g. `(1, 3, None, None)`, or use strings in place of the `None`'s for better understandability in the exported onnx graph, e.g. `(1, 3, 'Height', 'Width')`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09fce9e8",
   "metadata": {},
   "source": [
    "```python\n",
    "# The first input dimension will be dynamic in this case\n",
    "dynamic_input_shapes = [(None, 3, 224, 224)]\n",
    "converted_model_path = mx.onnx.export_model(sym, params, in_shapes, in_types, onnx_file,\n",
    "                                            dynamic=True, dynamic_input_shapes=dynamic_input_shapes)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a6dbb0d",
   "metadata": {},
   "source": [
    "## Validate the exported ONNX model\n",
    "\n",
    "Now that we have the converted model, we can validate its correctness with the ONNX checker tool."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46ba420c",
   "metadata": {},
   "source": [
    "```python\n",
    "from onnx import checker\n",
    "import onnx\n",
    "\n",
    "# Load the ONNX model\n",
    "model_proto = onnx.load_model(converted_model_path)\n",
    "\n",
    "# Check if the converted ONNX protobuf is valid\n",
    "checker.check_graph(model_proto.graph)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45dd2d28",
   "metadata": {},
   "source": [
    "Now that the model passes the check (hopefully :)), we can run it with inference frameworks or import it into other deep learning frameworks!\n",
    "\n",
    "## Simplify the exported ONNX model\n",
    "\n",
    "Okay, we already have the exported ONNX model now, but it may not be the end of the story. Due to differences in MXNet's and ONNX's operator specifications, sometimes helper operators/nodes will need to be created to help construct the ONNX graph from the MXNet blueprint. In that sense, we recommend our users to checkout [onnx-simplifier](https://github.com/daquexian/onnx-simplifier), which can greatly simplify the exported ONNX model by techniques such as constant folding, operator fusion and more."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}