versions/1.9.1/api/python/docs/_sources/tutorials/packages/ndarray/02-ndarray-operations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0d00bbc1",
   "metadata": {},
   "source": [
    "<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n",
    "<!--- or more contributor license agreements.  See the NOTICE file -->\n",
    "<!--- distributed with this work for additional information -->\n",
    "<!--- regarding copyright ownership.  The ASF licenses this file -->\n",
    "<!--- to you under the Apache License, Version 2.0 (the -->\n",
    "<!--- \"License\"); you may not use this file except in compliance -->\n",
    "<!--- with the License.  You may obtain a copy of the License at -->\n",
    "\n",
    "<!---   http://www.apache.org/licenses/LICENSE-2.0 -->\n",
    "\n",
    "<!--- Unless required by applicable law or agreed to in writing, -->\n",
    "<!--- software distributed under the License is distributed on an -->\n",
    "<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n",
    "<!--- KIND, either express or implied.  See the License for the -->\n",
    "<!--- specific language governing permissions and limitations -->\n",
    "<!--- under the License. -->\n",
    "\n",
    "# NDArray Operations\n",
    "\n",
    "## Overview\n",
    "This guide will introduce you to MXNet's array operations.\n",
    "\n",
    "This content was extracted and simplified from the gluon tutorials in\n",
    "[Dive Into Deep Learning](https://d2l.ai/).\n",
    "\n",
    "## Prerequisites\n",
    "* [MXNet installed in a Python environment](/get_started).\n",
    "* Python 2.7.x or Python 3.x\n",
    "\n",
    "\n",
    "## Operations\n",
    "\n",
    "NDArray supports a large number of standard mathematical operations.\n",
    "Such as element-wise addition:\n",
    "<!-- keeping it\n",
    "easy -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b78c85a",
   "metadata": {},
   "source": [
    "```python\n",
    "import mxnet as mx\n",
    "from mxnet import nd\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "735f758c",
   "metadata": {},
   "source": [
    "```python\n",
    "x = nd.ones((3, 4))\n",
    "y = nd.random_normal(0, 1, shape=(3, 4))\n",
    "print('x=', x)\n",
    "print('y=', y)\n",
    "x = x + y\n",
    "print('x = x + y, x=', x)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f97aca0",
   "metadata": {},
   "source": [
    "Multiplication:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cccfd6c5",
   "metadata": {},
   "source": [
    "```python\n",
    "x = nd.array([1, 2, 3])\n",
    "y = nd.array([2, 2, 2])\n",
    "x * y\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6be83077",
   "metadata": {},
   "source": [
    "And exponentiation:\n",
    "<!-- with these next ones we'll just have to take your word\n",
    "for it... -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f130ec67",
   "metadata": {},
   "source": [
    "```python\n",
    "nd.exp(x)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efc6714f",
   "metadata": {},
   "source": [
    "We can also grab a matrix's transpose to compute a proper matrix-matrix product.\n",
    "<!-- because we need to do that before we have coffee every day... and you know\n",
    "how those dirty, improper matrixeses can be... -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d25c864f",
   "metadata": {},
   "source": [
    "```python\n",
    "nd.dot(x, y.T)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98edb518",
   "metadata": {},
   "source": [
    "## In-place operations\n",
    "\n",
    "In the previous\n",
    "example, every time we ran an operation, we allocated new memory to host its\n",
    "results. For example, if we write `y = x + y`, we will dereference the matrix\n",
    "that `y` used to point to and instead point it at the newly allocated memory. We\n",
    "can show this using Python's `id()` function, which tells us precisely which\n",
    "object a variable refers to.\n",
    "\n",
    "<!-- dereference is something C++ people would\n",
    "know but everyone else... not so much. What's the point? ;) get it? Put it in\n",
    "more context as to why you care about this and why this is in front of so much\n",
    "other material. Seems like an optimization topic best suited for later...\n",
    "###edit### we just talked about this, so I have better context. Now I\n",
    "understand, but your new reader will not. This should be covered in much more\n",
    "detail, and quite possibily in its own notebook since I think it will help to\n",
    "show some gotchas like you mentioned verbally. I am still leaning toward\n",
    "delaying the introduction of this topic....-->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd236187",
   "metadata": {},
   "source": [
    "```python\n",
    "print('y=', y)\n",
    "print('id(y):', id(y))\n",
    "y = y + x\n",
    "print('after y=y+x, y=', y)\n",
    "print('id(y):', id(y))\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88b4038e",
   "metadata": {},
   "source": [
    "We can assign the result to a previously allocated array with slice notation,\n",
    "e.g., `result[:] = ...`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "572d44d6",
   "metadata": {},
   "source": [
    "```python\n",
    "print('x=', x)\n",
    "z = nd.zeros_like(x)\n",
    "print('z is zeros_like x, z=', z)\n",
    "print('id(z):', id(z))\n",
    "print('y=', y)\n",
    "z[:] = x + y\n",
    "print('z[:] = x + y, z=', z)\n",
    "print('id(z) is the same as before:', id(z))\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "668dcb79",
   "metadata": {},
   "source": [
    "However, `x+y` here will still allocate a temporary buffer to store the result\n",
    "before copying it to z. To make better use of memory, we can perform operations\n",
    "in place, avoiding temporary buffers. To do this we specify the `out` keyword\n",
    "argument every operator supports:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf3d364c",
   "metadata": {},
   "source": [
    "```python\n",
    "print('x=', x, 'is in id(x):', id(x))\n",
    "print('y=', y, 'is in id(y):', id(y))\n",
    "print('z=', z, 'is in id(z):', id(z))\n",
    "nd.elemwise_add(x, y, out=z)\n",
    "print('after nd.elemwise_add(x, y, out=z), x=', x, 'is in id(x):', id(x))\n",
    "print('after nd.elemwise_add(x, y, out=z), y=', y, 'is in id(y):', id(y))\n",
    "print('after nd.elemwise_add(x, y, out=z), z=', z, 'is in id(z):', id(z))\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c64e89b",
   "metadata": {},
   "source": [
    "If we're not planning to re-use ``x``, then we can assign the result to ``x``\n",
    "itself. There are two ways to do this in MXNet.\n",
    "1. By using slice notation x[:]\n",
    "= x op y\n",
    "2. By using the op-equals operators like `+=`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89554750",
   "metadata": {},
   "source": [
    "```python\n",
    "print('x=', x, 'is in id(x):', id(x))\n",
    "x += y\n",
    "print('x=', x, 'is in id(x):', id(x))\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a83f4316",
   "metadata": {},
   "source": [
    "## Slicing\n",
    "MXNet NDArrays support slicing in all the ridiculous ways you might\n",
    "imagine accessing your data. For a quick review:\n",
    "\n",
    "* items start through end-1: a[start:end]\n",
    "* items start through the rest of the\n",
    "array: a[start:]\n",
    "* items from the beginning through end-1: a[:end]\n",
    "* a copy of\n",
    "the whole array: a[:]\n",
    "\n",
    "Here's an example of reading the second and third rows from `x`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43a3b150",
   "metadata": {},
   "source": [
    "```python\n",
    "x = nd.array([1, 2, 3])\n",
    "print('1D complete array, x=', x)\n",
    "s = x[1:3]\n",
    "print('slicing the 2nd and 3rd elements, s=', s)\n",
    "x = nd.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])\n",
    "print('multi-D complete array, x=', x)\n",
    "s = x[1:3]\n",
    "print('slicing the 2nd and 3rd elements, s=', s)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0854760",
   "metadata": {},
   "source": [
    "Now let's try writing to a specific element."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0e3299b",
   "metadata": {},
   "source": [
    "```python\n",
    "print('original x, x=', x)\n",
    "x[2] = 9.0\n",
    "print('replaced entire row with x[2] = 9.0, x=', x)\n",
    "x[0,2] = 9.0\n",
    "print('replaced specific element with x[0,2] = 9.0, x=', x)\n",
    "x[1:2,1:3] = 5.0\n",
    "print('replaced range of elements with x[1:2,1:3] = 5.0, x=', x)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdca1b7d",
   "metadata": {},
   "source": [
    "Multi-dimensional slicing is also supported."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54e8f113",
   "metadata": {},
   "source": [
    "```python\n",
    "x = nd.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])\n",
    "print('original x, x=', x)\n",
    "s = x[1:2,1:3]\n",
    "print('plucking specific elements with x[1:2,1:3]', s)\n",
    "s = x[:,:1]\n",
    "print('first column with x[:,:1]', s)\n",
    "s = x[:1,:]\n",
    "print('first row with x[:1,:]', s)\n",
    "s = x[:,3:]\n",
    "print('last column with x[:,3:]', s)\n",
    "s = x[2:,:]\n",
    "print('last row with x[2:,:]', s)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae3fdbde",
   "metadata": {},
   "source": [
    "## Broadcasting\n",
    "\n",
    "You might wonder, what happens if you add a vector `y` to a\n",
    "matrix `X`? These operations, where we compose a low dimensional array `y` with\n",
    "a high-dimensional array `X` invoke a functionality called broadcasting. First\n",
    "we'll introduce `.arange` which is useful for filling out an array with evenly\n",
    "spaced data. Then we can take the low-dimensional array and duplicate it along\n",
    "any axis with dimension $1$ to match the shape of the high dimensional array.\n",
    "Consider the following example.\n",
    "\n",
    "Comment (visible to demonstrate with font):\n",
    "dimension one(1)? Or L(elle) or l(lil elle) or I(eye) or... ? We don't even use\n",
    "the notation later, so did it need to be introduced here?\n",
    "\n",
    "<!--Also, if you use\n",
    "a shape like (3,3) you lose some of the impact and miss some errors if people\n",
    "play with the values. Better to have a distinct shape so that it is more obvious\n",
    "what is happening and what can break.-->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce69caf9",
   "metadata": {},
   "source": [
    "```python\n",
    "x = nd.ones(shape=(3,6))\n",
    "print('x = ', x)\n",
    "y = nd.arange(6)\n",
    "print('y = ', y)\n",
    "print('x + y = ', x + y)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c67586fa",
   "metadata": {},
   "source": [
    "While `y` is initially of shape $6$,\n",
    "MXNet infers its shape to be (1,6),\n",
    "and then broadcasts along the rows to form a (3,6) matrix).\n",
    "You might wonder, why did MXNet choose to interpret `y` as a (1,6) matrix and not (6,1).\n",
    "That's because broadcasting prefers to duplicate along the left most axis.\n",
    "We can alter this behavior by explicitly giving `y` a $2$D shape using `.reshape`.\n",
    "You can also chain `.arange` and `.reshape` to do this in one step."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1edba4d2",
   "metadata": {},
   "source": [
    "```python\n",
    "y = y.reshape((3,1))\n",
    "print('y = ', y)\n",
    "print('x + y = ', x+y)\n",
    "y = nd.arange(6).reshape((3,1))\n",
    "print('y = ', y)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e3bfc52",
   "metadata": {},
   "source": [
    "## Converting from MXNet NDArray to NumPy\n",
    "Converting MXNet NDArrays to and from\n",
    "NumPy is easy. The converted arrays do not share memory."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adb4b6b7",
   "metadata": {},
   "source": [
    "```python\n",
    "a = x.asnumpy()\n",
    "type(a)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "799657e1",
   "metadata": {},
   "source": [
    "```python\n",
    "y = nd.array(a)\n",
    "print('id(a)=', id(a), 'id(x)=', id(x), 'id(y)=', id(y))\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34888a96",
   "metadata": {},
   "source": [
    "## Next Up\n",
    "\n",
    "[NDArray Contexts](03-ndarray-contexts.md)"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}