| { |
| "cells": [ |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n", |
| "<!--- or more contributor license agreements. See the NOTICE file -->\n", |
| "<!--- distributed with this work for additional information -->\n", |
| "<!--- regarding copyright ownership. The ASF licenses this file -->\n", |
| "<!--- to you under the Apache License, Version 2.0 (the -->\n", |
| "<!--- \"License\"); you may not use this file except in compliance -->\n", |
| "<!--- with the License. You may obtain a copy of the License at -->\n", |
| "\n", |
| "<!--- http://www.apache.org/licenses/LICENSE-2.0 -->\n", |
| "\n", |
| "<!--- Unless required by applicable law or agreed to in writing, -->\n", |
| "<!--- software distributed under the License is distributed on an -->\n", |
| "<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n", |
| "<!--- KIND, either express or implied. See the License for the -->\n", |
| "<!--- specific language governing permissions and limitations -->\n", |
| "<!--- under the License. -->\n", |
| "\n", |
| "# NDArray Operations\n", |
| "\n", |
| "## Overview\n", |
| "This guide will introduce you to MXNet's array operations.\n", |
| "\n", |
| "This content was extracted and simplified from the gluon tutorials in\n", |
| "[Dive Into Deep Learning](https://d2l.ai/).\n", |
| "\n", |
| "## Prerequisites\n", |
| "* [MXNet installed in a Python environment](/get_started).\n", |
| "* Python 2.7.x or Python 3.x\n", |
| "\n", |
| "\n", |
| "## Operations\n", |
| "\n", |
| "NDArray supports a large number of standard mathematical operations.\n", |
| "Such as element-wise addition:\n", |
| "<!-- keeping it\n", |
| "easy -->" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "import mxnet as mx\n", |
| "from mxnet import nd" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "x = nd.ones((3, 4))\n", |
| "y = nd.random_normal(0, 1, shape=(3, 4))\n", |
| "print('x=', x)\n", |
| "print('y=', y)\n", |
| "x = x + y\n", |
| "print('x = x + y, x=', x)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "Multiplication:" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "x = nd.array([1, 2, 3])\n", |
| "y = nd.array([2, 2, 2])\n", |
| "x * y" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "And exponentiation:\n", |
| "<!-- with these next ones we'll just have to take your word\n", |
| "for it... -->" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "nd.exp(x)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "We can also grab a matrix's transpose to compute a proper matrix-matrix product.\n", |
| "<!-- because we need to do that before we have coffee every day... and you know\n", |
| "how those dirty, improper matrixeses can be... -->" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "nd.dot(x, y.T)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "## In-place operations\n", |
| "\n", |
| "In the previous\n", |
| "example, every time we ran an operation, we allocated new memory to host its\n", |
| "results. For example, if we write `y = x + y`, we will dereference the matrix\n", |
| "that `y` used to point to and instead point it at the newly allocated memory. We\n", |
| "can show this using Python's `id()` function, which tells us precisely which\n", |
| "object a variable refers to.\n", |
| "\n", |
| "<!-- dereference is something C++ people would\n", |
| "know but everyone else... not so much. What's the point? ;) get it? Put it in\n", |
| "more context as to why you care about this and why this is in front of so much\n", |
| "other material. Seems like an optimization topic best suited for later...\n", |
| "###edit### we just talked about this, so I have better context. Now I\n", |
| "understand, but your new reader will not. This should be covered in much more\n", |
| "detail, and quite possibily in its own notebook since I think it will help to\n", |
| "show some gotchas like you mentioned verbally. I am still leaning toward\n", |
| "delaying the introduction of this topic....-->" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "print('y=', y)\n", |
| "print('id(y):', id(y))\n", |
| "y = y + x\n", |
| "print('after y=y+x, y=', y)\n", |
| "print('id(y):', id(y))" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "We can assign the result to a previously allocated array with slice notation,\n", |
| "e.g., `result[:] = ...`." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "print('x=', x)\n", |
| "z = nd.zeros_like(x)\n", |
| "print('z is zeros_like x, z=', z)\n", |
| "print('id(z):', id(z))\n", |
| "print('y=', y)\n", |
| "z[:] = x + y\n", |
| "print('z[:] = x + y, z=', z)\n", |
| "print('id(z) is the same as before:', id(z))" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "However, `x+y` here will still allocate a temporary buffer to store the result\n", |
| "before copying it to z. To make better use of memory, we can perform operations\n", |
| "in place, avoiding temporary buffers. To do this we specify the `out` keyword\n", |
| "argument every operator supports:" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "print('x=', x, 'is in id(x):', id(x))\n", |
| "print('y=', y, 'is in id(y):', id(y))\n", |
| "print('z=', z, 'is in id(z):', id(z))\n", |
| "nd.elemwise_add(x, y, out=z)\n", |
| "print('after nd.elemwise_add(x, y, out=z), x=', x, 'is in id(x):', id(x))\n", |
| "print('after nd.elemwise_add(x, y, out=z), y=', y, 'is in id(y):', id(y))\n", |
| "print('after nd.elemwise_add(x, y, out=z), z=', z, 'is in id(z):', id(z))" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "If we're not planning to re-use ``x``, then we can assign the result to ``x``\n", |
| "itself. There are two ways to do this in MXNet.\n", |
| "1. By using slice notation x[:]\n", |
| "= x op y\n", |
| "2. By using the op-equals operators like `+=`" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "print('x=', x, 'is in id(x):', id(x))\n", |
| "x += y\n", |
| "print('x=', x, 'is in id(x):', id(x))" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "## Slicing\n", |
| "MXNet NDArrays support slicing in all the ridiculous ways you might\n", |
| "imagine accessing your data. For a quick review:\n", |
| "\n", |
| "* items start through end-1: a[start:end]\n", |
| "* items start through the rest of the\n", |
| "array: a[start:]\n", |
| "* items from the beginning through end-1: a[:end]\n", |
| "* a copy of\n", |
| "the whole array: a[:]\n", |
| "\n", |
| "Here's an example of reading the second and third rows from `x`." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "x = nd.array([1, 2, 3])\n", |
| "print('1D complete array, x=', x)\n", |
| "s = x[1:3]\n", |
| "print('slicing the 2nd and 3rd elements, s=', s)\n", |
| "x = nd.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])\n", |
| "print('multi-D complete array, x=', x)\n", |
| "s = x[1:3]\n", |
| "print('slicing the 2nd and 3rd elements, s=', s)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "Now let's try writing to a specific element." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "print('original x, x=', x)\n", |
| "x[2] = 9.0\n", |
| "print('replaced entire row with x[2] = 9.0, x=', x)\n", |
| "x[0,2] = 9.0\n", |
| "print('replaced specific element with x[0,2] = 9.0, x=', x)\n", |
| "x[1:2,1:3] = 5.0\n", |
| "print('replaced range of elements with x[1:2,1:3] = 5.0, x=', x)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "Multi-dimensional slicing is also supported." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "x = nd.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])\n", |
| "print('original x, x=', x)\n", |
| "s = x[1:2,1:3]\n", |
| "print('plucking specific elements with x[1:2,1:3]', s)\n", |
| "s = x[:,:1]\n", |
| "print('first column with x[:,:1]', s)\n", |
| "s = x[:1,:]\n", |
| "print('first row with x[:1,:]', s)\n", |
| "s = x[:,3:]\n", |
| "print('last column with x[:,3:]', s)\n", |
| "s = x[2:,:]\n", |
| "print('last row with x[2:,:]', s)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "## Broadcasting\n", |
| "\n", |
| "You might wonder, what happens if you add a vector `y` to a\n", |
| "matrix `X`? These operations, where we compose a low dimensional array `y` with\n", |
| "a high-dimensional array `X` invoke a functionality called broadcasting. First\n", |
| "we'll introduce `.arange` which is useful for filling out an array with evenly\n", |
| "spaced data. Then we can take the low-dimensional array and duplicate it along\n", |
| "any axis with dimension $1$ to match the shape of the high dimensional array.\n", |
| "Consider the following example.\n", |
| "\n", |
| "Comment (visible to demonstrate with font):\n", |
| "dimension one(1)? Or L(elle) or l(lil elle) or I(eye) or... ? We don't even use\n", |
| "the notation later, so did it need to be introduced here?\n", |
| "\n", |
| "<!--Also, if you use\n", |
| "a shape like (3,3) you lose some of the impact and miss some errors if people\n", |
| "play with the values. Better to have a distinct shape so that it is more obvious\n", |
| "what is happening and what can break.-->" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "x = nd.ones(shape=(3,6))\n", |
| "print('x = ', x)\n", |
| "y = nd.arange(6)\n", |
| "print('y = ', y)\n", |
| "print('x + y = ', x + y)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "While `y` is initially of shape $6$,\n", |
| "MXNet infers its shape to be (1,6),\n", |
| "and then broadcasts along the rows to form a (3,6) matrix).\n", |
| "You might wonder, why did MXNet choose to interpret `y` as a (1,6) matrix and not (6,1).\n", |
| "That's because broadcasting prefers to duplicate along the left most axis.\n", |
| "We can alter this behavior by explicitly giving `y` a $2$D shape using `.reshape`.\n", |
| "You can also chain `.arange` and `.reshape` to do this in one step." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "y = y.reshape((3,1))\n", |
| "print('y = ', y)\n", |
| "print('x + y = ', x+y)\n", |
| "y = nd.arange(6).reshape((3,1))\n", |
| "print('y = ', y)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "## Converting from MXNet NDArray to NumPy\n", |
| "Converting MXNet NDArrays to and from\n", |
| "NumPy is easy. The converted arrays do not share memory." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "a = x.asnumpy()\n", |
| "type(a)" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": null, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "y = nd.array(a)\n", |
| "print('id(a)=', id(a), 'id(x)=', id(x), 'id(y)=', id(y))" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "## Next Up\n", |
| "\n", |
| "[NDArray Contexts](03-ndarray-contexts.md)" |
| ] |
| } |
| ], |
| "metadata": { |
| "language_info": { |
| "name": "python" |
| } |
| }, |
| "nbformat": 4, |
| "nbformat_minor": 4 |
| } |