Google Git
Sign in
apache / mxnet-site / d5c5a235eef72f64ccaf224b834a20ef3287469b / . / api / python / docs / _sources / tutorials / getting-started / crash-course / 7-use-gpus.ipynb
blob: 547302b740b729d2b790a080cfc15492512dc667 [file] [log] [blame]
{
"cells": [
{
"cell_type": "markdown",
"id": "d5265233",
"metadata": {},
"source": [
"<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n",
"<!--- or more contributor license agreements. See the NOTICE file -->\n",
"<!--- distributed with this work for additional information -->\n",
"<!--- regarding copyright ownership. The ASF licenses this file -->\n",
"<!--- to you under the Apache License, Version 2.0 (the -->\n",
"<!--- \"License\"); you may not use this file except in compliance -->\n",
"<!--- with the License. You may obtain a copy of the License at -->\n",
"\n",
"<!--- http://www.apache.org/licenses/LICENSE-2.0 -->\n",
"\n",
"<!--- Unless required by applicable law or agreed to in writing, -->\n",
"<!--- software distributed under the License is distributed on an -->\n",
"<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n",
"<!--- KIND, either express or implied. See the License for the -->\n",
"<!--- specific language governing permissions and limitations -->\n",
"<!--- under the License. -->\n",
"\n",
"# Step 7: Load and Run a NN using GPU\n",
"\n",
"In this step, you will learn how to use graphics processing units (GPUs) with MXNet. If you use GPUs to train and deploy neural networks, you may be able to train or perform inference quicker than with central processing units (CPUs).\n",
"\n",
"## Prerequisites\n",
"\n",
"Before you start the steps, make sure you have at least one Nvidia GPU on your machine and make sure that you have CUDA properly installed. GPUs from AMD and Intel are not supported. Additionally, you will need to install the GPU-enabled version of MXNet. You can find information about how to install the GPU version of MXNet for your system [here](https://mxnet.apache.org/versions/1.4.1/install/ubuntu_setup.html).\n",
"\n",
"You can use the following command to view the number GPUs that are available to MXNet."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "66e46547",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from mxnet import np, npx, gluon, autograd\n",
"from mxnet.gluon import nn\n",
"import time\n",
"npx.set_np()\n",
"\n",
"npx.num_gpus() #This command provides the number of GPUs MXNet can access"
]
},
{
"cell_type": "markdown",
"id": "842d1e01",
"metadata": {},
"source": [
"## Allocate data to a GPU\n",
"\n",
"MXNet's ndarray is very similar to NumPy's. One major difference is that MXNet's ndarray has a `device` attribute specifying which device an array is on. By default, arrays are stored on `npx.cpu()`. To change it to the first GPU, you can use the following code, `npx.gpu()` or `npx.gpu(0)` to indicate the first GPU."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "3fc4931b",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[04:21:32] /work/mxnet/src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for GPU\n"
]
},
{
"data": {
"text/plain": [
"array([[1., 1., 1., 1.],\n",
" [1., 1., 1., 1.],\n",
" [1., 1., 1., 1.]], device=gpu(0))"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gpu = npx.gpu() if npx.num_gpus() > 0 else npx.cpu()\n",
"x = np.ones((3,4), device=gpu)\n",
"x"
]
},
{
"cell_type": "markdown",
"id": "5c9d14ea",
"metadata": {},
"source": [
"If you're using a CPU, MXNet allocates data on the main memory and tries to use as many CPU cores as possible. If there are multiple GPUs, MXNet will tell you which GPUs the ndarray is allocated on.\n",
"\n",
"Assuming there is at least two GPUs. You can create another ndarray and assign it to a different GPU. If you only have one GPU, then you will get an error trying to run this code. In the example code here, you will copy `x` to the second GPU, `npx.gpu(1)`:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "d1c72d38",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[04:21:32] /work/mxnet/src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for CPU\n"
]
},
{
"data": {
"text/plain": [
"array([[1., 1., 1., 1.],\n",
" [1., 1., 1., 1.],\n",
" [1., 1., 1., 1.]])"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gpu_1 = npx.gpu(1) if npx.num_gpus() > 1 else npx.cpu()\n",
"x.copyto(gpu_1)"
]
},
{
"cell_type": "markdown",
"id": "986069fa",
"metadata": {},
"source": [
"MXNet requries that users explicitly move data between devices. But several operators such as `print`, and `asnumpy`, will implicitly move data to main memory.\n",
"\n",
"## Choosing GPU Ids\n",
"If you have multiple GPUs on your machine, MXNet can access each of them through 0-indexing with `npx`. As you saw before, the first GPU was accessed using `npx.gpu(0)`, and the second using `npx.gpu(1)`. This extends to however many GPUs your machine has. So if your machine has eight GPUs, the last GPU is accessed using `npx.gpu(7)`. This allows you to select which GPUs to use for operations and training. You might find it particularly useful when you want to leverage multiple GPUs while training neural networks.\n",
"\n",
"## Run an operation on a GPU\n",
"\n",
"To perform an operation on a particular GPU, you only need to guarantee that the input of an operation is already on that GPU. The output is allocated on the same GPU as well. Almost all operators in the `np` and `npx` module support running on a GPU."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "70f151a1",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[1.7707204, 1.6089329, 1.3154027, 1.416174 ],\n",
" [1.2215444, 1.203621 , 1.5886476, 1.3864298],\n",
" [1.0913873, 1.1142501, 1.8820239, 1.1949301]], device=gpu(0))"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y = np.random.uniform(size=(3,4), device=gpu)\n",
"x + y"
]
},
{
"cell_type": "markdown",
"id": "241b44f3",
"metadata": {},
"source": [
"Remember that if the inputs are not on the same GPU, you will get an error.\n",
"\n",
"## Run a neural network on a GPU\n",
"\n",
"To run a neural network on a GPU, you only need to copy and move the input data and parameters to the GPU. To demonstrate this you can reuse the previously defined LeafNetwork in [Training Neural Networks](6-train-nn.md). The following code example shows this."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "ae538408",
"metadata": {},
"outputs": [],
"source": [
"# The convolutional block has a convolution layer, a max pool layer and a batch normalization layer\n",
"def conv_block(filters, kernel_size=2, stride=2, batch_norm=True):\n",
" conv_block = nn.HybridSequential()\n",
" conv_block.add(nn.Conv2D(channels=filters, kernel_size=kernel_size, activation='relu'),\n",
" nn.MaxPool2D(pool_size=4, strides=stride))\n",
" if batch_norm:\n",
" conv_block.add(nn.BatchNorm())\n",
" return conv_block\n",
"\n",
"# The dense block consists of a dense layer and a dropout layer\n",
"def dense_block(neurons, activation='relu', dropout=0.2):\n",
" dense_block = nn.HybridSequential()\n",
" dense_block.add(nn.Dense(neurons, activation=activation))\n",
" if dropout:\n",
" dense_block.add(nn.Dropout(dropout))\n",
" return dense_block\n",
"\n",
"# Create neural network blueprint using the blocks\n",
"class LeafNetwork(nn.HybridBlock):\n",
" def __init__(self):\n",
" super(LeafNetwork, self).__init__()\n",
" self.conv1 = conv_block(32)\n",
" self.conv2 = conv_block(64)\n",
" self.conv3 = conv_block(128)\n",
" self.flatten = nn.Flatten()\n",
" self.dense1 = dense_block(100)\n",
" self.dense2 = dense_block(10)\n",
" self.dense3 = nn.Dense(2)\n",
"\n",
" def forward(self, batch):\n",
" batch = self.conv1(batch)\n",
" batch = self.conv2(batch)\n",
" batch = self.conv3(batch)\n",
" batch = self.flatten(batch)\n",
" batch = self.dense1(batch)\n",
" batch = self.dense2(batch)\n",
" batch = self.dense3(batch)\n",
"\n",
" return batch"
]
},
{
"cell_type": "markdown",
"id": "41a4b6b3",
"metadata": {},
"source": [
"Load the saved parameters onto GPU 0 directly as shown below; additionally, you could use `net.collect_params().reset_device(gpu)` to change the device."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "bf5ad7db",
"metadata": {},
"outputs": [],
"source": [
"net = LeafNetwork()\n",
"net.load_parameters('leaf_models.params', device=gpu)"
]
},
{
"cell_type": "markdown",
"id": "0dd2917a",
"metadata": {},
"source": [
"Use the following command to create input data on GPU 0. The forward function will then run on GPU 0."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "b6db6614",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[04:21:32] /work/mxnet/src/operator/cudnn_ops.cc:421: Auto-tuning cuDNN op, set MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[04:21:33] /work/mxnet/src/operator/cudnn_ops.cc:421: Auto-tuning cuDNN op, set MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable\n"
]
},
{
"data": {
"text/plain": [
"array([[ 5.52604 , -0.286441]], device=gpu(0))"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = np.random.uniform(size=(1, 3, 128, 128), device=gpu)\n",
"net(x)"
]
},
{
"cell_type": "markdown",
"id": "b0a0f4f8",
"metadata": {},
"source": [
"## Training with multiple GPUs\n",
"\n",
"Finally, you will see how you can use multiple GPUs to jointly train a neural network through data parallelism. To elaborate on what data parallelism is, assume there are *n* GPUs, then you can split each data batch into *n* parts, and use a GPU on each of these parts to run the forward and backward passes on the seperate chunks of the data.\n",
"\n",
"First copy the data definitions with the following commands, and the transform functions from the tutorial [Training Neural Networks](6-train-nn.md)."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "e86dc0be",
"metadata": {},
"outputs": [],
"source": [
"# Import transforms as compose a series of transformations to the images\n",
"from mxnet.gluon.data.vision import transforms\n",
"\n",
"jitter_param = 0.05\n",
"\n",
"# mean and std for normalizing image value in range (0,1)\n",
"mean = [0.485, 0.456, 0.406]\n",
"std = [0.229, 0.224, 0.225]\n",
"\n",
"training_transformer = transforms.Compose([\n",
" transforms.Resize(size=224, keep_ratio=True),\n",
" transforms.CenterCrop(128),\n",
" transforms.RandomFlipLeftRight(),\n",
" transforms.RandomColorJitter(contrast=jitter_param),\n",
" transforms.ToTensor(),\n",
" transforms.Normalize(mean, std)\n",
"])\n",
"\n",
"validation_transformer = transforms.Compose([\n",
" transforms.Resize(size=224, keep_ratio=True),\n",
" transforms.CenterCrop(128),\n",
" transforms.ToTensor(),\n",
" transforms.Normalize(mean, std)\n",
"])\n",
"\n",
"# Use ImageFolderDataset to create a Dataset object from directory structure\n",
"train_dataset = gluon.data.vision.ImageFolderDataset('./datasets/train')\n",
"val_dataset = gluon.data.vision.ImageFolderDataset('./datasets/validation')\n",
"test_dataset = gluon.data.vision.ImageFolderDataset('./datasets/test')\n",
"\n",
"# Create data loaders\n",
"batch_size = 4\n",
"train_loader = gluon.data.DataLoader(train_dataset.transform_first(training_transformer),batch_size=batch_size, shuffle=True, try_nopython=True)\n",
"validation_loader = gluon.data.DataLoader(val_dataset.transform_first(validation_transformer), batch_size=batch_size, try_nopython=True)\n",
"test_loader = gluon.data.DataLoader(test_dataset.transform_first(validation_transformer), batch_size=batch_size, try_nopython=True)"
]
},
{
"cell_type": "markdown",
"id": "9a65c84a",
"metadata": {},
"source": [
"### Define a helper function\n",
"This is the same test function defined previously in the **Step 6**."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "04f802e9",
"metadata": {},
"outputs": [],
"source": [
"# Function to return the accuracy for the validation and test set\n",
"def test(val_data, devices):\n",
" acc = gluon.metric.Accuracy()\n",
" for batch in val_data:\n",
" data, label = batch[0], batch[1]\n",
" data_list = gluon.utils.split_and_load(data, devices)\n",
" label_list = gluon.utils.split_and_load(label, devices)\n",
" outputs = [net(X) for X in data_list]\n",
" acc.update(label_list, outputs)\n",
"\n",
" _, accuracy = acc.get()\n",
" return accuracy"
]
},
{
"cell_type": "markdown",
"id": "e11efd0a",
"metadata": {},
"source": [
"The training loop is quite similar to that shown earlier. The major differences are highlighted in the following code."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "660fa3d7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Using 1 GPUs\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[5] Speed: 0.7173089899203501 samples/sec batch loss = 14.109387874603271 | accuracy = 0.4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[10] Speed: 1.2488635911932617 samples/sec batch loss = 27.856971979141235 | accuracy = 0.475\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[15] Speed: 1.2547331219178495 samples/sec batch loss = 41.62733340263367 | accuracy = 0.5166666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[20] Speed: 1.2574016512018573 samples/sec batch loss = 55.84801149368286 | accuracy = 0.4875\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[25] Speed: 1.249793820569105 samples/sec batch loss = 69.66716718673706 | accuracy = 0.49\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[30] Speed: 1.2504747636094988 samples/sec batch loss = 83.63934445381165 | accuracy = 0.48333333333333334\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[35] Speed: 1.2540013241700803 samples/sec batch loss = 98.12923693656921 | accuracy = 0.4785714285714286\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[40] Speed: 1.2433366882315402 samples/sec batch loss = 111.83499693870544 | accuracy = 0.48125\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[45] Speed: 1.2445212875529668 samples/sec batch loss = 125.18878197669983 | accuracy = 0.5\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[50] Speed: 1.245335507712452 samples/sec batch loss = 139.3703784942627 | accuracy = 0.495\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[55] Speed: 1.2469224778275927 samples/sec batch loss = 153.2837598323822 | accuracy = 0.4954545454545455\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[60] Speed: 1.2478066605108897 samples/sec batch loss = 167.36257028579712 | accuracy = 0.4875\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[65] Speed: 1.2488965939290726 samples/sec batch loss = 181.38084363937378 | accuracy = 0.49230769230769234\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[70] Speed: 1.2480735339955618 samples/sec batch loss = 194.63530731201172 | accuracy = 0.5071428571428571\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[75] Speed: 1.2476920559844125 samples/sec batch loss = 208.3918468952179 | accuracy = 0.5166666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[80] Speed: 1.2477543202853443 samples/sec batch loss = 221.81531071662903 | accuracy = 0.51875\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[85] Speed: 1.2461767111831574 samples/sec batch loss = 235.0783576965332 | accuracy = 0.5323529411764706\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[90] Speed: 1.251087407430828 samples/sec batch loss = 248.5095088481903 | accuracy = 0.5333333333333333\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[95] Speed: 1.2485950788408764 samples/sec batch loss = 261.928329706192 | accuracy = 0.5447368421052632\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[100] Speed: 1.2461065520652592 samples/sec batch loss = 275.105082988739 | accuracy = 0.5475\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[105] Speed: 1.2466023705424898 samples/sec batch loss = 288.4014744758606 | accuracy = 0.5547619047619048\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[110] Speed: 1.2527445482382205 samples/sec batch loss = 302.4630672931671 | accuracy = 0.5545454545454546\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[115] Speed: 1.249511507393301 samples/sec batch loss = 315.87427735328674 | accuracy = 0.5543478260869565\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[120] Speed: 1.2469830897237266 samples/sec batch loss = 330.0675938129425 | accuracy = 0.55625\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[125] Speed: 1.2485631141384241 samples/sec batch loss = 344.2156593799591 | accuracy = 0.552\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[130] Speed: 1.2492975072453407 samples/sec batch loss = 358.20309352874756 | accuracy = 0.551923076923077\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[135] Speed: 1.2539363730217234 samples/sec batch loss = 371.67721605300903 | accuracy = 0.5555555555555556\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[140] Speed: 1.254016133598948 samples/sec batch loss = 385.8363575935364 | accuracy = 0.5571428571428572\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[145] Speed: 1.2494756805205474 samples/sec batch loss = 399.2117648124695 | accuracy = 0.5586206896551724\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[150] Speed: 1.2451534305105048 samples/sec batch loss = 412.84080719947815 | accuracy = 0.56\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[155] Speed: 1.2439571132825729 samples/sec batch loss = 426.0250413417816 | accuracy = 0.5629032258064516\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[160] Speed: 1.250896556279965 samples/sec batch loss = 440.5059177875519 | accuracy = 0.5546875\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[165] Speed: 1.2510462659691461 samples/sec batch loss = 454.378933429718 | accuracy = 0.55\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[170] Speed: 1.2525483285410557 samples/sec batch loss = 468.0374982357025 | accuracy = 0.55\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[175] Speed: 1.250540848040186 samples/sec batch loss = 481.37460684776306 | accuracy = 0.5514285714285714\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[180] Speed: 1.2443146226482964 samples/sec batch loss = 494.53990387916565 | accuracy = 0.5541666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[185] Speed: 1.254542657901756 samples/sec batch loss = 507.77793407440186 | accuracy = 0.5594594594594594\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[190] Speed: 1.2550383602020694 samples/sec batch loss = 521.1585078239441 | accuracy = 0.5592105263157895\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[195] Speed: 1.248124601062049 samples/sec batch loss = 533.4047560691833 | accuracy = 0.5666666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[200] Speed: 1.2500795213800187 samples/sec batch loss = 545.6380257606506 | accuracy = 0.57375\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[205] Speed: 1.2508536554361183 samples/sec batch loss = 558.0965132713318 | accuracy = 0.5780487804878048\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[210] Speed: 1.2517014923235137 samples/sec batch loss = 570.9625518321991 | accuracy = 0.5785714285714286\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[215] Speed: 1.2552960322845683 samples/sec batch loss = 583.5726790428162 | accuracy = 0.5837209302325581\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[220] Speed: 1.2471114692340513 samples/sec batch loss = 597.2438080310822 | accuracy = 0.5852272727272727\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[225] Speed: 1.2522912222957368 samples/sec batch loss = 610.7035535573959 | accuracy = 0.5855555555555556\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[230] Speed: 1.257931683124963 samples/sec batch loss = 623.757507443428 | accuracy = 0.5869565217391305\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[235] Speed: 1.2581769575322561 samples/sec batch loss = 636.8130487203598 | accuracy = 0.5851063829787234\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[240] Speed: 1.2578736802786734 samples/sec batch loss = 649.8954354524612 | accuracy = 0.5854166666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[245] Speed: 1.2543922040718885 samples/sec batch loss = 663.0327650308609 | accuracy = 0.5867346938775511\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[250] Speed: 1.248841094553405 samples/sec batch loss = 676.6164935827255 | accuracy = 0.588\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[255] Speed: 1.2531124621277645 samples/sec batch loss = 690.2022820711136 | accuracy = 0.5852941176470589\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[260] Speed: 1.2475061352092753 samples/sec batch loss = 702.0996137857437 | accuracy = 0.5884615384615385\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[265] Speed: 1.2421387996904651 samples/sec batch loss = 714.8796883821487 | accuracy = 0.5915094339622642\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[270] Speed: 1.244270787869061 samples/sec batch loss = 727.0544770956039 | accuracy = 0.5935185185185186\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[275] Speed: 1.2443309576611774 samples/sec batch loss = 740.4612075090408 | accuracy = 0.5927272727272728\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[280] Speed: 1.2476181078018296 samples/sec batch loss = 753.5070110559464 | accuracy = 0.5919642857142857\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[285] Speed: 1.2465933858212954 samples/sec batch loss = 766.1015297174454 | accuracy = 0.5929824561403508\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[290] Speed: 1.2449173639880178 samples/sec batch loss = 779.6410132646561 | accuracy = 0.593103448275862\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[295] Speed: 1.2430146432604277 samples/sec batch loss = 792.6876677274704 | accuracy = 0.5949152542372881\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[300] Speed: 1.2515361276164 samples/sec batch loss = 804.2858747243881 | accuracy = 0.5975\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[305] Speed: 1.2583303026141537 samples/sec batch loss = 816.6289802789688 | accuracy = 0.5991803278688524\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[310] Speed: 1.2502803723413254 samples/sec batch loss = 828.6005524396896 | accuracy = 0.6024193548387097\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[315] Speed: 1.2525589890425457 samples/sec batch loss = 839.8461601734161 | accuracy = 0.6031746031746031\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[320] Speed: 1.249410452970806 samples/sec batch loss = 852.4019663333893 | accuracy = 0.603125\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[325] Speed: 1.2496238402843836 samples/sec batch loss = 868.4544899463654 | accuracy = 0.6\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[330] Speed: 1.2523530116511297 samples/sec batch loss = 881.5271074771881 | accuracy = 0.6007575757575757\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[335] Speed: 1.2546176168868743 samples/sec batch loss = 893.2346479892731 | accuracy = 0.6037313432835821\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[340] Speed: 1.246955470728363 samples/sec batch loss = 906.9669165611267 | accuracy = 0.6044117647058823\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[345] Speed: 1.2506402208528993 samples/sec batch loss = 919.7020897865295 | accuracy = 0.6057971014492753\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[350] Speed: 1.2492334145122315 samples/sec batch loss = 933.1867297887802 | accuracy = 0.6057142857142858\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[355] Speed: 1.2538524994592892 samples/sec batch loss = 947.3878728151321 | accuracy = 0.6035211267605634\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[360] Speed: 1.2499582595160668 samples/sec batch loss = 960.3634434938431 | accuracy = 0.6041666666666666\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[365] Speed: 1.2516683411814322 samples/sec batch loss = 972.6316422224045 | accuracy = 0.6047945205479452\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[370] Speed: 1.2519167840129217 samples/sec batch loss = 984.2569028139114 | accuracy = 0.606081081081081\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[375] Speed: 1.2528885254228845 samples/sec batch loss = 998.0109266042709 | accuracy = 0.606\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[380] Speed: 1.2489419639171562 samples/sec batch loss = 1011.4422169923782 | accuracy = 0.6065789473684211\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[385] Speed: 1.2500232648709093 samples/sec batch loss = 1025.3609365224838 | accuracy = 0.6051948051948052\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[390] Speed: 1.251817768659265 samples/sec batch loss = 1038.6429842710495 | accuracy = 0.6051282051282051\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[395] Speed: 1.2551697184272432 samples/sec batch loss = 1051.1454488039017 | accuracy = 0.6044303797468354\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[400] Speed: 1.2505904392522167 samples/sec batch loss = 1063.5903500318527 | accuracy = 0.604375\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[405] Speed: 1.2501570220818985 samples/sec batch loss = 1077.7765802145004 | accuracy = 0.6030864197530864\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[410] Speed: 1.256009407709015 samples/sec batch loss = 1090.1866561174393 | accuracy = 0.6036585365853658\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[415] Speed: 1.2470800439360186 samples/sec batch loss = 1102.3599358797073 | accuracy = 0.6042168674698796\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[420] Speed: 1.2568073817848802 samples/sec batch loss = 1114.840255856514 | accuracy = 0.6053571428571428\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[425] Speed: 1.247749958798082 samples/sec batch loss = 1130.673286318779 | accuracy = 0.6023529411764705\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[430] Speed: 1.250030436362818 samples/sec batch loss = 1143.257992386818 | accuracy = 0.6034883720930233\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[435] Speed: 1.2464936363295949 samples/sec batch loss = 1153.495651960373 | accuracy = 0.6063218390804598\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[440] Speed: 1.2447175858443076 samples/sec batch loss = 1165.5605726242065 | accuracy = 0.6073863636363637\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[445] Speed: 1.2548845022973882 samples/sec batch loss = 1177.6937124729156 | accuracy = 0.6067415730337079\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[450] Speed: 1.253411855657703 samples/sec batch loss = 1192.1498646736145 | accuracy = 0.6066666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[455] Speed: 1.2513164870329272 samples/sec batch loss = 1205.8402743339539 | accuracy = 0.6065934065934065\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[460] Speed: 1.247389916487588 samples/sec batch loss = 1217.7081812620163 | accuracy = 0.6081521739130434\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[465] Speed: 1.2509053233362604 samples/sec batch loss = 1230.8523312807083 | accuracy = 0.6069892473118279\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[470] Speed: 1.2474988071322672 samples/sec batch loss = 1243.3499066829681 | accuracy = 0.6069148936170212\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[475] Speed: 1.249558690331585 samples/sec batch loss = 1256.9940313100815 | accuracy = 0.6068421052631578\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[480] Speed: 1.2498210999148223 samples/sec batch loss = 1269.0932389497757 | accuracy = 0.6078125\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[485] Speed: 1.249798754970041 samples/sec batch loss = 1282.1142412424088 | accuracy = 0.6087628865979381\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[490] Speed: 1.2474131955984435 samples/sec batch loss = 1296.9114240407944 | accuracy = 0.6071428571428571\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[495] Speed: 1.2518477518440956 samples/sec batch loss = 1309.8559930324554 | accuracy = 0.6080808080808081\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[500] Speed: 1.2459789345602494 samples/sec batch loss = 1323.4580008983612 | accuracy = 0.6085\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[505] Speed: 1.255630017740401 samples/sec batch loss = 1336.2770595550537 | accuracy = 0.6084158415841584\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[510] Speed: 1.255058170153684 samples/sec batch loss = 1351.4411013126373 | accuracy = 0.6058823529411764\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[515] Speed: 1.2495108559772763 samples/sec batch loss = 1363.1846477985382 | accuracy = 0.6063106796116505\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[520] Speed: 1.2527150833033232 samples/sec batch loss = 1375.6460919380188 | accuracy = 0.6076923076923076\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[525] Speed: 1.25798063603048 samples/sec batch loss = 1386.192439198494 | accuracy = 0.61\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[530] Speed: 1.248925228701789 samples/sec batch loss = 1397.0663534402847 | accuracy = 0.6113207547169811\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[535] Speed: 1.259530798723834 samples/sec batch loss = 1409.4639719724655 | accuracy = 0.611214953271028\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[540] Speed: 1.256628993998189 samples/sec batch loss = 1423.510990023613 | accuracy = 0.6101851851851852\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[545] Speed: 1.2539642084042697 samples/sec batch loss = 1437.0432263612747 | accuracy = 0.6096330275229358\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[550] Speed: 1.2498961475354633 samples/sec batch loss = 1450.1948345899582 | accuracy = 0.6086363636363636\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[555] Speed: 1.2492934140440306 samples/sec batch loss = 1463.197319149971 | accuracy = 0.6072072072072072\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[560] Speed: 1.2499930896246991 samples/sec batch loss = 1476.8904753923416 | accuracy = 0.6066964285714286\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[565] Speed: 1.2524270547336498 samples/sec batch loss = 1489.2849966287613 | accuracy = 0.6079646017699115\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[570] Speed: 1.2518859567991858 samples/sec batch loss = 1503.6431480646133 | accuracy = 0.6065789473684211\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[575] Speed: 1.2527742951975238 samples/sec batch loss = 1515.261520266533 | accuracy = 0.6078260869565217\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[580] Speed: 1.2509416053332658 samples/sec batch loss = 1526.9548864364624 | accuracy = 0.6090517241379311\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[585] Speed: 1.2541969680625324 samples/sec batch loss = 1537.8622829914093 | accuracy = 0.6106837606837607\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[590] Speed: 1.2524829668018274 samples/sec batch loss = 1548.5477073192596 | accuracy = 0.6122881355932204\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[595] Speed: 1.2558429968697522 samples/sec batch loss = 1561.7891867160797 | accuracy = 0.6109243697478992\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[600] Speed: 1.2572451406118823 samples/sec batch loss = 1575.6118171215057 | accuracy = 0.61125\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[605] Speed: 1.2513090207948667 samples/sec batch loss = 1588.3178189992905 | accuracy = 0.6119834710743801\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[610] Speed: 1.2537173882242802 samples/sec batch loss = 1599.6253324747086 | accuracy = 0.6131147540983607\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[615] Speed: 1.2523310434813115 samples/sec batch loss = 1611.0392497777939 | accuracy = 0.6142276422764228\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[620] Speed: 1.253102728154078 samples/sec batch loss = 1625.3391102552414 | accuracy = 0.6137096774193549\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[625] Speed: 1.2540733125243213 samples/sec batch loss = 1636.2634762525558 | accuracy = 0.616\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[630] Speed: 1.2508428374270442 samples/sec batch loss = 1648.5310891866684 | accuracy = 0.6158730158730159\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[635] Speed: 1.253749523694156 samples/sec batch loss = 1659.243582367897 | accuracy = 0.6173228346456693\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[640] Speed: 1.2542251900675794 samples/sec batch loss = 1672.3194509744644 | accuracy = 0.6171875\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[645] Speed: 1.2545985713897927 samples/sec batch loss = 1683.6487039327621 | accuracy = 0.6186046511627907\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[650] Speed: 1.2556925129521328 samples/sec batch loss = 1696.872985959053 | accuracy = 0.6180769230769231\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[655] Speed: 1.2500393775588894 samples/sec batch loss = 1710.985644698143 | accuracy = 0.616793893129771\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[660] Speed: 1.2501229280412516 samples/sec batch loss = 1724.8238681554794 | accuracy = 0.6162878787878788\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[665] Speed: 1.249774735045506 samples/sec batch loss = 1739.9732121229172 | accuracy = 0.6150375939849624\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[670] Speed: 1.250887136494731 samples/sec batch loss = 1753.7249166965485 | accuracy = 0.6149253731343284\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[675] Speed: 1.2546571170269594 samples/sec batch loss = 1765.994879245758 | accuracy = 0.6159259259259259\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[680] Speed: 1.2519335994966958 samples/sec batch loss = 1776.8267105817795 | accuracy = 0.6165441176470589\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[685] Speed: 1.2471131378782268 samples/sec batch loss = 1789.9577577114105 | accuracy = 0.6167883211678832\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[690] Speed: 1.2509222049290933 samples/sec batch loss = 1799.4462422132492 | accuracy = 0.6192028985507246\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[695] Speed: 1.2527358489227634 samples/sec batch loss = 1810.1979557275772 | accuracy = 0.6201438848920864\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[700] Speed: 1.2511445994904804 samples/sec batch loss = 1823.6723712682724 | accuracy = 0.6196428571428572\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[705] Speed: 1.2504711287024335 samples/sec batch loss = 1836.2821657657623 | accuracy = 0.6205673758865248\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[710] Speed: 1.2511894797924352 samples/sec batch loss = 1848.7483506202698 | accuracy = 0.6214788732394366\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[715] Speed: 1.2487104999072243 samples/sec batch loss = 1862.0921961069107 | accuracy = 0.6202797202797202\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[720] Speed: 1.251740715660875 samples/sec batch loss = 1873.925968170166 | accuracy = 0.6204861111111111\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[725] Speed: 1.254980248359765 samples/sec batch loss = 1887.4272947311401 | accuracy = 0.6206896551724138\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[730] Speed: 1.2538383498303711 samples/sec batch loss = 1897.7921460866928 | accuracy = 0.6222602739726028\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[735] Speed: 1.2528853442828891 samples/sec batch loss = 1911.5076615810394 | accuracy = 0.6221088435374149\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[740] Speed: 1.2564302383524641 samples/sec batch loss = 1924.5269718170166 | accuracy = 0.6222972972972973\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[745] Speed: 1.2524705310773974 samples/sec batch loss = 1935.5363252162933 | accuracy = 0.6228187919463087\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[750] Speed: 1.2544477289644191 samples/sec batch loss = 1947.629676580429 | accuracy = 0.6233333333333333\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[755] Speed: 1.2525294393261557 samples/sec batch loss = 1960.0555703639984 | accuracy = 0.6235099337748344\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[760] Speed: 1.2522337384312432 samples/sec batch loss = 1971.902447938919 | accuracy = 0.6236842105263158\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[765] Speed: 1.2590799191383462 samples/sec batch loss = 1981.1101183891296 | accuracy = 0.6258169934640523\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[770] Speed: 1.2517838641712986 samples/sec batch loss = 1993.2993923425674 | accuracy = 0.6266233766233766\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[775] Speed: 1.249143379517365 samples/sec batch loss = 2005.0542666912079 | accuracy = 0.6274193548387097\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[780] Speed: 1.2507209612379153 samples/sec batch loss = 2017.8336730003357 | accuracy = 0.6278846153846154\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[1] Batch[785] Speed: 1.2525516014827356 samples/sec batch loss = 2031.7250916957855 | accuracy = 0.6270700636942675\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[Epoch 1] training: accuracy=0.6272208121827412\n",
"[Epoch 1] time cost: 650.7169613838196\n",
"[Epoch 1] validation: validation accuracy=0.7366666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[5] Speed: 1.2340196984579255 samples/sec batch loss = 13.191937804222107 | accuracy = 0.6\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[10] Speed: 1.2310370853126993 samples/sec batch loss = 26.464403986930847 | accuracy = 0.6\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[15] Speed: 1.2326619696754513 samples/sec batch loss = 41.108960032463074 | accuracy = 0.6\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[20] Speed: 1.2320190106082836 samples/sec batch loss = 55.04953134059906 | accuracy = 0.5875\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[25] Speed: 1.235256537546213 samples/sec batch loss = 65.04635787010193 | accuracy = 0.66\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[30] Speed: 1.2322364535608838 samples/sec batch loss = 75.13800632953644 | accuracy = 0.7\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[35] Speed: 1.233094214476106 samples/sec batch loss = 85.9286288022995 | accuracy = 0.7071428571428572\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[40] Speed: 1.2300262424703376 samples/sec batch loss = 96.62003815174103 | accuracy = 0.70625\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[45] Speed: 1.2289717740089405 samples/sec batch loss = 106.36261355876923 | accuracy = 0.7222222222222222\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[50] Speed: 1.2303652312370434 samples/sec batch loss = 117.58973848819733 | accuracy = 0.725\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[55] Speed: 1.2362896637213086 samples/sec batch loss = 131.9102178812027 | accuracy = 0.7136363636363636\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[60] Speed: 1.2333847524966035 samples/sec batch loss = 143.2937877178192 | accuracy = 0.7125\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[65] Speed: 1.236500870669069 samples/sec batch loss = 153.73799800872803 | accuracy = 0.7192307692307692\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[70] Speed: 1.23287393197801 samples/sec batch loss = 165.10248506069183 | accuracy = 0.7142857142857143\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[75] Speed: 1.231276952872063 samples/sec batch loss = 179.1421970129013 | accuracy = 0.71\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[80] Speed: 1.230124636409622 samples/sec batch loss = 189.26601195335388 | accuracy = 0.709375\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[85] Speed: 1.2319504365338176 samples/sec batch loss = 198.50052571296692 | accuracy = 0.7147058823529412\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[90] Speed: 1.2315103148763311 samples/sec batch loss = 210.53342199325562 | accuracy = 0.7111111111111111\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[95] Speed: 1.235229981223303 samples/sec batch loss = 222.21577048301697 | accuracy = 0.7105263157894737\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[100] Speed: 1.232857986979065 samples/sec batch loss = 235.56106781959534 | accuracy = 0.705\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[105] Speed: 1.228153899896607 samples/sec batch loss = 246.29658603668213 | accuracy = 0.7047619047619048\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[110] Speed: 1.2345419159369972 samples/sec batch loss = 256.8897407054901 | accuracy = 0.7045454545454546\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[115] Speed: 1.230914703045376 samples/sec batch loss = 268.92195451259613 | accuracy = 0.7065217391304348\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[120] Speed: 1.2312387304247892 samples/sec batch loss = 281.6716022491455 | accuracy = 0.70625\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[125] Speed: 1.230405474878182 samples/sec batch loss = 290.8304713964462 | accuracy = 0.71\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[130] Speed: 1.2320699485360205 samples/sec batch loss = 301.62862718105316 | accuracy = 0.7115384615384616\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[135] Speed: 1.2331189569992693 samples/sec batch loss = 312.6072155237198 | accuracy = 0.7074074074074074\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[140] Speed: 1.230335817146746 samples/sec batch loss = 323.0715608596802 | accuracy = 0.7107142857142857\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[145] Speed: 1.2294742297399517 samples/sec batch loss = 333.3850485086441 | accuracy = 0.7120689655172414\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[150] Speed: 1.228943416736621 samples/sec batch loss = 344.21285820007324 | accuracy = 0.7133333333333334\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[155] Speed: 1.230386615983145 samples/sec batch loss = 351.91108351945877 | accuracy = 0.7161290322580646\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[160] Speed: 1.2290947606893128 samples/sec batch loss = 363.82177871465683 | accuracy = 0.7140625\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[165] Speed: 1.2334957460494567 samples/sec batch loss = 375.11759382486343 | accuracy = 0.7151515151515152\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[170] Speed: 1.2350278450031669 samples/sec batch loss = 383.88711553812027 | accuracy = 0.7176470588235294\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[175] Speed: 1.2359369322991975 samples/sec batch loss = 397.7323417067528 | accuracy = 0.7157142857142857\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[180] Speed: 1.2326006591320082 samples/sec batch loss = 408.222943007946 | accuracy = 0.7180555555555556\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[185] Speed: 1.2286690039663009 samples/sec batch loss = 419.9541248679161 | accuracy = 0.7162162162162162\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[190] Speed: 1.2348820352113132 samples/sec batch loss = 436.4125308394432 | accuracy = 0.7118421052631579\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[195] Speed: 1.229741250438049 samples/sec batch loss = 445.3180300593376 | accuracy = 0.7153846153846154\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[200] Speed: 1.2276386885677262 samples/sec batch loss = 458.6773495078087 | accuracy = 0.71375\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[205] Speed: 1.2322305708230772 samples/sec batch loss = 474.16221684217453 | accuracy = 0.7060975609756097\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[210] Speed: 1.236188641564287 samples/sec batch loss = 486.45931726694107 | accuracy = 0.705952380952381\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[215] Speed: 1.2269664998651428 samples/sec batch loss = 497.142437517643 | accuracy = 0.7069767441860465\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[220] Speed: 1.227698877596039 samples/sec batch loss = 511.4827534556389 | accuracy = 0.7034090909090909\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[225] Speed: 1.2295424383087572 samples/sec batch loss = 521.8850936293602 | accuracy = 0.7044444444444444\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[230] Speed: 1.2296271464053932 samples/sec batch loss = 534.2236570715904 | accuracy = 0.7043478260869566\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[235] Speed: 1.228783470222361 samples/sec batch loss = 547.6792865395546 | accuracy = 0.7031914893617022\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[240] Speed: 1.226652788615943 samples/sec batch loss = 560.1972559094429 | accuracy = 0.7020833333333333\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[245] Speed: 1.2333460364060655 samples/sec batch loss = 570.8941144943237 | accuracy = 0.7010204081632653\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[250] Speed: 1.2323818202773085 samples/sec batch loss = 581.0161567926407 | accuracy = 0.702\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[255] Speed: 1.2315246882396889 samples/sec batch loss = 590.5841120481491 | accuracy = 0.703921568627451\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[260] Speed: 1.2313567487492096 samples/sec batch loss = 603.2001624107361 | accuracy = 0.7038461538461539\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[265] Speed: 1.2319485368330394 samples/sec batch loss = 614.1992572546005 | accuracy = 0.7037735849056603\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[270] Speed: 1.228155608101228 samples/sec batch loss = 626.8009022474289 | accuracy = 0.7027777777777777\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[275] Speed: 1.2277637446187692 samples/sec batch loss = 637.8115049004555 | accuracy = 0.7027272727272728\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[280] Speed: 1.2313637980328085 samples/sec batch loss = 653.7936922907829 | accuracy = 0.7\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[285] Speed: 1.2298324767914899 samples/sec batch loss = 665.3686112761497 | accuracy = 0.7008771929824561\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[290] Speed: 1.2271632231151068 samples/sec batch loss = 674.8924590945244 | accuracy = 0.7043103448275863\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[295] Speed: 1.2271321668035655 samples/sec batch loss = 685.4287278056145 | accuracy = 0.7025423728813559\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[300] Speed: 1.229344050771688 samples/sec batch loss = 696.8948901295662 | accuracy = 0.7033333333333334\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[305] Speed: 1.2305269436237125 samples/sec batch loss = 709.539255797863 | accuracy = 0.7032786885245902\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[310] Speed: 1.2277334664989021 samples/sec batch loss = 720.9653010964394 | accuracy = 0.7024193548387097\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[315] Speed: 1.2320834301489585 samples/sec batch loss = 735.4295864701271 | accuracy = 0.7\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[320] Speed: 1.2337472783931034 samples/sec batch loss = 747.0269188284874 | accuracy = 0.69921875\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[325] Speed: 1.2303642387141367 samples/sec batch loss = 758.9655331969261 | accuracy = 0.6976923076923077\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[330] Speed: 1.2318759907398658 samples/sec batch loss = 770.8926784396172 | accuracy = 0.696969696969697\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[335] Speed: 1.2302050046415316 samples/sec batch loss = 781.7621143460274 | accuracy = 0.6977611940298507\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[340] Speed: 1.2321256865118086 samples/sec batch loss = 793.5896945595741 | accuracy = 0.6970588235294117\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[345] Speed: 1.232044162380111 samples/sec batch loss = 804.6628307700157 | accuracy = 0.696376811594203\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[350] Speed: 1.2317640223959818 samples/sec batch loss = 817.6521719098091 | accuracy = 0.6957142857142857\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[355] Speed: 1.2342185993178596 samples/sec batch loss = 828.8623519539833 | accuracy = 0.6957746478873239\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[360] Speed: 1.2368537417348167 samples/sec batch loss = 838.1571943163872 | accuracy = 0.6979166666666666\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[365] Speed: 1.229889545215764 samples/sec batch loss = 850.3256699442863 | accuracy = 0.6972602739726027\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[370] Speed: 1.22365342601773 samples/sec batch loss = 862.8044571280479 | accuracy = 0.6959459459459459\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[375] Speed: 1.2250326060241592 samples/sec batch loss = 874.7381972670555 | accuracy = 0.696\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[380] Speed: 1.2292805478547912 samples/sec batch loss = 885.6196042895317 | accuracy = 0.6953947368421053\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[385] Speed: 1.2262928060364084 samples/sec batch loss = 897.9612233042717 | accuracy = 0.6948051948051948\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[390] Speed: 1.2257007963847661 samples/sec batch loss = 910.6947954297066 | accuracy = 0.6935897435897436\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[395] Speed: 1.2271841376441512 samples/sec batch loss = 920.7800042033195 | accuracy = 0.6955696202531646\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[400] Speed: 1.228623565336638 samples/sec batch loss = 932.4678912758827 | accuracy = 0.694375\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[405] Speed: 1.228689159998872 samples/sec batch loss = 944.510540664196 | accuracy = 0.6938271604938272\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[410] Speed: 1.2343303786888913 samples/sec batch loss = 956.4577099680901 | accuracy = 0.6932926829268292\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[415] Speed: 1.2211435847516017 samples/sec batch loss = 968.2406663298607 | accuracy = 0.6933734939759036\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[420] Speed: 1.223634684290708 samples/sec batch loss = 979.0851874947548 | accuracy = 0.694047619047619\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[425] Speed: 1.2231536646748846 samples/sec batch loss = 990.9242896437645 | accuracy = 0.6935294117647058\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[430] Speed: 1.222433289278115 samples/sec batch loss = 1001.2824978232384 | accuracy = 0.6947674418604651\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[435] Speed: 1.2288869761558563 samples/sec batch loss = 1013.8018770813942 | accuracy = 0.6948275862068966\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[440] Speed: 1.2258045001065998 samples/sec batch loss = 1025.6744262576103 | accuracy = 0.6943181818181818\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[445] Speed: 1.2294223350267197 samples/sec batch loss = 1039.92163926363 | accuracy = 0.6932584269662921\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[450] Speed: 1.2305351567149998 samples/sec batch loss = 1052.8086115717888 | accuracy = 0.6933333333333334\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[455] Speed: 1.226263227823574 samples/sec batch loss = 1063.562524497509 | accuracy = 0.6928571428571428\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[460] Speed: 1.2290582942821577 samples/sec batch loss = 1074.8319408297539 | accuracy = 0.6929347826086957\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[465] Speed: 1.2308020064374732 samples/sec batch loss = 1086.642512023449 | accuracy = 0.6924731182795699\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[470] Speed: 1.2269165213879796 samples/sec batch loss = 1096.9285895228386 | accuracy = 0.6925531914893617\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[475] Speed: 1.2326932159469668 samples/sec batch loss = 1112.122159421444 | accuracy = 0.6915789473684211\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[480] Speed: 1.2319340631147153 samples/sec batch loss = 1125.8898395895958 | accuracy = 0.6916666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[485] Speed: 1.227795192318318 samples/sec batch loss = 1135.0219516158104 | accuracy = 0.6938144329896907\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[490] Speed: 1.2330217146314293 samples/sec batch loss = 1143.4660302996635 | accuracy = 0.6948979591836735\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[495] Speed: 1.2283184487878263 samples/sec batch loss = 1153.8384173512459 | accuracy = 0.6944444444444444\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[500] Speed: 1.231757058962938 samples/sec batch loss = 1166.0878913998604 | accuracy = 0.6945\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[505] Speed: 1.2287922900483754 samples/sec batch loss = 1178.929158270359 | accuracy = 0.694059405940594\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[510] Speed: 1.228906599246756 samples/sec batch loss = 1190.3485252261162 | accuracy = 0.6936274509803921\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[515] Speed: 1.2278663599174895 samples/sec batch loss = 1202.1567012667656 | accuracy = 0.6941747572815534\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[520] Speed: 1.2290040038628482 samples/sec batch loss = 1212.99599558115 | accuracy = 0.6942307692307692\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[525] Speed: 1.2229375431525127 samples/sec batch loss = 1221.1800859570503 | accuracy = 0.6957142857142857\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[530] Speed: 1.2294278306135973 samples/sec batch loss = 1234.8584129214287 | accuracy = 0.6943396226415094\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[535] Speed: 1.2295529811274073 samples/sec batch loss = 1245.7880756258965 | accuracy = 0.6939252336448598\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[540] Speed: 1.2256907672490176 samples/sec batch loss = 1255.1600200533867 | accuracy = 0.6949074074074074\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[545] Speed: 1.2301580091479405 samples/sec batch loss = 1262.4292892813683 | accuracy = 0.6972477064220184\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[550] Speed: 1.2303730812473594 samples/sec batch loss = 1272.4460882544518 | accuracy = 0.6981818181818182\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[555] Speed: 1.2239685514824734 samples/sec batch loss = 1285.0484514832497 | accuracy = 0.6981981981981982\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[560] Speed: 1.2348070529339519 samples/sec batch loss = 1294.4679728150368 | accuracy = 0.7\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[565] Speed: 1.2309475767769544 samples/sec batch loss = 1303.1254943013191 | accuracy = 0.7013274336283186\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[570] Speed: 1.230075482565809 samples/sec batch loss = 1316.4713049530983 | accuracy = 0.7017543859649122\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[575] Speed: 1.227909316072026 samples/sec batch loss = 1324.9961784482002 | accuracy = 0.7030434782608695\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[580] Speed: 1.2274785426314483 samples/sec batch loss = 1336.1549338698387 | accuracy = 0.7030172413793103\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[585] Speed: 1.226800159993097 samples/sec batch loss = 1348.2013884186745 | accuracy = 0.7029914529914529\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[590] Speed: 1.2298689891540981 samples/sec batch loss = 1358.4156247973442 | accuracy = 0.7038135593220339\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[595] Speed: 1.2273475288875058 samples/sec batch loss = 1370.7123524546623 | accuracy = 0.7033613445378152\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[600] Speed: 1.2289090296731893 samples/sec batch loss = 1381.9451199173927 | accuracy = 0.7041666666666667\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[605] Speed: 1.23099020686782 samples/sec batch loss = 1394.8828805088997 | accuracy = 0.702892561983471\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[610] Speed: 1.22789475737333 samples/sec batch loss = 1405.62037473917 | accuracy = 0.7024590163934427\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[615] Speed: 1.2283783447381098 samples/sec batch loss = 1414.0183553099632 | accuracy = 0.7028455284552846\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[620] Speed: 1.2275000965774134 samples/sec batch loss = 1424.5221813321114 | accuracy = 0.7032258064516129\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[625] Speed: 1.2284641518170887 samples/sec batch loss = 1438.8212504982948 | accuracy = 0.7028\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[630] Speed: 1.2223265039260094 samples/sec batch loss = 1452.665615260601 | accuracy = 0.7027777777777777\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[635] Speed: 1.2281521017916135 samples/sec batch loss = 1463.2709195017815 | accuracy = 0.7031496062992126\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[640] Speed: 1.2262747900459958 samples/sec batch loss = 1474.8328509926796 | accuracy = 0.703515625\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[645] Speed: 1.2305037490018833 samples/sec batch loss = 1482.5473827123642 | accuracy = 0.7046511627906977\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[650] Speed: 1.2268855672815617 samples/sec batch loss = 1496.133915424347 | accuracy = 0.7038461538461539\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[655] Speed: 1.228560586655208 samples/sec batch loss = 1508.042652606964 | accuracy = 0.7034351145038168\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[660] Speed: 1.2305761335556562 samples/sec batch loss = 1518.958142399788 | accuracy = 0.703030303030303\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[665] Speed: 1.23144324361659 samples/sec batch loss = 1528.3985501527786 | accuracy = 0.7041353383458646\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[670] Speed: 1.2258400571491157 samples/sec batch loss = 1539.1386450529099 | accuracy = 0.7044776119402985\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[675] Speed: 1.2295986687642204 samples/sec batch loss = 1549.0479488372803 | accuracy = 0.7051851851851851\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[680] Speed: 1.2276081471389826 samples/sec batch loss = 1557.7756605148315 | accuracy = 0.7066176470588236\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[685] Speed: 1.2255831434630442 samples/sec batch loss = 1568.49676913023 | accuracy = 0.7065693430656934\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[690] Speed: 1.2269875871585485 samples/sec batch loss = 1576.7246295809746 | accuracy = 0.7076086956521739\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[695] Speed: 1.2281175791967158 samples/sec batch loss = 1586.2875626683235 | accuracy = 0.7079136690647482\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[700] Speed: 1.2299273232787562 samples/sec batch loss = 1603.1909262537956 | accuracy = 0.7075\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[705] Speed: 1.2270075983242643 samples/sec batch loss = 1615.0498873591423 | accuracy = 0.7074468085106383\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[710] Speed: 1.2273998771518544 samples/sec batch loss = 1626.8721486926079 | accuracy = 0.7073943661971831\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[715] Speed: 1.2253525591257681 samples/sec batch loss = 1638.3502686619759 | accuracy = 0.7076923076923077\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[720] Speed: 1.2282458799898357 samples/sec batch loss = 1647.9793664813042 | accuracy = 0.7086805555555555\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[725] Speed: 1.2295865030854698 samples/sec batch loss = 1657.6263877749443 | accuracy = 0.7089655172413794\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[730] Speed: 1.2310131488591025 samples/sec batch loss = 1668.8062285780907 | accuracy = 0.7095890410958904\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[735] Speed: 1.2307631814565543 samples/sec batch loss = 1676.3949342370033 | accuracy = 0.7105442176870749\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[740] Speed: 1.2271674418682899 samples/sec batch loss = 1688.0925043225288 | accuracy = 0.7097972972972973\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[745] Speed: 1.223377445182976 samples/sec batch loss = 1696.91601729393 | accuracy = 0.710738255033557\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[750] Speed: 1.225483147285371 samples/sec batch loss = 1709.0300940275192 | accuracy = 0.7103333333333334\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[755] Speed: 1.2295042333696065 samples/sec batch loss = 1716.6224944591522 | accuracy = 0.7112582781456953\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[760] Speed: 1.2277638344670445 samples/sec batch loss = 1727.1613991260529 | accuracy = 0.7115131578947368\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[765] Speed: 1.2299555456373832 samples/sec batch loss = 1739.5258980989456 | accuracy = 0.711437908496732\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[770] Speed: 1.2265002521395307 samples/sec batch loss = 1748.1015099287033 | accuracy = 0.7123376623376624\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[775] Speed: 1.225936886655823 samples/sec batch loss = 1758.5067769289017 | accuracy = 0.7129032258064516\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[780] Speed: 1.2259969985127022 samples/sec batch loss = 1768.2216209173203 | accuracy = 0.7141025641025641\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch[2] Batch[785] Speed: 1.2287310938641942 samples/sec batch loss = 1779.2798628807068 | accuracy = 0.7136942675159236\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[Epoch 2] training: accuracy=0.7131979695431472\n",
"[Epoch 2] time cost: 657.6945281028748\n",
"[Epoch 2] validation: validation accuracy=0.7822222222222223\n"
]
}
],
"source": [
"# Diff 1: Use two GPUs for training.\n",
"available_gpus = [npx.gpu(i) for i in range(npx.num_gpus())]\n",
"num_gpus = 2\n",
"devices = available_gpus[:num_gpus]\n",
"print('Using {} GPUs'.format(len(devices)))\n",
"\n",
"# Diff 2: reinitialize the parameters and place them on multiple GPUs\n",
"net.initialize(force_reinit=True, device=devices)\n",
"\n",
"# Loss and trainer are the same as before\n",
"loss_fn = gluon.loss.SoftmaxCrossEntropyLoss()\n",
"optimizer = 'sgd'\n",
"optimizer_params = {'learning_rate': 0.001}\n",
"trainer = gluon.Trainer(net.collect_params(), optimizer, optimizer_params)\n",
"\n",
"epochs = 2\n",
"accuracy = gluon.metric.Accuracy()\n",
"log_interval = 5\n",
"\n",
"for epoch in range(epochs):\n",
" train_loss = 0.\n",
" tic = time.time()\n",
" btic = time.time()\n",
" accuracy.reset()\n",
" for idx, batch in enumerate(train_loader):\n",
" data, label = batch[0], batch[1]\n",
"\n",
" # Diff 3: split batch and load into corresponding devices\n",
" data_list = gluon.utils.split_and_load(data, devices)\n",
" label_list = gluon.utils.split_and_load(label, devices)\n",
"\n",
" # Diff 4: run forward and backward on each devices.\n",
" # MXNet will automatically run them in parallel\n",
" with autograd.record():\n",
" outputs = [net(X)\n",
" for X in data_list]\n",
" losses = [loss_fn(output, label)\n",
" for output, label in zip(outputs, label_list)]\n",
" for l in losses:\n",
" l.backward()\n",
" trainer.step(batch_size)\n",
"\n",
" # Diff 5: sum losses over all devices. Here, the float\n",
" # function will copy data into CPU.\n",
" train_loss += sum([float(l.sum()) for l in losses])\n",
" accuracy.update(label_list, outputs)\n",
" if log_interval and (idx + 1) % log_interval == 0:\n",
" _, acc = accuracy.get()\n",
"\n",
" print(f\"\"\"Epoch[{epoch + 1}] Batch[{idx + 1}] Speed: {batch_size / (time.time() - btic)} samples/sec \\\n",
" batch loss = {train_loss} | accuracy = {acc}\"\"\")\n",
" btic = time.time()\n",
"\n",
" _, acc = accuracy.get()\n",
"\n",
" acc_val = test(validation_loader, devices)\n",
" print(f\"[Epoch {epoch + 1}] training: accuracy={acc}\")\n",
" print(f\"[Epoch {epoch + 1}] time cost: {time.time() - tic}\")\n",
" print(f\"[Epoch {epoch + 1}] validation: validation accuracy={acc_val}\")"
]
},
{
"cell_type": "markdown",
"id": "0ef2f71a",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"Now that you have completed training and predicting with a neural network on GPUs, you reached the conclusion of the crash course. Congratulations.\n",
"If you are keen on studying more, checkout [D2L.ai](https://d2l.ai),\n",
"[GluonCV](https://cv.gluon.ai/tutorials/index.html), [GluonNLP](https://nlp.gluon.ai),\n",
"[GluonTS](https://ts.gluon.ai/), [AutoGluon](https://auto.gluon.ai)."
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}