| { |
| "cells": [ |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "# Define model architecture\n", |
| "This function loads model architectures and weights into a table for use by deep learning algorithms.\n", |
| "\n", |
| "Model architecture is in JSON form and model weights are in the form of PostgreSQL binary data types (bytea). If the output table already exists, a new row is inserted into the table so it can act as a repository for multiple model architectures and weights.\n", |
| "\n", |
| "There is also a function to delete a model from the table.\n", |
| "\n", |
| "MADlib's deep learning methods are designed to use the TensorFlow package and its built in Keras functions. To ensure consistency, please use tensorflow.keras objects (models, layers, etc.) instead of importing Keras and using its objects.\n", |
| "\n", |
| "The model architecture loader was added in MADlib 1.16 and updated after that.\n", |
| "\n", |
| "## Table of contents\n", |
| "\n", |
| "<a href=\"#define_model_arch\">1. Define model architecture</a>\n", |
| "\n", |
| "<a href=\"#load_model_arch\">2. Load model architecture</a>\n", |
| "\n", |
| "<a href=\"#load_model_wts\">3. Load model weights</a>\n", |
| "* <a href=\"#load_model_wts_madlib\">3a. Load weights from previous MADlib run</a>\n", |
| "* <a href=\"#load_model_wts_keras1\">3b. Load weights from Keras using a PL/Python function</a>\n", |
| "* <a href=\"#load_model_wts_keras2\">3c. Load weights from Keras using psycopg2</a>\n", |
| "\n", |
| "<a href=\"#delete_model\">4. Delete model</a>" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 34, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "The sql extension is already loaded. To reload it, use:\n", |
| " %reload_ext sql\n" |
| ] |
| } |
| ], |
| "source": [ |
| "%load_ext sql" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 35, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "# Greenplum Database 5.x on GCP - via tunnel\n", |
| "%sql postgresql://gpadmin@localhost:8000/madlib\n", |
| " \n", |
| "# PostgreSQL local\n", |
| "#%sql postgresql://fmcquillan@localhost:5432/madlib" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 36, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "1 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>version</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>MADlib version: 1.18.0-dev, git revision: rel/v1.17.0-89-g9d9f756, cmake configuration time: Thu Mar 4 23:11:53 UTC 2021, build type: release, build system: Linux-3.10.0-1160.11.1.el7.x86_64, C compiler: gcc 4.8.5, C++ compiler: g++ 4.8.5</td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[(u'MADlib version: 1.18.0-dev, git revision: rel/v1.17.0-89-g9d9f756, cmake configuration time: Thu Mar 4 23:11:53 UTC 2021, build type: release, build system: Linux-3.10.0-1160.11.1.el7.x86_64, C compiler: gcc 4.8.5, C++ compiler: g++ 4.8.5',)]" |
| ] |
| }, |
| "execution_count": 36, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%sql select madlib.version();\n", |
| "#%sql select version();" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "<a id=\"define_model_arch\"></a>\n", |
| "# 1. Define model architecture\n", |
| "\n", |
| "Import Keras libraries" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 37, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "from tensorflow import keras\n", |
| "from tensorflow.keras.models import Sequential\n", |
| "from tensorflow.keras.layers import Dense" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "Define model architecture" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 38, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "Model: \"sequential_3\"\n", |
| "_________________________________________________________________\n", |
| "Layer (type) Output Shape Param # \n", |
| "=================================================================\n", |
| "dense_9 (Dense) (None, 10) 50 \n", |
| "_________________________________________________________________\n", |
| "dense_10 (Dense) (None, 10) 110 \n", |
| "_________________________________________________________________\n", |
| "dense_11 (Dense) (None, 3) 33 \n", |
| "=================================================================\n", |
| "Total params: 193\n", |
| "Trainable params: 193\n", |
| "Non-trainable params: 0\n", |
| "_________________________________________________________________\n" |
| ] |
| } |
| ], |
| "source": [ |
| "model = Sequential()\n", |
| "model.add(Dense(10, activation='relu', input_shape=(4,)))\n", |
| "model.add(Dense(10, activation='relu'))\n", |
| "model.add(Dense(3, activation='softmax'))\n", |
| " \n", |
| "model.summary();" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 39, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "data": { |
| "text/plain": [ |
| "'{\"class_name\": \"Sequential\", \"keras_version\": \"2.2.4-tf\", \"config\": {\"layers\": [{\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"GlorotUniform\", \"config\": {\"dtype\": \"float32\", \"seed\": null}}, \"name\": \"dense_9\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"dtype\": \"float32\", \"activation\": \"relu\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {\"dtype\": \"float32\"}}, \"units\": 10, \"batch_input_shape\": [null, 4], \"use_bias\": true, \"activity_regularizer\": null}}, {\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"GlorotUniform\", \"config\": {\"dtype\": \"float32\", \"seed\": null}}, \"name\": \"dense_10\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"dtype\": \"float32\", \"activation\": \"relu\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {\"dtype\": \"float32\"}}, \"units\": 10, \"use_bias\": true, \"activity_regularizer\": null}}, {\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"GlorotUniform\", \"config\": {\"dtype\": \"float32\", \"seed\": null}}, \"name\": \"dense_11\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"dtype\": \"float32\", \"activation\": \"softmax\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {\"dtype\": \"float32\"}}, \"units\": 3, \"use_bias\": true, \"activity_regularizer\": null}}], \"name\": \"sequential_3\"}, \"backend\": \"tensorflow\"}'" |
| ] |
| }, |
| "execution_count": 39, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "model.to_json()" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "'{\"class_name\": \"Sequential\", \"keras_version\": \"2.1.6\", \"config\": [{\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"dense_1\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"dtype\": \"float32\", \"activation\": \"linear\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 2, \"batch_input_shape\": [null, 3], \"use_bias\": true, \"activity_regularizer\": null}}, {\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"new_dense\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"activation\": \"linear\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 10, \"use_bias\": true, \"activity_regularizer\": null}}], \"backend\": \"tensorflow\"}'\n", |
| " " |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "<a id=\"load_model_arch\"></a>\n", |
| "# 2. Load model architecture\n", |
| "\n", |
| "Load into model architecture table:" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 40, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "Done.\n", |
| "1 rows affected.\n", |
| "1 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>model_id</th>\n", |
| " <th>model_arch</th>\n", |
| " <th>model_weights</th>\n", |
| " <th>name</th>\n", |
| " <th>description</th>\n", |
| " <th>__internal_madlib_id__</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>1</td>\n", |
| " <td>{u'class_name': u'Sequential', u'keras_version': u'2.1.6', u'config': [{u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_1', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'dtype': u'float32', u'activation': u'relu', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 10, u'batch_input_shape': [None, 4], u'use_bias': True, u'activity_regularizer': None}}, {u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_2', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'activation': u'relu', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 10, u'use_bias': True, u'activity_regularizer': None}}, {u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_3', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'activation': u'softmax', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 3, u'use_bias': True, u'activity_regularizer': None}}], u'backend': u'tensorflow'}</td>\n", |
| " <td>None</td>\n", |
| " <td>Sophie</td>\n", |
| " <td>A simple model</td>\n", |
| " <td>__madlib_temp_27065614_1614901189_16021319__</td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[(1, {u'class_name': u'Sequential', u'keras_version': u'2.1.6', u'config': [{u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u' ... (1340 characters truncated) ... s_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 3, u'use_bias': True, u'activity_regularizer': None}}], u'backend': u'tensorflow'}, None, u'Sophie', u'A simple model', u'__madlib_temp_27065614_1614901189_16021319__')]" |
| ] |
| }, |
| "execution_count": 40, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%%sql\n", |
| "DROP TABLE IF EXISTS model_arch_library;\n", |
| "SELECT madlib.load_keras_model('model_arch_library', -- Output table,\n", |
| " \n", |
| "$$\n", |
| "{\"class_name\": \"Sequential\", \"keras_version\": \"2.1.6\", \"config\": [{\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"dense_1\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"dtype\": \"float32\", \"activation\": \"relu\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 10, \"batch_input_shape\": [null, 4], \"use_bias\": true, \"activity_regularizer\": null}}, {\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"dense_2\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"activation\": \"relu\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 10, \"use_bias\": true, \"activity_regularizer\": null}}, {\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"dense_3\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"activation\": \"softmax\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 3, \"use_bias\": true, \"activity_regularizer\": null}}], \"backend\": \"tensorflow\"}\n", |
| "$$\n", |
| "::json, -- JSON blob\n", |
| " NULL, -- Weights\n", |
| " 'Sophie', -- Name\n", |
| " 'A simple model' -- Descr\n", |
| ");\n", |
| "\n", |
| "SELECT * FROM model_arch_library;" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "Load another model architecture:" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 41, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "1 rows affected.\n", |
| "2 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>model_id</th>\n", |
| " <th>model_arch</th>\n", |
| " <th>model_weights</th>\n", |
| " <th>name</th>\n", |
| " <th>description</th>\n", |
| " <th>__internal_madlib_id__</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>2</td>\n", |
| " <td>{u'class_name': u'Sequential', u'keras_version': u'2.1.6', u'config': [{u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_1', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'dtype': u'float32', u'activation': u'relu', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 10, u'batch_input_shape': [None, 4], u'use_bias': True, u'activity_regularizer': None}}, {u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_2', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'activation': u'relu', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 10, u'use_bias': True, u'activity_regularizer': None}}, {u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_3', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'activation': u'softmax', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 3, u'use_bias': True, u'activity_regularizer': None}}], u'backend': u'tensorflow'}</td>\n", |
| " <td>None</td>\n", |
| " <td>Maria</td>\n", |
| " <td>Also a simple model</td>\n", |
| " <td>__madlib_temp_87665369_1614901189_11144097__</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>1</td>\n", |
| " <td>{u'class_name': u'Sequential', u'keras_version': u'2.1.6', u'config': [{u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_1', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'dtype': u'float32', u'activation': u'relu', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 10, u'batch_input_shape': [None, 4], u'use_bias': True, u'activity_regularizer': None}}, {u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_2', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'activation': u'relu', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 10, u'use_bias': True, u'activity_regularizer': None}}, {u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u'VarianceScaling', u'config': {u'distribution': u'uniform', u'scale': 1.0, u'seed': None, u'mode': u'fan_avg'}}, u'name': u'dense_3', u'kernel_constraint': None, u'bias_regularizer': None, u'bias_constraint': None, u'activation': u'softmax', u'trainable': True, u'kernel_regularizer': None, u'bias_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 3, u'use_bias': True, u'activity_regularizer': None}}], u'backend': u'tensorflow'}</td>\n", |
| " <td>None</td>\n", |
| " <td>Sophie</td>\n", |
| " <td>A simple model</td>\n", |
| " <td>__madlib_temp_27065614_1614901189_16021319__</td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[(2, {u'class_name': u'Sequential', u'keras_version': u'2.1.6', u'config': [{u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u' ... (1340 characters truncated) ... s_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 3, u'use_bias': True, u'activity_regularizer': None}}], u'backend': u'tensorflow'}, None, u'Maria', u'Also a simple model', u'__madlib_temp_87665369_1614901189_11144097__'),\n", |
| " (1, {u'class_name': u'Sequential', u'keras_version': u'2.1.6', u'config': [{u'class_name': u'Dense', u'config': {u'kernel_initializer': {u'class_name': u' ... (1340 characters truncated) ... s_initializer': {u'class_name': u'Zeros', u'config': {}}, u'units': 3, u'use_bias': True, u'activity_regularizer': None}}], u'backend': u'tensorflow'}, None, u'Sophie', u'A simple model', u'__madlib_temp_27065614_1614901189_16021319__')]" |
| ] |
| }, |
| "execution_count": 41, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%%sql\n", |
| "SELECT madlib.load_keras_model('model_arch_library', -- Output table,\n", |
| " \n", |
| "$$\n", |
| "{\"class_name\": \"Sequential\", \"keras_version\": \"2.1.6\", \"config\": [{\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"dense_1\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"dtype\": \"float32\", \"activation\": \"relu\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 10, \"batch_input_shape\": [null, 4], \"use_bias\": true, \"activity_regularizer\": null}}, {\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"dense_2\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"activation\": \"relu\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 10, \"use_bias\": true, \"activity_regularizer\": null}}, {\"class_name\": \"Dense\", \"config\": {\"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"distribution\": \"uniform\", \"scale\": 1.0, \"seed\": null, \"mode\": \"fan_avg\"}}, \"name\": \"dense_3\", \"kernel_constraint\": null, \"bias_regularizer\": null, \"bias_constraint\": null, \"activation\": \"softmax\", \"trainable\": true, \"kernel_regularizer\": null, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"units\": 3, \"use_bias\": true, \"activity_regularizer\": null}}], \"backend\": \"tensorflow\"}\n", |
| "$$\n", |
| "::json, -- JSON blob\n", |
| " NULL, -- Weights\n", |
| " 'Maria', -- Name\n", |
| " 'Also a simple model' -- Descr\n", |
| ");\n", |
| "\n", |
| "SELECT * FROM model_arch_library;" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "<a id=\"load_model_wts\"></a>\n", |
| "# 3. Load model weights\n", |
| "\n", |
| "<a id=\"load_model_wts_madlib\"></a>\n", |
| "## 3a. Load weights from previous MADlib run\n", |
| "\n", |
| "Use UPDATE to load directly into the table. For example, if 'model_weights' are the weights in the output table 'iris_model' from a previous run of 'madlib_keras_fit()' :" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 42, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "1 rows affected.\n", |
| "2 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>model_id</th>\n", |
| " <th>name</th>\n", |
| " <th>description</th>\n", |
| " <th>has_model_weights</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>1</td>\n", |
| " <td>Sophie</td>\n", |
| " <td>A simple model</td>\n", |
| " <td>False</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>2</td>\n", |
| " <td>Maria</td>\n", |
| " <td>Also a simple model</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[(1, u'Sophie', u'A simple model', False),\n", |
| " (2, u'Maria', u'Also a simple model', True)]" |
| ] |
| }, |
| "execution_count": 42, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%%sql\n", |
| "UPDATE model_arch_library\n", |
| "SET model_weights = iris_model.model_weights\n", |
| "FROM iris_model\n", |
| "WHERE model_arch_library.model_id = 2;\n", |
| "\n", |
| "-- Check weights loaded OK\n", |
| "SELECT model_id, name, description, (model_weights IS NOT NULL) AS has_model_weights FROM model_arch_library ORDER BY model_id;" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "<a id=\"load_model_wts_keras1\"></a>\n", |
| "## 3b. Load weights from Keras using a PL/Python function \n", |
| "We need to flatten then serialize the weights to store as a PostgreSQL binary data type. Byte format is more efficient on space and memory compared to a numeric array. The model weights will be de-serialized when passed to Keras functions." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 43, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "Done.\n", |
| "1 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>load_weights</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td></td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[('',)]" |
| ] |
| }, |
| "execution_count": 43, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%%sql\n", |
| "CREATE OR REPLACE FUNCTION load_weights() RETURNS VOID AS\n", |
| "$$\n", |
| "from tensorflow.keras.layers import *\n", |
| "from tensorflow.keras import Sequential\n", |
| "import numpy as np\n", |
| "import plpy\n", |
| "\n", |
| "# create model\n", |
| "model = Sequential()\n", |
| "model.add(Dense(10, activation='relu', input_shape=(4,)))\n", |
| "model.add(Dense(10, activation='relu'))\n", |
| "model.add(Dense(3, activation='softmax'))\n", |
| "\n", |
| "# get weights, flatten and serialize\n", |
| "weights = model.get_weights()\n", |
| "weights_flat = [w.flatten() for w in weights]\n", |
| "weights1d = np.concatenate(weights_flat).ravel()\n", |
| "weights_bytea = weights1d.tostring()\n", |
| "\n", |
| "# load query\n", |
| "load_query = plpy.prepare(\"\"\"SELECT madlib.load_keras_model(\n", |
| " 'model_arch_library',\n", |
| " $1, $2, $3, $4)\n", |
| " \"\"\", ['json','bytea', 'text', 'text'])\n", |
| "plpy.execute(load_query, [model.to_json(), weights_bytea, \"Ella\", \"Model x\"])\n", |
| "$$ language plpythonu;\n", |
| "\n", |
| "-- Call load function\n", |
| "SELECT load_weights();" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 44, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "3 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>model_id</th>\n", |
| " <th>name</th>\n", |
| " <th>description</th>\n", |
| " <th>has_model_weights</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>1</td>\n", |
| " <td>Sophie</td>\n", |
| " <td>A simple model</td>\n", |
| " <td>False</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>2</td>\n", |
| " <td>Maria</td>\n", |
| " <td>Also a simple model</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>3</td>\n", |
| " <td>Ella</td>\n", |
| " <td>Model x</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[(1, u'Sophie', u'A simple model', False),\n", |
| " (2, u'Maria', u'Also a simple model', True),\n", |
| " (3, u'Ella', u'Model x', True)]" |
| ] |
| }, |
| "execution_count": 44, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%%sql\n", |
| "SELECT model_id, name, description, (model_weights IS NOT NULL) AS has_model_weights FROM model_arch_library ORDER BY model_id;" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "<a id=\"load_model_wts_keras2\"></a>\n", |
| "## 3c. Load weights from Keras using psycopg2\n", |
| "Psycopg is a PostgreSQL database adapter for the Python programming language. As above we need to flatten then serialize the weights to store as a PostgreSQL binary data type.\n", |
| "\n", |
| "Note that the psycopg2.Binary function will increase the size of the Python object for the weights, so if your model is large it might be better to use a PL/Python function as in 3b. above." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 45, |
| "metadata": {}, |
| "outputs": [], |
| "source": [ |
| "import psycopg2 as p2\n", |
| "conn = p2.connect('postgresql://gpadmin@localhost:8000/madlib')\n", |
| "cur = conn.cursor()\n", |
| "\n", |
| "from tensorflow.keras.layers import *\n", |
| "from tensorflow.keras import Sequential\n", |
| "import numpy as np\n", |
| "\n", |
| "# create model\n", |
| "model = Sequential()\n", |
| "model.add(Dense(10, activation='relu', input_shape=(4,)))\n", |
| "model.add(Dense(10, activation='relu'))\n", |
| "model.add(Dense(3, activation='softmax'))\n", |
| "\n", |
| "# get weights, flatten and serialize\n", |
| "weights = model.get_weights()\n", |
| "weights_flat = [w.flatten() for w in weights]\n", |
| "weights1d = np.concatenate(weights_flat).ravel()\n", |
| "weights_bytea = p2.Binary(weights1d.tostring())\n", |
| "\n", |
| "query = \"SELECT madlib.load_keras_model('model_arch_library', %s,%s,%s,%s)\"\n", |
| "cur.execute(query,[model.to_json(), weights_bytea, \"Grace\", \"Model y\"])\n", |
| "conn.commit()" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 46, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "4 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>model_id</th>\n", |
| " <th>name</th>\n", |
| " <th>description</th>\n", |
| " <th>has_model_weights</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>1</td>\n", |
| " <td>Sophie</td>\n", |
| " <td>A simple model</td>\n", |
| " <td>False</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>2</td>\n", |
| " <td>Maria</td>\n", |
| " <td>Also a simple model</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>3</td>\n", |
| " <td>Ella</td>\n", |
| " <td>Model x</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>4</td>\n", |
| " <td>Grace</td>\n", |
| " <td>Model y</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[(1, u'Sophie', u'A simple model', False),\n", |
| " (2, u'Maria', u'Also a simple model', True),\n", |
| " (3, u'Ella', u'Model x', True),\n", |
| " (4, u'Grace', u'Model y', True)]" |
| ] |
| }, |
| "execution_count": 46, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%%sql\n", |
| "SELECT model_id, name, description, (model_weights IS NOT NULL) AS has_model_weights FROM model_arch_library ORDER BY model_id;" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "<a id=\"delete_model\"></a>\n", |
| "# 4. Delete model" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 47, |
| "metadata": {}, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "1 rows affected.\n", |
| "3 rows affected.\n" |
| ] |
| }, |
| { |
| "data": { |
| "text/html": [ |
| "<table>\n", |
| " <tr>\n", |
| " <th>model_id</th>\n", |
| " <th>name</th>\n", |
| " <th>description</th>\n", |
| " <th>has_model_weights</th>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>2</td>\n", |
| " <td>Maria</td>\n", |
| " <td>Also a simple model</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>3</td>\n", |
| " <td>Ella</td>\n", |
| " <td>Model x</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| " <tr>\n", |
| " <td>4</td>\n", |
| " <td>Grace</td>\n", |
| " <td>Model y</td>\n", |
| " <td>True</td>\n", |
| " </tr>\n", |
| "</table>" |
| ], |
| "text/plain": [ |
| "[(2, u'Maria', u'Also a simple model', True),\n", |
| " (3, u'Ella', u'Model x', True),\n", |
| " (4, u'Grace', u'Model y', True)]" |
| ] |
| }, |
| "execution_count": 47, |
| "metadata": {}, |
| "output_type": "execute_result" |
| } |
| ], |
| "source": [ |
| "%%sql\n", |
| "SELECT madlib.delete_keras_model('model_arch_library', -- Output table\n", |
| " 1 -- Model id\n", |
| " );\n", |
| "SELECT model_id, name, description, (model_weights IS NOT NULL) AS has_model_weights FROM model_arch_library ORDER BY model_id;" |
| ] |
| } |
| ], |
| "metadata": { |
| "kernelspec": { |
| "display_name": "Python 2", |
| "language": "python", |
| "name": "python2" |
| }, |
| "language_info": { |
| "codemirror_mode": { |
| "name": "ipython", |
| "version": 2 |
| }, |
| "file_extension": ".py", |
| "mimetype": "text/x-python", |
| "name": "python", |
| "nbconvert_exporter": "python", |
| "pygments_lexer": "ipython2", |
| "version": "2.7.16" |
| } |
| }, |
| "nbformat": 4, |
| "nbformat_minor": 1 |
| } |
