There are three different ways to implement a Deep Learning model in SystemML:
NN library | Caffe2DML | Keras2DML | |
---|---|---|---|
External dependency | None | None | Keras |
Ability to add custom layers | Yes | No | No |
The user needs to know | DML | Caffe's proto API | Keras' API |
Can be invoked using pyspark | Yes. Please see Python MLContext API | Yes. | Yes. |
Can be invoked using spark-shell | Yes. Please see Scala MLContext API | Limited support | No |
Can be invoked via command-line or JMLC API | Yes | No | No |
GPU and native BLAS support | Yes | Yes | Yes |
Part of SystemML's mllearn API | No | Yes | Yes |
Before we go any further, let us briefly discuss the training and prediction functions in the mllearn API (i.e. Caffe2DML and Keras2DML).
Please note that when training using mllearn API (i.e. model.fit(X_df)
), SystemML expects that labels have been converted to 1-based value. This avoids unnecessary decoding overhead for large dataset if the label columns has already been decoded. For scikit-learn API, there is no such requirement.
Download the MNIST dataset using mlxtend package.
from mlxtend.data import mnist_data import numpy as np from sklearn.utils import shuffle # Download the MNIST dataset X, y = mnist_data() X, y = shuffle(X, y) # Split the data into training and test n_samples = len(X) X_train = X[:int(.9 * n_samples)] y_train = y[:int(.9 * n_samples)] X_test = X[int(.9 * n_samples):] y_test = y[int(.9 * n_samples):]
ml = MLContext(sc) ml.setStatistics(True)
script = """ source(“nn/examples/mnist_lenet.dml”) as mnist_lenet
images = (images / 255) * 2 - 1 n = nrow(images) labels = table(seq(1, n), labels+1, n, 10)
X = images[501:nrow(images),] X_val = images[1:500,] y = labels[501:nrow(images),] y_val = labels[1:500,]
[W1, b1, W2, b2, W3, b3, W4, b4] = mnist_lenet::train(X, y, X_val, y_val, C, Hin, Win, epochs) """ out = (‘W1’, ‘b1’, ‘W2’, ‘b2’, ‘W3’, ‘b3’, ‘W4’, ‘b4’) prog = (dml(script).input(images=X_train, labels=y_train.reshape((-1, 1)), epochs=1, C=1, Hin=28, Win=28) .output(*out))
W1, b1, W2, b2, W3, b3, W4, b4 = ml.execute(prog).get(*out)
script_predict = """ source(“nn/examples/mnist_lenet.dml”) as mnist_lenet
X_test = (X_test / 255) * 2 - 1
y_prob = mnist_lenet::predict(X_test, C, Hin, Win, W1, b1, W2, b2, W3, b3, W4, b4) y_pred = rowIndexMax(y_prob) - 1 """ prog = (dml(script_predict).input(X_test=X_test, C=1, Hin=28, Win=28, W1=W1, b1=b1, W2=W2, b2=b2, W3=W3, b3=b3, W4=W4, b4=b4) .output(“y_pred”))
y_pred = ml.execute(prog).get(“y_pred”).toNumPy() {% endhighlight %}
urllib.urlretrieve(‘https://raw.githubusercontent.com/apache/systemml/master/scripts/nn/examples/caffe2dml/models/mnist_lenet/lenet.proto’, ‘lenet.proto’) urllib.urlretrieve(‘https://raw.githubusercontent.com/apache/systemml/master/scripts/nn/examples/caffe2dml/models/mnist_lenet/lenet_solver.proto’, ‘lenet_solver.proto’)
lenet = Caffe2DML(spark, solver=‘lenet_solver.proto’, input_shape=(1, 28, 28)) lenet.setStatistics(True)
lenet.fit(X_train, y_train)
lenet.score(X_test, y_test) {% endhighlight %}
scale = 0.00390625 X_train = X_trainscale X_test = X_testscale
from systemml.mllearn import Keras2DML sysml_model = Keras2DML(spark, keras_model, input_shape=(1,28,28), weights=‘weights_dir’)
sysml_model.summary() sysml_model.fit(X_train, y_train) sysml_model.score(X_test, y_test) {% endhighlight %}
keras_model = ResNet50(weights=‘imagenet’,include_top=True,pooling=‘None’,input_shape=(224,224,3)) keras_model.compile(optimizer=‘sgd’, loss= ‘categorical_crossentropy’)
sysml_model = Keras2DML(spark,keras_model,input_shape=(3,224,224), weights=‘weights_dir’, labels=‘https://raw.githubusercontent.com/apache/systemml/master/scripts/nn/examples/caffe2dml/models/imagenet/labels.txt’) sysml_model.summary() urllib.urlretrieve(‘https://upload.wikimedia.org/wikipedia/commons/f/f4/Cougar_sitting.jpg’, ‘test.jpg’) img_shape = (3, 224, 224) input_image = sml.convertImageToNumPyArr(Image.open(‘test.jpg’), img_shape=img_shape) sysml_model.predict(input_image) {% endhighlight %}