Each Tensor instance is a multi-dimensional array allocated on a specific Device instance. Tensor instances store variables and provide linear algebra operations over different types of hardware devices without user awareness. Note that users need to make sure the tensor operands are allocated on the same device except copy functions.
>>> import numpy as np >>> from singa import tensor >>> tensor.from_numpy( np.asarray([[1, 0, 0], [0, 1, 0]], dtype=np.float32) ) [[1. 0. 0.] [0. 1. 0.]]
>>> a = np.asarray([[1, 0, 0], [0, 1, 0]], dtype=np.float32) >>> tensor.from_numpy(a) [[1. 0. 0.] [0. 1. 0.]] >>> tensor.to_numpy(tensor.from_numpy(a)) array([[1., 0., 0.], [0., 1., 0.]], dtype=float32)
>>> t = tensor.from_numpy(a) >>> t.transpose([1,0]) [[1. 0.] [0. 1.] [0. 0.]]
tensor
is evaluated in real time.
>>> t + 1 [[2. 1. 1.] [1. 2. 1.]] >>> t / 5 [[0.2 0. 0. ] [0. 0.2 0. ]]
Functions in module singa.tensor
return new tensor
object after applying defined transformation in the function.
>>> tensor.log(t+1) [[0.6931472 0. 0. ] [0. 0.6931472 0. ]]
tensor
is created on host(CPU) by default, and can also be created on different backends by specifiying the device
. Existing tensor
could also be moved between device
by to_device()
.
>>> from singa import device >>> x = tensor.Tensor((2, 3), device.create_cuda_gpu()) >>> x.gaussian(1,1) >>> x [[1.531889 1.0128608 0.12691343] [2.1674204 3.083676 2.7421203 ]] >>> # move to host >>> x.to_device(device.get_default_device())
from singa import device from singa import tensor from singa import opt from singa import autograd class MLP: def __init__(self): self.linear1 = autograd.Linear(3, 4) self.linear2 = autograd.Linear(4, 5) def forward(self, x): y=self.linear1(x) return self.linear2(y) def train(model, x, t, dev, epochs=10): for i in range(epochs): y = model.forward(x) loss = autograd.mse_loss(y, t) print("loss: ", loss) sgd = opt.SGD() for p, gp in autograd.backward(loss): sgd.update(p, gp) sgd.step() print("training completed") if __name__ == "__main__": autograd.training = True model = MLP() dev = device.get_default_device() x = tensor.Tensor((2, 3), dev) t = tensor.Tensor((2, 5), dev) x.gaussian(1,1) t.gaussian(1,1) train(model, x, t, dev)
Output:
loss: [4.917431] loss: [2.5147934] loss: [2.0670078] loss: [1.9179827] loss: [1.8192691] loss: [1.7269677] loss: [1.6308627] loss: [1.52674] loss: [1.4122975] loss: [1.2866782] training completed
SINGA has three different sets of implmentations of Tensor functions, one for each type of Device.
work in progress
work in progress