tree 58f437b519ee586846ea6da1f6730dde1d5ecb1c
parent 1fb8c5e680171d4fafa5a42a23be927b5600ccac
author e-strauss <92718421+e-strauss@users.noreply.github.com> 1686305736 +0200
committer Arnab Phani <phaniarnab@gmail.com> 1686305758 +0200

[SYSTEMDS-3580] Add word embedding encoder

This patch extends the transformapply API to accept the pre-trained word
embeddings along with the dictionary as inputs. The new word embedding
column encoder is placed after recode and replace the recoded indices
with the embedding vectors. This addition removes the requirement of
a matrix multiplication to produce the embedding matrix.
The current implementation is slower than the baseline (w/ MatMult).
The future commits will introduce a new dense block to deduplicate
the large embeddings and multi-threading.

Closes #1839