tree 08c3a97c98185fcbca8f9bc650d7396451e97ea6
parent 8d19611be76386ed1f6eb17b357f1202ed2e3d1a
author Rodrigo Pardo Meza <ro.pardo.meza@gmail.com> 1647867780 +0100
committer bertty <bertty@apache.org> 1647901459 +0100

Wayang 8 (#89)

* [WAYANG-8][API-PYTHON] Creation of functions to be consumed by MapPartitionsDescriptor

* [WAYANG-8][API-PYTHON] Included PythonProcessCaller that manages the python process execution and Java - Python connection

* [WAYANG-8][API-PYTHON] POM fixes plus minor test

* [WAYANG-8][API-PYTHON] Python connection through TCP socket enabled

* [WAYANG-8][API-PYTHON] Writing from Java to Python. Not taking into care about Iterator Datatypes.

* [WAYANG-8][API-PYTHON] Java Socket Writter improvements

* [WAYANG-8][API-PYTHON] Python UTF8 Deserializer included

* [WAYANG-8][API-PYTHON] Python UTF8 Reading Stream

* [WAYANG-8][API-PYTHON] Getting results from Python and continue processing

* [WAYANG-8][API-PYTHON] Config files for pywayang

* [WAYANG-8][API-PYTHON] Structures to save the plan with functional fashion plus most basic operators

* [WAYANG-8][API-PYTHON] Main program to test plan executions locally

* [WAYANG-8][API-PYTHON] Minor comments and
TODOs

* [WAYANG-8][API-PYTHON] Most basic test for protobuff communication with java

* [WAYANG-8][API-PYTHON] Addjacency list from PyWayang Plan

* [WAYANG-8][API-PYTHON] Graph traversal implementation with visitor pattern

* [WAYANG-8][API-PYTHON] Protobuf python message generator

* [WAYANG-8][API-PYTHON] Wayang Web Service project structure

* [WAYANG-8][API-PYTHON] Protobuf message generation fixes

* [WAYANG-8][API-PYTHON] Wayang Web Service executes most basic plans directly

* [WAYANG-8][API-PYTHON] Receiving Base64 passing to byte array and unpickling

* [WAYANG-8][API-PYTHON] Updated classes to process a single Serialized UDF

* [WAYANG-8][API-PYTHON] New test with single UDF

* [WAYANG-8][API-PYTHON] Protobuf command

* [WAYANG-8][API-PYTHON] Protobuf message template updated

* [WAYANG-8][API-PYTHON] POM fixes

* [WAYANG-8][API-PYTHON] License comments added

* [WAYANG-8][API-PYTHON] Correction on missing licenses

* [WAYANG-8][API-PYTHON] Serializable module creation

* [WAYANG-8][API-PYTHON] adding protoc to travis

* [WAYANG-8][API-PYTHON] protoc executable path correction

* [WAYANG-8][API-PYTHON] Commenting objc_class_prefix

* [WAYANG-8][API-PYTHON] Obtaining pipelines

* [WAYANG-8][API-PYTHON] Dataquanta writing message

* [WAYANG-8][API-PYTHON] Plan writer pipeline based adjustments

* [WAYANG-8][API-PYTHON] Operator Python executable indicator

* [WAYANG-8][API-PYTHON] Plan writer improved to use less sockets

* [WAYANG-8][API-PYTHON] New version of Wayang protobuf message

* [WAYANG-8][API-PYTHON] Wayang REST improved to allow multi pipelined executions

* [WAYANG-8][API-PYTHON] More test programs

* [WAYANG-8][API-PYTHON] Commentaries and logging for Graph module

* [WAYANG-8][API-PYTHON] Commentaries and logging for Orchestrator module

* [WAYANG-8][API-PYTHON] Commentaries and logging for Protobuf module

* [WAYANG-8][API-PYTHON] Fix usage of relative paths

* [WAYANG-8][API-PYTHON] Scripts to compile protobuf has been deleted. Now Maven executes them

* [WAYANG-8][API-PYTHON] Execution Log configuration

* [WAYANG-8][API-PYTHON] Fix - Python Map partition with single operator

* [WAYANG-8][API-PYTHON] Unitary Testing preparing the Wayang Plan

* [WAYANG-8][API-PYTHON] Plugin selection through Plan Descriptor

* [WAYANG-8][API-PYTHON] Unitary Testing preparing the Wayang Plan with Spark Execution

* [WAYANG-8][API-PYTHON] Pywayang sends protobuf message in API request as bytes using base64

* [WAYANG-8][API-PYTHON] New Operators Flatmap group by, reduce and Reduce By Key. Only Python Side.

* [WAYANG-8][API-PYTHON] Protobuf Wayang Plan message updated to allow more Complex Java-Python Operators

* [WAYANG-8][API-PYTHON] Adding TPC-H 1st Test

* [WAYANG-8][API-PYTHON] Last changes, not working

* [WAYANG-8] Fixing errors with dependencies

* [WAYANG-8] Fix to Pom versions problem

* [WAYANG-8] Protoc path updated

* [WAYANG-8] Correction in the pom.xml for flags

Signed-off-by: bertty <bertty@apache.org>

Co-authored-by: berttty <bertty@scalytics.io>
Co-authored-by: Bertty Contreras-Rojas <bertty@databloom.ai>