[SYSTEMDS-3330] Documentation of builtin functions (main) This commit adds new automatic generation of separated doc files for all builtin functions, and automatic generation of tests for builtin functions. All based on the original documentation in individual builtin scripts. This change will simplify the development of builtin functions and help new users across both the Python api and internal DML testing. An example of documented code is: .. code-block:: python >>> import numpy as np >>> from systemds.context import SystemDSContext >>> from systemds.operator.algorithm import dist >>> >>> with SystemDSContext() as sds: ... X = sds.from_numpy(np.array([[0], [3], [4]])) ... out = dist(X).compute() ... print(out) [[0. 3. 4.] [3. 0. 1.] [4. 1. 0.]] Closes #2292
Overview: Apache SystemDS is an open-source machine learning (ML) system for the end-to-end data science lifecycle from data preparation and cleaning, over efficient ML model training, to debugging and serving. ML algorithms or pipelines are specified in a high-level language with R-like syntax or related Python and Java APIs (with many builtin primitives), and the system automatically generates hybrid runtime plans of local, in-memory operations and distributed operations on Apache Spark. Additional backends exist for GPUs and federated learning.
| Resource | Links |
|---|---|
| Quick Start | Install, Quick Start and Hello World |
| Documentation: | SystemDS Documentation |
| Python Documentation | Python SystemDS Documentation |
| Issue Tracker | Jira Dashboard |
Status and Build: SystemDS is renamed from SystemML which is an Apache Top Level Project. To build from source visit SystemDS Install from source