| commit | 505f8711229b954198c0cee1f1f727d42b767632 | [log] [tgz] |
|---|---|---|
| author | MaximilianTUB <maxschreff@gmail.com> | Sat Mar 30 19:37:51 2024 +0100 |
| committer | Arnab Phani <phaniarnab@gmail.com> | Sat Mar 30 20:20:18 2024 +0100 |
| tree | 69aa63792c66016b3007890394049c5297c4acf9 | |
| parent | 14e2995409dfb65aca49cee9e56f096cd6b0fb7f [diff] |
[SYSTEMDS-3689] Complete ResNet Integration These commits add the full integration of ResNets. The forward passes for the basic residual block architecture (ResNet18 & 34) have been previously added. These commits add the forward passes for the bottleneck architecture (ResNet50, 101 & 152) and the backward passes for all architectures. Additionally, since the iteration through the model parameters, especially for the bigger models, gets quite complex, I implemented utiltiy functions for the updating the parameters with differen optimizers. For each model and for each optimizer, a uitlity method to initialize the optimizer state parameters and to update the model parameters have been added. There is also an example added which uses random data to showcase the full training loop of the ResNets (forward, backward, param updating). The example uses ResNet18 but can changed by simply modifying the import statement to another model. Also, the ADAM optimizer is used but I also added all the other optimizers in the example and can be used by uncommenting the respective lines. The example was tested for all ResNets (18, 34, 50, 101, 152) and for all optimizers. Every combination trains correctly (i.e. the loss shrinks). About testing, I've added comprehensive testing for the forward and backward pass of the residual blocks (basic and bottleneck architectures) and for the residual layer's forward pass (multiple residual blocks) but not for the residual layer backward pass and the full networks because the example data would take up to much space. Closes #1992 Closes #2010
Overview: SystemDS is an open source ML system for the end-to-end data science lifecycle from data integration, cleaning, and feature engineering, over efficient, local and distributed ML model training, to deployment and serving. To this end, we aim to provide a stack of declarative languages with R-like syntax for (1) the different tasks of the data-science lifecycle, and (2) users with different expertise. These high-level scripts are compiled into hybrid execution plans of local, in-memory CPU and GPU operations, as well as distributed operations on Apache Spark. In contrast to existing systems - that either provide homogeneous tensors or 2D Datasets - and in order to serve the entire data science lifecycle, the underlying data model are DataTensors, i.e., tensors (multi-dimensional arrays) whose first dimension may have a heterogeneous and nested schema.
| Resource | Links |
|---|---|
| Quick Start | Install, Quick Start and Hello World |
| Documentation: | SystemDS Documentation |
| Python Documentation | Python SystemDS Documentation |
| Issue Tracker | Jira Dashboard |
Status and Build: SystemDS is renamed from SystemML which is an Apache Top Level Project. To build from source visit SystemDS Install from source
