| <!-- |
| |
| --> |
| <!-- |
| |
| --> |
| <!DOCTYPE html> |
| <html lang="en"> |
| <!-- |
| |
| --> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| |
| <title>SystemML 0.10.0-incubating Release Notes</title> |
| |
| <meta name="description" content="Project Release Notes"> |
| <meta name="author" content="Apache SystemDS"> |
| |
| <!-- Enable responsive viewport --> |
| <meta name="HandheldFriendly" content="True"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| |
| <!-- You can use Open Graph tags to customize link previews. |
| Learn more: https://developers.facebook.com/docs/sharing/webmasters --> |
| <meta property="og:url" content="https://systemds.apache.org/" /> |
| <meta property="og:type" content="website" /> |
| <meta property="og:title" content="SystemML 0.10.0-incubating Release Notes" /> |
| <meta property="og:description" content="Apache SystemDS is an open source ML system for the end-to-end data science lifecycle" /> |
| <meta property="og:image" content="" /> |
| |
| <!-- Le HTML5 shim, for IE6-8 support of HTML elements --> |
| <!--[if lt IE 9]> |
| <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script> |
| <![endif]--> |
| |
| <!-- Le styles --> |
| <link rel="stylesheet" href="/assets/css/main.css"> |
| |
| <!-- favicons --> |
| <link rel="shortcut icon" href="/assets/img/favicon.png"> |
| </head> <!-- META --> |
| <body class="vcard"> |
| <!-- |
| |
| --> |
| <header class="site-header site-header--not-home"> |
| <h1 class="logo"><a class="url" href="/"><i class="logo-mark"></i><span class="fn org">Apache SystemDS<sup |
| id="trademark">™</sup></span></a></h1> |
| <nav class="main-nav"> |
| <ul> |
| |
| <li role="presentation"> |
| |
| |
| <a href="/download" target="_self">Download</a> |
| |
| </li> |
| |
| <li role="presentation"> |
| |
| |
| <a href="/get-started" target="_self">Get Started</a> |
| |
| </li> |
| |
| <li role="presentation"> |
| |
| <a class="nav-link--hover">Docs <i class="icon icon-chevron-down"></i></a> |
| <ul> |
| |
| |
| <li><a href="https://apache.github.io/systemds/" target="_blank">latest</a></li> |
| |
| |
| <li><a href="https://systemds.apache.org/docs/3.1.0/" target="_blank">3.1.0 (stable)</a></li> |
| |
| |
| <li><a href="https://systemds.apache.org/release-notes/systemds-release-notes-3.1.0" target="_blank">Release Notes 3.1.0</a></li> |
| |
| </ul> |
| |
| </li> |
| |
| <li role="presentation"> |
| |
| <a class="nav-link--hover">Community <i class="icon icon-chevron-down"></i></a> |
| <ul> |
| |
| |
| <li><a href="/community" target="_self">Get Involved</a></li> |
| |
| |
| <li><a href="https://issues.apache.org/jira/browse/SYSTEMDS" target="_blank">Issue Tracker</a></li> |
| |
| |
| <li><a href="https://github.com/apache/systemds" target="_blank">Source Code</a></li> |
| |
| |
| <li><a href="https://github.com/apache/systemds-website" target="_blank">Website Source Code</a></li> |
| |
| </ul> |
| |
| </li> |
| |
| <li role="presentation"> |
| |
| <a class="nav-link--hover">Apache <i class="icon icon-chevron-down"></i></a> |
| <ul> |
| |
| |
| <li><a href="https://www.apache.org/foundation/how-it-works" target="_blank">Apache Software Foundation</a></li> |
| |
| |
| <li><a href="https://www.apache.org/licenses/" target="_blank">Apache License</a></li> |
| |
| |
| <li><a href="https://www.apache.org/foundation/sponsorship" target="_blank">Sponsorship</a></li> |
| |
| |
| <li><a href="https://www.apache.org/foundation/thanks" target="_blank">Thanks</a></li> |
| |
| |
| <li><a href="/privacy-policy" target="_self">Privacy Policy</a></li> |
| |
| |
| <li><a href="/security" target="_self">Security</a></li> |
| |
| </ul> |
| |
| </li> |
| |
| </ul> |
| </nav> |
| </header> <!-- GLOBAL HEADER --> |
| <!-- |
| |
| --> |
| <!-- |
| |
| --> |
| <div> |
| <!-- |
| |
| --> |
| |
| <section class="full-stripe full-stripe--subpage-header clear-header"> |
| <div class="ml-container ml-container--horizontally-center"> |
| <div class="col col-12 content-group content-group--center-content content-group--center-align"> |
| <h1>Apache SystemDS<sup id="trademark">™</sup> 0.10.0-incubating Release Notes</h1> |
| </div> |
| </div> |
| </section> |
| |
| <section class="full-stripe full-stripe--alternate"> |
| <div class="ml-container"> |
| <div class="col col-12 content-group content-group--medium-bottom-margin"> |
| |
| <p>The <strong>Apache SystemML 0.10.0-incubating release</strong> was approved on June 3rd, 2016. It is the second release of Apache SystemML since it |
| became an incubator project on November 2nd, 2015. Extensive updates have been made to the project in several areas, as detailed |
| below.</p> |
| |
| <h3 id="different-types-of-spark-matrix-blocks">Different Types of Spark Matrix Blocks</h3> |
| <ul> |
| <li>Supported internal formats: MCSR (default), CSR, COO</li> |
| <li>Automatic MCSR➡CSR on Spark read/caching (for memory efficiency)</li> |
| <li>Automatic MCSR➡CSR on sparse update-in-place (avoid serialization)</li> |
| </ul> |
| |
| <h3 id="frame-support-for-jmlc-apicp">Frame Support for JMLC API/CP</h3> |
| <ul> |
| <li>New frame data type, deeply integrated into compiler and runtime</li> |
| <li>New builtin functions: transformapply, transformencode, transformdecode, transformmeta</li> |
| <li>Supported operations: read/write, left/right indexing, casting, append, transform/transformapply</li> |
| </ul> |
| |
| <h3 id="framework-compatibilityconfiguration">Framework Compatibility/Configuration</h3> |
| <ul> |
| <li>[SYSTEMML-418] Version-specific Spark memory budgets (>=1.6, legacy)</li> |
| <li>[SYSTEMML-158] Updated deprecated Hadoop properties</li> |
| <li>[SYSTEMML-476] Version-specific MR configuration handling (MRv2, MRv1)</li> |
| <li>Fixes for backwards compatibility to MRv1 (Guava dependency conflicts, runtime changes such as task handling for multiple output committer)</li> |
| <li>New pass-through mapred/mapreduce configurations through SystemML-config</li> |
| <li>[SYSTEMML-584/585] New thread-local configuration handling (compiler/DML config)</li> |
| </ul> |
| |
| <h3 id="deep-learning-support">Deep Learning Support</h3> |
| <ul> |
| <li>[SYSTMEML-618] New DML-script NN library</li> |
| <li>[SYSTEMML-540] New built-in singlenode operations: conv2d, maxpooling, im2col, col2im, rotate</li> |
| <li>New lenet-train DML script</li> |
| </ul> |
| |
| <h3 id="apiscript-usability">API/Script Usability</h3> |
| <ul> |
| <li>[SYSTEMML-607/604/611] Parser error handling</li> |
| <li>[SYSTEMML-506/508/544/577/649/651] Extended MLContext/JMLC APIs</li> |
| <li>[SYSTEMML-625/626/632] Improved source statement handling (e.g., imports, absolute paths)</li> |
| <li>[SYSTEMML-617/631/654] Improved namespace handling</li> |
| <li>[SYSTEMML-240] Extended stats outputs for Spark collect/broadcast/parallelize</li> |
| <li>[SYSTEMML-495] SystemML configuration handling</li> |
| <li>[SYSTEMML-209] Include algorithms in SystemML jar</li> |
| <li>[SYSTEMML-647/648] Deprecated castAsScalar, ppred</li> |
| <li>[SYSTEMML-477] JSON meta data handling</li> |
| <li>[SYSTEMML-294] Print matrix built-in function</li> |
| <li>[SYSTEMML-296/676/670] Improved PyDML syntax: slicing, rand, cdf, elif</li> |
| <li>[SYSTEMML-675] Support for negative for/parfor loop increments</li> |
| </ul> |
| |
| <h3 id="new-fused-physical-operators">New Fused Physical Operators</h3> |
| <ul> |
| <li>[SYSTEMML-488] Fused wdivmm w/ 4 operands</li> |
| <li>[SYSTEMML-510] Fused wdivmm/wcemm w/ eps term</li> |
| </ul> |
| |
| <h3 id="various-performance-features">Various Performance Features</h3> |
| <ul> |
| <li>[SYSTEMML-427/512] Extended IPA (propagate scalar variables)</li> |
| <li>[SYSTEMML-282] Extended update-in-place support for parfor intermediates</li> |
| <li>[SYSTEMML-552/399] Performance parallel binary/text readers (sort sparse/nnz handling)</li> |
| <li>[SYSTEMML-552/641] Cache-conscious operations: sparse-dense wdivmm/wsloss, sparse-dense/sparse-sparse mm, dense-dense skinny rhs mm</li> |
| <li>[SYSTEMML-641] Tuned special cases for block matrix multiplication: e.g., mm w/ skinny rhs, colwise parallelization wide rhs</li> |
| <li>[SYSTEMML-396/400] New/extended multithreaded operations: cumsum/cummin/cummax/cumprod, transpose, and rand</li> |
| <li>[SYSTEMML-510/694] New simplification rewrites: “pushdown unaryagg-transpose”, “simplify transpose-aggbin-binary chains”, “reorder minus-mmult”, “canonicalize matmult-add-scalar”, improved constant folding (all unary)</li> |
| <li>[SYSTEMML-653] Asynchronous bufferpool cleanup of evicted files/nio file eviction</li> |
| <li>MR iqm/quantile/median (qsort num reducers, qpick buffer size)</li> |
| </ul> |
| |
| <h3 id="dml-script-updates">DML Script Updates</h3> |
| <ul> |
| <li>[SYSTEMML-536] New KNN algorithm (still staging)</li> |
| <li>[SYSTEMML-534] Optional console output univariate statistics</li> |
| <li>[SYSTMEML-494] GLM compiler warnings</li> |
| <li>Robustness input/output handling L2SVM, MSVM, and Naive Bayes</li> |
| <li>Random data generator for ALS</li> |
| </ul> |
| |
| <h3 id="various-fixes">Various Fixes</h3> |
| <ul> |
| <li>Dozens of fixes for diverse issues, fix pack for 0.9 release</li> |
| </ul> |
| |
| <h3 id="build-documentation-examples">Build, Documentation, Examples</h3> |
| <ul> |
| <li>[SYSTEMML-551] Enhanced JMLC javadoc</li> |
| <li>[SYSTEMML-484] Build javadoc jar</li> |
| <li>[SYSTEMML-468] Contributing to SystemML doc</li> |
| <li>[SYSTEMML-517/524] DML Language Reference updates</li> |
| <li>[SYSTEMML-498] Troubleshooting guide</li> |
| <li>SystemML Jupyter/Zeppelin Notebook examples</li> |
| </ul> |
| |
| </div></div></section> |
| |
| </div> |
| |
| |
| <!-- |
| |
| --> |
| <footer class="site-footer"> |
| <div class="ml-container ml-container--horizontally-center"> |
| <div class="footer-col-wrapper"> |
| <div style="margin-top: 20px; margin-bottom: 20px;"> |
| <a href="https://www.apache.org"><img src="/assets/img/apache-foundation-logo.png" alt="Apache Incubator" /></a> |
| </div> |
| <!--Footer copyright year update credit: http://updateyourfooter.com/--> |
| <div> |
| Copyright © 2015 |
| <script>new Date().getFullYear() > 2015 && document.write("-" + new Date().getFullYear());</script> |
| <a href="https://www.apache.org">The Apache Software Foundation</a>. |
| Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>. |
| <br> |
| Apache and the Apache Feather logo are trademarks of The Apache Software Foundation. |
| </div> |
| </div> |
| </div> |
| </footer> <!-- GLOBAL FOOTER--> |
| <script src="assets/js/bundle.min.js"></script> |
| </body> |
| </html> |
| |
