docs/api/python/index.html - systemds - Git at Google



 <!DOCTYPE html>
 <html class="writer-html5" lang="en" >
 <head>
   <meta charset="utf-8">

   <meta name="viewport" content="width=device-width, initial-scale=1.0">

   <title>SystemDS &mdash; SystemDS 2.0.0 documentation</title>


   <link rel="stylesheet" href="static/css/theme.css" type="text/css" />
   <link rel="stylesheet" href="static/pygments.css" type="text/css" />


   <!--[if lt IE 9]>
     <script src="static/js/html5shiv.min.js"></script>
   <![endif]-->


       <script type="text/javascript" id="documentation_options" data-url_root="./" src="static/documentation_options.js"></script>
         <script src="static/jquery.js"></script>
         <script src="static/underscore.js"></script>
         <script src="static/doctools.js"></script>
         <script src="static/language_data.js"></script>

     <script type="text/javascript" src="static/js/theme.js"></script>


     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
     <link rel="next" title="Install SystemDS" href="getting_started/install.html" />
 </head>

 <body class="wy-body-for-nav">


   <div class="wy-grid-for-nav">

     <nav data-toggle="wy-nav-shift" class="wy-nav-side">
       <div class="wy-side-scroll">
         <div class="wy-side-nav-search" >


             <a href="#" class="icon icon-home" alt="Documentation Home"> SystemDS


           </a>


 <div role="search">
   <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
     <input type="text" name="q" placeholder="Search docs" />
     <input type="hidden" name="check_keywords" value="yes" />
     <input type="hidden" name="area" value="default" />
   </form>
 </div>


         </div>


         <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">


               <p class="caption"><span class="caption-text">Getting Started:</span></p>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="getting_started/install.html">Install SystemDS</a></li>
 <li class="toctree-l1"><a class="reference internal" href="getting_started/simple_examples.html">QuickStart</a></li>
 </ul>
 <p class="caption"><span class="caption-text">Guides</span></p>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="guide/federated.html">Federated Environment</a></li>
 <li class="toctree-l1"><a class="reference internal" href="guide/algorithms_basics.html">Built-in Algorithms</a></li>
 </ul>
 <p class="caption"><span class="caption-text">API</span></p>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="api/operator/algorithms.html">Algorithms</a></li>
 <li class="toctree-l1"><a class="reference internal" href="api/context/systemds_context.html">SystemDSContext</a></li>
 <li class="toctree-l1"><a class="reference internal" href="api/matrix/matrix.html">Matrix</a></li>
 <li class="toctree-l1"><a class="reference internal" href="api/matrix/federated.html">Federated</a></li>
 <li class="toctree-l1"><a class="reference internal" href="api/operator/operation_node.html">Operation Node</a></li>
 </ul>
 <p class="caption"><span class="caption-text">Internals API</span></p>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="api/script_building/dag.html">Dag</a></li>
 <li class="toctree-l1"><a class="reference internal" href="api/script_building/script.html">Script</a></li>
 <li class="toctree-l1"><a class="reference internal" href="api/utils/converters.html">Converters</a></li>
 <li class="toctree-l1"><a class="reference internal" href="api/utils/helpers.html">Helpers</a></li>
 </ul>


         </div>

       </div>
     </nav>

     <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">


       <nav class="wy-nav-top" aria-label="top navigation">

           <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
           <a href="#">SystemDS</a>

       </nav>


       <div class="wy-nav-content">

         <div class="rst-content">


 <div role="navigation" aria-label="breadcrumbs navigation">

   <ul class="wy-breadcrumbs">

       <li><a href="#" class="icon icon-home"></a> &raquo;</li>

       <li>SystemDS</li>


       <li class="wy-breadcrumbs-aside">


             <a href="sources/index.rst.txt" rel="nofollow"> View page source</a>


       </li>

   </ul>


   <hr/>
 </div>
           <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
            <div itemprop="articleBody">

   <div class="section" id="systemds">
 <h1>SystemDS<a class="headerlink" href="#systemds" title="Permalink to this headline">¶</a></h1>
 <p>This package provides a Pythonic interface for working with SystemDS.</p>
 <p>SystemDS is a versatile system for the end-to-end data science lifecycle from data integration,
 cleaning, and feature engineering, over efficient, local and distributed ML model training,
 to deployment and serving.
 To facilitate this, bindings from different languages and different system abstractions provide help for:</p>
 <ol class="arabic simple">
 <li><p>The different tasks of the data-science lifecycle, and</p></li>
 <li><p>users with different expertise.</p></li>
 </ol>
 <p>These high-level scripts are compiled into hybrid execution plans of local, in-memory CPU and GPU operations,
 as well as distributed operations on Apache Spark. In contrast to existing systems - that either
 provide homogeneous tensors or 2D datasets - and in order to serve the entire
 data science lifecycle, the underlying data model are DataTensors, i.e.,
 tensors (multi-dimensional arrays) whose first dimension may have a heterogeneous and nested schema.</p>
 <div class="toctree-wrapper compound">
 </div>
 <div class="toctree-wrapper compound">
 </div>
 <div class="toctree-wrapper compound">
 </div>
 <div class="toctree-wrapper compound">
 </div>
 </div>


            </div>

           </div>
           <footer>

     <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">

         <a href="getting_started/install.html" class="btn btn-neutral float-right" title="Install SystemDS" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>


     </div>


   <hr/>

   <div role="contentinfo">
     <p>

         &copy; Copyright 2020, Apache SystemDS

     </p>
   </div>


     Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a

     <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a>

     provided by <a href="https://readthedocs.org">Read the Docs</a>.

 </footer>

         </div>
       </div>

     </section>

   </div>


   <script type="text/javascript">
       jQuery(function () {
           SphinxRtdTheme.Navigation.enable(true);
       });
   </script>


 </body>
 </html>


	<!DOCTYPE html>
	<html class="writer-html5" lang="en" >
	<head>
	<meta charset="utf-8">

	<meta name="viewport" content="width=device-width, initial-scale=1.0">

	<title>SystemDS — SystemDS 2.0.0 documentation</title>



	<link rel="stylesheet" href="static/css/theme.css" type="text/css" />
	<link rel="stylesheet" href="static/pygments.css" type="text/css" />







	<!--[if lt IE 9]>
	<script src="static/js/html5shiv.min.js"></script>
	<![endif]-->


	<script type="text/javascript" id="documentation_options" data-url_root="./" src="static/documentation_options.js"></script>
	<script src="static/jquery.js"></script>
	<script src="static/underscore.js"></script>
	<script src="static/doctools.js"></script>
	<script src="static/language_data.js"></script>

	<script type="text/javascript" src="static/js/theme.js"></script>


	<link rel="index" title="Index" href="genindex.html" />
	<link rel="search" title="Search" href="search.html" />
	<link rel="next" title="Install SystemDS" href="getting_started/install.html" />
	</head>

	<body class="wy-body-for-nav">


	<div class="wy-grid-for-nav">

	<nav data-toggle="wy-nav-shift" class="wy-nav-side">
	<div class="wy-side-scroll">
	<div class="wy-side-nav-search" >



	<a href="#" class="icon icon-home" alt="Documentation Home"> SystemDS



	</a>







	<div role="search">
	<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
	<input type="text" name="q" placeholder="Search docs" />
	<input type="hidden" name="check_keywords" value="yes" />
	<input type="hidden" name="area" value="default" />
	</form>
	</div>


	</div>


	<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">






	<p class="caption"><span class="caption-text">Getting Started:</span></p>
	<ul>
	<li class="toctree-l1"><a class="reference internal" href="getting_started/install.html">Install SystemDS</a></li>
	<li class="toctree-l1"><a class="reference internal" href="getting_started/simple_examples.html">QuickStart</a></li>
	</ul>
	<p class="caption"><span class="caption-text">Guides</span></p>
	<ul>
	<li class="toctree-l1"><a class="reference internal" href="guide/federated.html">Federated Environment</a></li>
	<li class="toctree-l1"><a class="reference internal" href="guide/algorithms_basics.html">Built-in Algorithms</a></li>
	</ul>
	<p class="caption"><span class="caption-text">API</span></p>
	<ul>
	<li class="toctree-l1"><a class="reference internal" href="api/operator/algorithms.html">Algorithms</a></li>
	<li class="toctree-l1"><a class="reference internal" href="api/context/systemds_context.html">SystemDSContext</a></li>
	<li class="toctree-l1"><a class="reference internal" href="api/matrix/matrix.html">Matrix</a></li>
	<li class="toctree-l1"><a class="reference internal" href="api/matrix/federated.html">Federated</a></li>
	<li class="toctree-l1"><a class="reference internal" href="api/operator/operation_node.html">Operation Node</a></li>
	</ul>
	<p class="caption"><span class="caption-text">Internals API</span></p>
	<ul>
	<li class="toctree-l1"><a class="reference internal" href="api/script_building/dag.html">Dag</a></li>
	<li class="toctree-l1"><a class="reference internal" href="api/script_building/script.html">Script</a></li>
	<li class="toctree-l1"><a class="reference internal" href="api/utils/converters.html">Converters</a></li>
	<li class="toctree-l1"><a class="reference internal" href="api/utils/helpers.html">Helpers</a></li>
	</ul>



	</div>

	</div>
	</nav>

	<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">


	<nav class="wy-nav-top" aria-label="top navigation">

	<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
	<a href="#">SystemDS</a>

	</nav>


	<div class="wy-nav-content">

	<div class="rst-content">

















	<div role="navigation" aria-label="breadcrumbs navigation">

	<ul class="wy-breadcrumbs">

	<li><a href="#" class="icon icon-home"></a> »</li>

	<li>SystemDS</li>


	<li class="wy-breadcrumbs-aside">


	<a href="sources/index.rst.txt" rel="nofollow"> View page source</a>


	</li>

	</ul>


	<hr/>
	</div>
	<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
	<div itemprop="articleBody">

	<div class="section" id="systemds">
	<h1>SystemDS<a class="headerlink" href="#systemds" title="Permalink to this headline">¶</a></h1>
	<p>This package provides a Pythonic interface for working with SystemDS.</p>
	<p>SystemDS is a versatile system for the end-to-end data science lifecycle from data integration,
	cleaning, and feature engineering, over efficient, local and distributed ML model training,
	to deployment and serving.
	To facilitate this, bindings from different languages and different system abstractions provide help for:</p>
	<ol class="arabic simple">
	<li><p>The different tasks of the data-science lifecycle, and</p></li>
	<li><p>users with different expertise.</p></li>
	</ol>
	<p>These high-level scripts are compiled into hybrid execution plans of local, in-memory CPU and GPU operations,
	as well as distributed operations on Apache Spark. In contrast to existing systems - that either
	provide homogeneous tensors or 2D datasets - and in order to serve the entire
	data science lifecycle, the underlying data model are DataTensors, i.e.,
	tensors (multi-dimensional arrays) whose first dimension may have a heterogeneous and nested schema.</p>
	<div class="toctree-wrapper compound">
	</div>
	<div class="toctree-wrapper compound">
	</div>
	<div class="toctree-wrapper compound">
	</div>
	<div class="toctree-wrapper compound">
	</div>
	</div>


	</div>

	</div>
	<footer>

	<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">

	<a href="getting_started/install.html" class="btn btn-neutral float-right" title="Install SystemDS" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>


	</div>


	<hr/>

	<div role="contentinfo">
	<p>

	© Copyright 2020, Apache SystemDS

	</p>
	</div>



	Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a

	<a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a>

	provided by <a href="https://readthedocs.org">Read the Docs</a>.

	</footer>

	</div>
	</div>

	</section>

	</div>


	<script type="text/javascript">
	jQuery(function () {
	SphinxRtdTheme.Navigation.enable(true);
	});
	</script>






	</body>
	</html>