docs/v1.9.1/group__grp__pca__project.html - madlib-site - Git at Google

 <!-- HTML header for doxygen 1.8.4-->
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
 <meta http-equiv="X-UA-Compatible" content="IE=9"/>
 <meta name="generator" content="Doxygen 1.8.10"/>
 <meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
 <title>MADlib: Principal Component Projection</title>
 <link href="tabs.css" rel="stylesheet" type="text/css"/>
 <script type="text/javascript" src="jquery.js"></script>
 <script type="text/javascript" src="dynsections.js"></script>
 <link href="navtree.css" rel="stylesheet" type="text/css"/>
 <script type="text/javascript" src="resize.js"></script>
 <script type="text/javascript" src="navtreedata.js"></script>
 <script type="text/javascript" src="navtree.js"></script>
 <script type="text/javascript">
   $(document).ready(initResizable);
   $(window).load(resizeHeight);
 </script>
 <link href="search/search.css" rel="stylesheet" type="text/css"/>
 <script type="text/javascript" src="search/searchdata.js"></script>
 <script type="text/javascript" src="search/search.js"></script>
 <script type="text/javascript">
   $(document).ready(function() { init_search(); });
 </script>
 <!-- hack in the navigation tree -->
 <script type="text/javascript" src="eigen_navtree_hacks.js"></script>
 <link href="doxygen.css" rel="stylesheet" type="text/css" />
 <link href="madlib_extra.css" rel="stylesheet" type="text/css"/>
 <!-- google analytics -->
 <script>
   (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
   (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
   m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
   })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
   ga('create', 'UA-45382226-1', 'madlib.net');
   ga('send', 'pageview');
 </script>
 </head>
 <body>
 <div id="top"><!-- do not remove this div, it is closed by doxygen! -->
 <div id="titlearea">
 <table cellspacing="0" cellpadding="0">
  <tbody>
  <tr style="height: 56px;">
   <td id="projectlogo"><a href="http://madlib.net"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
   <td style="padding-left: 0.5em;">
    <div id="projectname">
    <span id="projectnumber">1.9.1</span>
    </div>
    <div id="projectbrief">User Documentation for MADlib</div>
   </td>
    <td>        <div id="MSearchBox" class="MSearchBoxInactive">
         <span class="left">
           <img id="MSearchSelect" src="search/mag_sel.png"
                onmouseover="return searchBox.OnSearchSelectShow()"
                onmouseout="return searchBox.OnSearchSelectHide()"
                alt=""/>
           <input type="text" id="MSearchField" value="Search" accesskey="S"
                onfocus="searchBox.OnSearchFieldFocus(true)"
                onblur="searchBox.OnSearchFieldFocus(false)"
                onkeyup="searchBox.OnSearchFieldChange(event)"/>
           </span><span class="right">
             <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
           </span>
         </div>
 </td>
  </tr>
  </tbody>
 </table>
 </div>
 <!-- end header part -->
 <!-- Generated by Doxygen 1.8.10 -->
 <script type="text/javascript">
 var searchBox = new SearchBox("searchBox", "search",false,'Search');
 </script>
 </div><!-- top -->
 <div id="side-nav" class="ui-resizable side-nav-resizable">
   <div id="nav-tree">
     <div id="nav-tree-contents">
       <div id="nav-sync" class="sync"></div>
     </div>
   </div>
   <div id="splitbar" style="-moz-user-select:none;"
        class="ui-resizable-handle">
   </div>
 </div>
 <script type="text/javascript">
 $(document).ready(function(){initNavTree('group__grp__pca__project.html','');});
 </script>
 <div id="doc-content">
 <!-- window showing the filter options -->
 <div id="MSearchSelectWindow"
      onmouseover="return searchBox.OnSearchSelectShow()"
      onmouseout="return searchBox.OnSearchSelectHide()"
      onkeydown="return searchBox.OnSearchSelectKey(event)">
 </div>

 <!-- iframe showing the search results (closed by default) -->
 <div id="MSearchResultsWindow">
 <iframe src="javascript:void(0)" frameborder="0"
         name="MSearchResults" id="MSearchResults">
 </iframe>
 </div>

 <div class="header">
   <div class="headertitle">
 <div class="title">Principal Component Projection<div class="ingroups"><a class="el" href="group__grp__datatrans.html">Data Types and Transformations</a> &raquo; <a class="el" href="group__grp__pca.html">Dimensionality Reduction</a></div></div>  </div>
 </div><!--header-->
 <div class="contents">
 <div class="toc"><b>Contents</b> </p><ul>
 <li class="level1">
 <a href="#project">Projection Function</a> </li>
 <li class="level1">
 <a href="#examples">Examples</a> </li>
 <li class="level1">
 <a href="#notes">Notes</a> </li>
 <li class="level1">
 <a href="#background">Technical Background</a> </li>
 <li class="level1">
 <a href="#related">Related Topics</a> </li>
 </ul>
 </div><p>Principal component projection is a mathematical procedure that projects high dimensional data onto a lower dimensional space. This lower dimensional space is defined by the <img class="formulaInl" alt="$ k $" src="form_97.png"/> principal components with the highest variance in the training data. More details on the mathematics of PCA can be found in <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a> and some details about the principal component projection calculations can be found in the <a class="el" href="group__grp__mlogreg.html#background">Technical Background</a>.</p>
 <p><a class="anchor" id="project"></a></p><dl class="section user"><dt>Projection Function</dt><dd>The projection functions have the following formats: <pre class="syntax">
 madlib.pca_project( source_table,
                     pc_table,
                     out_table,
                     row_id,
                     residual_table,
                     result_summary_table
                   )
 </pre> and <pre class="syntax">
 madlib.pca_sparse_project( source_table,
                            pc_table,
                            out_table,
                            row_id,
                            col_id,
                            val_id,
                            row_dim,
                            col_dim,
                            residual_table,
                            result_summary_table
                          )
 </pre></dd></dl>
 <dl class="section user"><dt>Arguments</dt><dd><dl class="arglist">
 <dt>source_table </dt>
 <dd><p class="startdd">TEXT. Source table name. Identical to <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a>, the input data matrix should have <img class="formulaInl" alt="$ N $" src="form_219.png"/> rows and <img class="formulaInl" alt="$ M $" src="form_174.png"/> columns, where <img class="formulaInl" alt="$ N $" src="form_219.png"/> is the number of data points, and <img class="formulaInl" alt="$ M $" src="form_174.png"/> is the number of features for each data point.</p>
 <p>The input table for <em> pca_project </em> is expected to be in the one of the two standard MADlib dense matrix formats, and the sparse input table for <em> pca_sparse_project </em> should be in the standard MADlib sparse matrix format. These formats are described in the documentation for <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a>.</p>
 <p class="enddd"></p>
 </dd>
 <dt>pc_table </dt>
 <dd><p class="startdd">TEXT. Table name for the table containing principal components. </p>
 <p class="enddd"></p>
 </dd>
 <dt>out_table </dt>
 <dd><p class="startdd">TEXT. Name of the table that will contain the low-dimensional representation of the input data.</p>
 <p>The <em>out_table</em> encodes a dense matrix with the projection onto the principal components. The table has the following columns:</p>
 <table  class="output">
 <tr>
 <th>row_id </th><td>Row id of the output matrix.  </td></tr>
 <tr>
 <th>row_vec </th><td>A vector containing elements in the row of the matrix.  </td></tr>
 </table>
 <p class="enddd"></p>
 </dd>
 <dt>row_id </dt>
 <dd><p class="startdd">TEXT. Column name containing the row IDs in the input source table. The column needs to be of type that can be cast to INT and should only contain values between 1 and <em>N</em>. For dense matrix format, it should contain all continguous integers from 1 to <em>N</em>.</p>
 <p class="enddd"></p>
 </dd>
 <dt>col_id </dt>
 <dd><p class="startdd">TEXT. Column name of containing the col IDS in sparse matrix representation (sparse matrices only). The column should be of type that can be cast to INT and contain values between 1 and <em>M</em>.</p>
 <p class="enddd"></p>
 </dd>
 <dt>val_id </dt>
 <dd><p class="startdd">TEXT. Name of 'val_id' column in sparse matrix representation (sparse matrices only). </p>
 <p class="enddd"></p>
 </dd>
 <dt>row_dim </dt>
 <dd><p class="startdd">INTEGER. The number of rows in the sparse matrix (sparse matrices only). </p>
 <p class="enddd"></p>
 </dd>
 <dt>col_dim </dt>
 <dd><p class="startdd">INTEGER. The number of columns in the sparse matrix (sparse matrices only). </p>
 <p class="enddd"></p>
 </dd>
 <dt>residual_table (optional) </dt>
 <dd><p class="startdd">TEXT, default: NULL. Name of the optional residual table.</p>
 <p>The <em>residual_table</em> encodes a dense residual matrix. The table has the following columns:</p>
 <table  class="output">
 <tr>
 <th>row_id </th><td>Row id of the output matrix.  </td></tr>
 <tr>
 <th>row_vec </th><td>A vector containing elements in the row of the residual matrix.  </td></tr>
 </table>
 <p class="enddd"></p>
 </dd>
 <dt>result_summary_table (optional) </dt>
 <dd><p class="startdd">TEXT, default: NULL. Name of the optional summary table.</p>
 <p class="enddd">The <em>result_summary_table</em> contains information about the performance time of the PCA projection. The table has the following columns: </p><table  class="output">
 <tr>
 <th>exec_time </th><td>Wall clock time (ms) of the function.  </td></tr>
 <tr>
 <th>residual_norm </th><td>Absolute error of the residuals.  </td></tr>
 <tr>
 <th>relative_residual_norm </th><td>Relative error of the residuals.  </td></tr>
 </table>
 </dd>
 </dl>
 </dd></dl>
 <p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd><ol type="1">
 <li>View online help for the PCA projection function. <pre class="example">
 SELECT madlib.pca_project();
 </pre></li>
 <li>Create the sample data. <pre class="example">
 DROP TABLE IF EXISTS mat;
 CREATE TABLE mat (
     row_id integer,
     row_vec double precision[]
 );
 INSERT INTO mat VALUES
 (1, ARRAY[4,7,5]),
 (2, ARRAY[1,2,5]),
 (3, ARRAY[7,4,4]),
 (4, ARRAY[9,2,4]),
 (5, ARRAY[8,5,7]),
 (6, ARRAY[0,5,5]);
 </pre></li>
 <li>Run the PCA function and keep only the top two PCs: <pre class="example">
 DROP TABLE IF EXISTS result_table;
 SELECT pca_train ( 'mat',
                    'result_table',
                    'row_id',
                    2
                  );
 </pre></li>
 <li>Project the original data into a low-dimensional representation. <pre class="example">
 DROP TABLE IF EXISTS residual_table, result_summary_table, out_table;
 SELECT pca_project( 'mat',
                     'result_table',
                     'out_table'
                     'row_id',
                     'residual_table',
                     'result_summary_table'
                   );
 </pre></li>
 <li>Check the error in the projection. <pre class="example">
 SELECT * FROM result_summary_table;
 </pre> Result: <pre class="result">
    exec_time   | residual_norm | relative_residual_norm
 ---------------+---------------+------------------------
  5685.40501595 | 2.19726255664 |         0.099262204234
 </pre></li>
 </ol>
 </dd></dl>
 <p><a class="anchor" id="notes"></a></p><dl class="section user"><dt>Notes</dt><dd><ul>
 <li>This function is intended to operate on the principal component tables generated by <em> pca_train </em> or <em> pca_sparse_train</em>. The MADlib PCA functions generate a table containing the column-means in addition to a table containing the principal components. If this table is not found by the MADlib projection function, it will trigger an error. As long the principal component tables are created with MADlib functions, then the column-means table will be automatically found by the MADlib projection functions.</li>
 <li>Because of the centering step in PCA projection (see "Technical Background"), sparse matrices almost always become dense during the projection process. Thus, this implementation automatically densifies sparse matrix input, and there should be no expected performance improvement in using sparse matrix input over dense matrix input.</li>
 <li>Table names can be optionally schema qualified (current_schemas() is searched if a schema name is not provided) and all table and column names should follow case-sensitivity and quoting rules per the database. (For instance, 'mytable' and 'MyTable' both resolve to the same entity, i.e. 'mytable'. If mixed-case or multi-byte characters are desired for entity names then the string should be double-quoted; in this case the input would be '"MyTable"').</li>
 </ul>
 </dd></dl>
 <p><a class="anchor" id="background"></a></p><dl class="section user"><dt>Technical Background</dt><dd></dd></dl>
 <p>Given a table containing some principal components <img class="formulaInl" alt="$ \boldsymbol P $" src="form_230.png"/> and some input data <img class="formulaInl" alt="$ \boldsymbol X $" src="form_220.png"/>, the low-dimensional representation <img class="formulaInl" alt="$ {\boldsymbol X}' $" src="form_231.png"/> is computed as </p><p class="formulaDsp">
 <img class="formulaDsp" alt="\begin{align*} {\boldsymbol {\hat{X}}} &amp; = {\boldsymbol X} - \vec{e} \hat{x}^T \\ {\boldsymbol X}' &amp; = {\boldsymbol {\hat {X}}} {\boldsymbol P}. \end{align*}" src="form_232.png"/>
 </p>
 <p> where <img class="formulaInl" alt="$\hat{x} $" src="form_233.png"/> is the column means of <img class="formulaInl" alt="$ \boldsymbol X $" src="form_220.png"/> and <img class="formulaInl" alt="$ \vec{e} $" src="form_225.png"/> is the vector of all ones. This step is equivalent to centering the data around the origin.</p>
 <p>The residual table <img class="formulaInl" alt="$ \boldsymbol R $" src="form_234.png"/> is a measure of how well the low-dimensional representation approximates the true input data, and is computed as </p><p class="formulaDsp">
 <img class="formulaDsp" alt="\[ {\boldsymbol R} = {\boldsymbol {\hat{X}}} - {\boldsymbol X}' {\boldsymbol P}^T. \]" src="form_235.png"/>
 </p>
 <p> A residual matrix with entries mostly close to zero indicates a good representation.</p>
 <p>The residual norm <img class="formulaInl" alt="$ r $" src="form_236.png"/> is simply </p><p class="formulaDsp">
 <img class="formulaDsp" alt="\[ r = \|{\boldsymbol R}\|_F \]" src="form_237.png"/>
 </p>
 <p> where <img class="formulaInl" alt="$ \|\cdot\|_F $" src="form_238.png"/> is the Frobenius norm. The relative residual norm <img class="formulaInl" alt="$ r' $" src="form_239.png"/> is </p><p class="formulaDsp">
 <img class="formulaDsp" alt="\[ r' = \frac{ \|{\boldsymbol R}\|_F }{\|{\boldsymbol X}\|_F } \]" src="form_240.png"/>
 </p>
 <p><a class="anchor" id="related"></a></p><dl class="section user"><dt>Related Topics</dt><dd>File <a class="el" href="pca__project_8sql__in.html" title="Principal Component Analysis Projection. ">pca_project.sql_in</a> documenting the SQL functions</dd></dl>
 <p><a class="el" href="group__grp__pca__train.html">Principal Component Analysis</a> </p>
 </div><!-- contents -->
 </div><!-- doc-content -->
 <!-- start footer part -->
 <div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
   <ul>
     <li class="footer">Generated on Tue Sep 20 2016 11:27:01 for MADlib by
     <a href="http://www.doxygen.org/index.html">
     <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
   </ul>
 </div>
 </body>
 </html>
	<!-- HTML header for doxygen 1.8.4-->
	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
	<html xmlns="http://www.w3.org/1999/xhtml">
	<head>
	<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
	<meta http-equiv="X-UA-Compatible" content="IE=9"/>
	<meta name="generator" content="Doxygen 1.8.10"/>
	<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
	<title>MADlib: Principal Component Projection</title>
	<link href="tabs.css" rel="stylesheet" type="text/css"/>
	<script type="text/javascript" src="jquery.js"></script>
	<script type="text/javascript" src="dynsections.js"></script>
	<link href="navtree.css" rel="stylesheet" type="text/css"/>
	<script type="text/javascript" src="resize.js"></script>
	<script type="text/javascript" src="navtreedata.js"></script>
	<script type="text/javascript" src="navtree.js"></script>
	<script type="text/javascript">
	$(document).ready(initResizable);
	$(window).load(resizeHeight);
	</script>
	<link href="search/search.css" rel="stylesheet" type="text/css"/>
	<script type="text/javascript" src="search/searchdata.js"></script>
	<script type="text/javascript" src="search/search.js"></script>
	<script type="text/javascript">
	$(document).ready(function() { init_search(); });
	</script>
	<!-- hack in the navigation tree -->
	<script type="text/javascript" src="eigen_navtree_hacks.js"></script>
	<link href="doxygen.css" rel="stylesheet" type="text/css" />
	<link href="madlib_extra.css" rel="stylesheet" type="text/css"/>
	<!-- google analytics -->
	<script>
	(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]\|\|function(){
	(i[r].q=i[r].q\|\|[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
	m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
	})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
	ga('create', 'UA-45382226-1', 'madlib.net');
	ga('send', 'pageview');
	</script>
	</head>
	<body>
	<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
	<div id="titlearea">
	<table cellspacing="0" cellpadding="0">
	<tbody>
	<tr style="height: 56px;">
	<td id="projectlogo"><a href="http://madlib.net"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
	<td style="padding-left: 0.5em;">
	<div id="projectname">
	<span id="projectnumber">1.9.1</span>
	</div>
	<div id="projectbrief">User Documentation for MADlib</div>
	</td>
	<td> <div id="MSearchBox" class="MSearchBoxInactive">
	<span class="left">
	<img id="MSearchSelect" src="search/mag_sel.png"
	onmouseover="return searchBox.OnSearchSelectShow()"
	onmouseout="return searchBox.OnSearchSelectHide()"
	alt=""/>
	<input type="text" id="MSearchField" value="Search" accesskey="S"
	onfocus="searchBox.OnSearchFieldFocus(true)"
	onblur="searchBox.OnSearchFieldFocus(false)"
	onkeyup="searchBox.OnSearchFieldChange(event)"/>
	</span><span class="right">
	<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
	</span>
	</div>
	</td>
	</tr>
	</tbody>
	</table>
	</div>
	<!-- end header part -->
	<!-- Generated by Doxygen 1.8.10 -->
	<script type="text/javascript">
	var searchBox = new SearchBox("searchBox", "search",false,'Search');
	</script>
	</div><!-- top -->
	<div id="side-nav" class="ui-resizable side-nav-resizable">
	<div id="nav-tree">
	<div id="nav-tree-contents">
	<div id="nav-sync" class="sync"></div>
	</div>
	</div>
	<div id="splitbar" style="-moz-user-select:none;"
	class="ui-resizable-handle">
	</div>
	</div>
	<script type="text/javascript">
	$(document).ready(function(){initNavTree('group__grp__pca__project.html','');});
	</script>
	<div id="doc-content">
	<!-- window showing the filter options -->
	<div id="MSearchSelectWindow"
	onmouseover="return searchBox.OnSearchSelectShow()"
	onmouseout="return searchBox.OnSearchSelectHide()"
	onkeydown="return searchBox.OnSearchSelectKey(event)">
	</div>

	<!-- iframe showing the search results (closed by default) -->
	<div id="MSearchResultsWindow">
	<iframe src="javascript:void(0)" frameborder="0"
	name="MSearchResults" id="MSearchResults">
	</iframe>
	</div>

	<div class="header">
	<div class="headertitle">
	<div class="title">Principal Component Projection<div class="ingroups"><a class="el" href="group__grp__datatrans.html">Data Types and Transformations</a> » <a class="el" href="group__grp__pca.html">Dimensionality Reduction</a></div></div> </div>
	</div><!--header-->
	<div class="contents">
	<div class="toc"><b>Contents</b> </p><ul>
	<li class="level1">
	<a href="#project">Projection Function</a> </li>
	<li class="level1">
	<a href="#examples">Examples</a> </li>
	<li class="level1">
	<a href="#notes">Notes</a> </li>
	<li class="level1">
	<a href="#background">Technical Background</a> </li>
	<li class="level1">
	<a href="#related">Related Topics</a> </li>
	</ul>
	</div><p>Principal component projection is a mathematical procedure that projects high dimensional data onto a lower dimensional space. This lower dimensional space is defined by the <img class="formulaInl" alt="$ k $" src="form_97.png"/> principal components with the highest variance in the training data. More details on the mathematics of PCA can be found in <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a> and some details about the principal component projection calculations can be found in the <a class="el" href="group__grp__mlogreg.html#background">Technical Background</a>.</p>
	<p><a class="anchor" id="project"></a></p><dl class="section user"><dt>Projection Function</dt><dd>The projection functions have the following formats: <pre class="syntax">
	madlib.pca_project( source_table,
	pc_table,
	out_table,
	row_id,
	residual_table,
	result_summary_table
	)
	</pre> and <pre class="syntax">
	madlib.pca_sparse_project( source_table,
	pc_table,
	out_table,
	row_id,
	col_id,
	val_id,
	row_dim,
	col_dim,
	residual_table,
	result_summary_table
	)
	</pre></dd></dl>
	<dl class="section user"><dt>Arguments</dt><dd><dl class="arglist">
	<dt>source_table </dt>
	<dd><p class="startdd">TEXT. Source table name. Identical to <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a>, the input data matrix should have <img class="formulaInl" alt="$ N $" src="form_219.png"/> rows and <img class="formulaInl" alt="$ M $" src="form_174.png"/> columns, where <img class="formulaInl" alt="$ N $" src="form_219.png"/> is the number of data points, and <img class="formulaInl" alt="$ M $" src="form_174.png"/> is the number of features for each data point.</p>
	<p>The input table for <em> pca_project </em> is expected to be in the one of the two standard MADlib dense matrix formats, and the sparse input table for <em> pca_sparse_project </em> should be in the standard MADlib sparse matrix format. These formats are described in the documentation for <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a>.</p>
	<p class="enddd"></p>
	</dd>
	<dt>pc_table </dt>
	<dd><p class="startdd">TEXT. Table name for the table containing principal components. </p>
	<p class="enddd"></p>
	</dd>
	<dt>out_table </dt>
	<dd><p class="startdd">TEXT. Name of the table that will contain the low-dimensional representation of the input data.</p>
	<p>The <em>out_table</em> encodes a dense matrix with the projection onto the principal components. The table has the following columns:</p>
	<table class="output">
	<tr>
	<th>row_id </th><td>Row id of the output matrix. </td></tr>
	<tr>
	<th>row_vec </th><td>A vector containing elements in the row of the matrix. </td></tr>
	</table>
	<p class="enddd"></p>
	</dd>
	<dt>row_id </dt>
	<dd><p class="startdd">TEXT. Column name containing the row IDs in the input source table. The column needs to be of type that can be cast to INT and should only contain values between 1 and <em>N</em>. For dense matrix format, it should contain all continguous integers from 1 to <em>N</em>.</p>
	<p class="enddd"></p>
	</dd>
	<dt>col_id </dt>
	<dd><p class="startdd">TEXT. Column name of containing the col IDS in sparse matrix representation (sparse matrices only). The column should be of type that can be cast to INT and contain values between 1 and <em>M</em>.</p>
	<p class="enddd"></p>
	</dd>
	<dt>val_id </dt>
	<dd><p class="startdd">TEXT. Name of 'val_id' column in sparse matrix representation (sparse matrices only). </p>
	<p class="enddd"></p>
	</dd>
	<dt>row_dim </dt>
	<dd><p class="startdd">INTEGER. The number of rows in the sparse matrix (sparse matrices only). </p>
	<p class="enddd"></p>
	</dd>
	<dt>col_dim </dt>
	<dd><p class="startdd">INTEGER. The number of columns in the sparse matrix (sparse matrices only). </p>
	<p class="enddd"></p>
	</dd>
	<dt>residual_table (optional) </dt>
	<dd><p class="startdd">TEXT, default: NULL. Name of the optional residual table.</p>
	<p>The <em>residual_table</em> encodes a dense residual matrix. The table has the following columns:</p>
	<table class="output">
	<tr>
	<th>row_id </th><td>Row id of the output matrix. </td></tr>
	<tr>
	<th>row_vec </th><td>A vector containing elements in the row of the residual matrix. </td></tr>
	</table>
	<p class="enddd"></p>
	</dd>
	<dt>result_summary_table (optional) </dt>
	<dd><p class="startdd">TEXT, default: NULL. Name of the optional summary table.</p>
	<p class="enddd">The <em>result_summary_table</em> contains information about the performance time of the PCA projection. The table has the following columns: </p><table class="output">
	<tr>
	<th>exec_time </th><td>Wall clock time (ms) of the function. </td></tr>
	<tr>
	<th>residual_norm </th><td>Absolute error of the residuals. </td></tr>
	<tr>
	<th>relative_residual_norm </th><td>Relative error of the residuals. </td></tr>
	</table>
	</dd>
	</dl>
	</dd></dl>
	<p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd><ol type="1">
	<li>View online help for the PCA projection function. <pre class="example">
	SELECT madlib.pca_project();
	</pre></li>
	<li>Create the sample data. <pre class="example">
	DROP TABLE IF EXISTS mat;
	CREATE TABLE mat (
	row_id integer,
	row_vec double precision[]
	);
	INSERT INTO mat VALUES
	(1, ARRAY[4,7,5]),
	(2, ARRAY[1,2,5]),
	(3, ARRAY[7,4,4]),
	(4, ARRAY[9,2,4]),
	(5, ARRAY[8,5,7]),
	(6, ARRAY[0,5,5]);
	</pre></li>
	<li>Run the PCA function and keep only the top two PCs: <pre class="example">
	DROP TABLE IF EXISTS result_table;
	SELECT pca_train ( 'mat',
	'result_table',
	'row_id',
	2
	);
	</pre></li>
	<li>Project the original data into a low-dimensional representation. <pre class="example">
	DROP TABLE IF EXISTS residual_table, result_summary_table, out_table;
	SELECT pca_project( 'mat',
	'result_table',
	'out_table'
	'row_id',
	'residual_table',
	'result_summary_table'
	);
	</pre></li>
	<li>Check the error in the projection. <pre class="example">
	SELECT * FROM result_summary_table;
	</pre> Result: <pre class="result">
	exec_time \| residual_norm \| relative_residual_norm
	---------------+---------------+------------------------
	5685.40501595 \| 2.19726255664 \| 0.099262204234
	</pre></li>
	</ol>
	</dd></dl>
	<p><a class="anchor" id="notes"></a></p><dl class="section user"><dt>Notes</dt><dd><ul>
	<li>This function is intended to operate on the principal component tables generated by <em> pca_train </em> or <em> pca_sparse_train</em>. The MADlib PCA functions generate a table containing the column-means in addition to a table containing the principal components. If this table is not found by the MADlib projection function, it will trigger an error. As long the principal component tables are created with MADlib functions, then the column-means table will be automatically found by the MADlib projection functions.</li>
	<li>Because of the centering step in PCA projection (see "Technical Background"), sparse matrices almost always become dense during the projection process. Thus, this implementation automatically densifies sparse matrix input, and there should be no expected performance improvement in using sparse matrix input over dense matrix input.</li>
	<li>Table names can be optionally schema qualified (current_schemas() is searched if a schema name is not provided) and all table and column names should follow case-sensitivity and quoting rules per the database. (For instance, 'mytable' and 'MyTable' both resolve to the same entity, i.e. 'mytable'. If mixed-case or multi-byte characters are desired for entity names then the string should be double-quoted; in this case the input would be '"MyTable"').</li>
	</ul>
	</dd></dl>
	<p><a class="anchor" id="background"></a></p><dl class="section user"><dt>Technical Background</dt><dd></dd></dl>
	<p>Given a table containing some principal components <img class="formulaInl" alt="$ \boldsymbol P $" src="form_230.png"/> and some input data <img class="formulaInl" alt="$ \boldsymbol X $" src="form_220.png"/>, the low-dimensional representation <img class="formulaInl" alt="$ {\boldsymbol X}' $" src="form_231.png"/> is computed as </p><p class="formulaDsp">
	<img class="formulaDsp" alt="\begin{align} {\boldsymbol {\hat{X}}} & = {\boldsymbol X} - \vec{e} \hat{x}^T \\ {\boldsymbol X}' & = {\boldsymbol {\hat {X}}} {\boldsymbol P}. \end{align}" src="form_232.png"/>
	</p>
	<p> where <img class="formulaInl" alt="$\hat{x} $" src="form_233.png"/> is the column means of <img class="formulaInl" alt="$ \boldsymbol X $" src="form_220.png"/> and <img class="formulaInl" alt="$ \vec{e} $" src="form_225.png"/> is the vector of all ones. This step is equivalent to centering the data around the origin.</p>
	<p>The residual table <img class="formulaInl" alt="$ \boldsymbol R $" src="form_234.png"/> is a measure of how well the low-dimensional representation approximates the true input data, and is computed as </p><p class="formulaDsp">
	<img class="formulaDsp" alt="\[ {\boldsymbol R} = {\boldsymbol {\hat{X}}} - {\boldsymbol X}' {\boldsymbol P}^T. \]" src="form_235.png"/>
	</p>
	<p> A residual matrix with entries mostly close to zero indicates a good representation.</p>
	<p>The residual norm <img class="formulaInl" alt="$ r $" src="form_236.png"/> is simply </p><p class="formulaDsp">
	<img class="formulaDsp" alt="\[ r = \\|{\boldsymbol R}\\|_F \]" src="form_237.png"/>
	</p>
	<p> where <img class="formulaInl" alt="$ \\|\cdot\\|_F $" src="form_238.png"/> is the Frobenius norm. The relative residual norm <img class="formulaInl" alt="$ r' $" src="form_239.png"/> is </p><p class="formulaDsp">
	<img class="formulaDsp" alt="\[ r' = \frac{ \\|{\boldsymbol R}\\|_F }{\\|{\boldsymbol X}\\|_F } \]" src="form_240.png"/>
	</p>
	<p><a class="anchor" id="related"></a></p><dl class="section user"><dt>Related Topics</dt><dd>File <a class="el" href="pca__project_8sql__in.html" title="Principal Component Analysis Projection. ">pca_project.sql_in</a> documenting the SQL functions</dd></dl>
	<p><a class="el" href="group__grp__pca__train.html">Principal Component Analysis</a> </p>
	</div><!-- contents -->
	</div><!-- doc-content -->
	<!-- start footer part -->
	<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
	<ul>
	<li class="footer">Generated on Tue Sep 20 2016 11:27:01 for MADlib by
	<a href="http://www.doxygen.org/index.html">
	<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
	</ul>
	</div>
	</body>
	</html>