blob: 455a4235956c51cad672c186b19e93abb6922d08 [file] [log] [blame]
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<!-- NewPage -->
<html lang="en">
<head>
<!-- Generated by javadoc -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>UnifiedMemoryManager (SystemDS 2.1.0-SNAPSHOT API)</title>
<link rel="stylesheet" type="text/css" href="../../../../../../stylesheet.css" title="Style">
<script type="text/javascript" src="../../../../../../script.js"></script>
</head>
<body>
<script type="text/javascript"><!--
try {
if (location.href.indexOf('is-external=true') == -1) {
parent.document.title="UnifiedMemoryManager (SystemDS 2.1.0-SNAPSHOT API)";
}
}
catch(err) {
}
//-->
var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10};
var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]};
var altColor = "altColor";
var rowColor = "rowColor";
var tableTab = "tableTab";
var activeTableTab = "activeTableTab";
</script>
<noscript>
<div>JavaScript is disabled on your browser.</div>
</noscript>
<!-- ========= START OF TOP NAVBAR ======= -->
<div class="topNav"><a name="navbar.top">
<!-- -->
</a>
<div class="skipNav"><a href="#skip.navbar.top" title="Skip navigation links">Skip navigation links</a></div>
<a name="navbar.top.firstrow">
<!-- -->
</a>
<ul class="navList" title="Navigation">
<li><a href="../../../../../../overview-summary.html">Overview</a></li>
<li><a href="package-summary.html">Package</a></li>
<li class="navBarCell1Rev">Class</li>
<li><a href="class-use/UnifiedMemoryManager.html">Use</a></li>
<li><a href="package-tree.html">Tree</a></li>
<li><a href="../../../../../../deprecated-list.html">Deprecated</a></li>
<li><a href="../../../../../../index-all.html">Index</a></li>
<li><a href="../../../../../../help-doc.html">Help</a></li>
</ul>
</div>
<div class="subNav">
<ul class="navList">
<li><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/TensorObject.html" title="class in org.apache.sysds.runtime.controlprogram.caching"><span class="typeNameLink">Prev&nbsp;Class</span></a></li>
<li>Next&nbsp;Class</li>
</ul>
<ul class="navList">
<li><a href="../../../../../../index.html?org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html" target="_top">Frames</a></li>
<li><a href="UnifiedMemoryManager.html" target="_top">No&nbsp;Frames</a></li>
</ul>
<ul class="navList" id="allclasses_navbar_top">
<li><a href="../../../../../../allclasses-noframe.html">All&nbsp;Classes</a></li>
</ul>
<div>
<script type="text/javascript"><!--
allClassesLink = document.getElementById("allclasses_navbar_top");
if(window==top) {
allClassesLink.style.display = "block";
}
else {
allClassesLink.style.display = "none";
}
//-->
</script>
</div>
<div>
<ul class="subNavList">
<li>Summary:&nbsp;</li>
<li>Nested&nbsp;|&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.summary">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.summary">Method</a></li>
</ul>
<ul class="subNavList">
<li>Detail:&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.detail">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.detail">Method</a></li>
</ul>
</div>
<a name="skip.navbar.top">
<!-- -->
</a></div>
<!-- ========= END OF TOP NAVBAR ========= -->
<!-- ======== START OF CLASS DATA ======== -->
<div class="header">
<div class="subTitle">org.apache.sysds.runtime.controlprogram.caching</div>
<h2 title="Class UnifiedMemoryManager" class="title">Class UnifiedMemoryManager</h2>
</div>
<div class="contentContainer">
<ul class="inheritance">
<li><a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">java.lang.Object</a></li>
<li>
<ul class="inheritance">
<li>org.apache.sysds.runtime.controlprogram.caching.UnifiedMemoryManager</li>
</ul>
</li>
</ul>
<div class="description">
<ul class="blockList">
<li class="blockList">
<hr>
<br>
<pre>public class <span class="typeNameLabel">UnifiedMemoryManager</span>
extends <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">Object</a></pre>
<div class="block">Unified Memory Manager - Initial Design
Motivation:
The Unified Memory Manager, henceforth UMM, will act as a central manager of in-memory
matrix (uncompressed and compressed), frame, and tensor blocks within SystemDS control
program. So far, operation memory (70%) and buffer pool memory (15%, LazyWriteBuffer)
are managed independently, which causes unnecessary evictions. New components like the
LineageCache also use and manage statically provisioned memory areas. Ultimately, the
UMM aims to eliminate these shortcomings by providing a central, potentially thread-local,
memory management.
Memory Areas:
Initially, the UMM only handles CacheBlock objects (e.g., MatrixBlock, FrameBlock, and
TensorBlock), and manages two memory areas:
(1) operation memory (pinned cache blocks and reserved memory) and
(2) dirty objects (dirty cache blocks that need to be written to local FS before eviction)
The UMM is configured with a capacity (absolute size in byte). Relative to this capacity,
the operations and buffer pool memory areas each will have a min and max amount of memory
they can occupy, meaning that the boundary for the areas can shift dynamically depending
on the current load. Most importantly, though, dirty objects must not be counted twice
when pinning such an object for an operation. The min/max constraints are not exposed but
configured internally. An good starting point are the following constraints (relative to
JVM max heap size):
___________________________
| operations | 0% | 70% | (pin requests always accepted)
| buffer pool | 15% | 85% | (eviction on demand)
Object Lifecycle:
The UMM will also need to keep track of the current state of individual cache blocks, for
which it will have a few member variables. A queue similar to the current EvictionQueue is
used to add/remove entries with LRU as its eviction policy. In general, there are three
properties of object status to consider:
(1) Non-dirty/dirty: non-dirty objects have a representation on HDFS or can be recomputed
from lineage trace (e.g., rand/seq outputs), while dirty objects need to be preserved.
(2) FS Persisted: on eviction, dirty objects need to be written to local file system.
As long the local file representation exist, dirty objects can simply be dropped.
(3) Pinned/unpinned: For operations, objects are pinned into memory to guard against
eviction. All pin requests have to be accepted, and once a non-dirty object is released
(unpinned) it can be dropped without persisting it to local FS.
Thread-safeness:
Initially, the UMM will be used in an instance-based manner. For global visibility and
use in parallel for loops, the UMM would need to provide a static, synchronized API, but
this constitutes a source of severe contention. In the future, we will consider a design
with thread-local UMMs for the individual parfor workers.
Testing:
The UMM will be developed bottom up, and thus initially tested via component tests for
evaluating the eviction behavior for sequences of API requests.</div>
</li>
</ul>
</div>
<div class="summary">
<ul class="blockList">
<li class="blockList">
<!-- ======== CONSTRUCTOR SUMMARY ======== -->
<ul class="blockList">
<li class="blockList"><a name="constructor.summary">
<!-- -->
</a>
<h3>Constructor Summary</h3>
<table class="memberSummary" border="0" cellpadding="3" cellspacing="0" summary="Constructor Summary table, listing constructors, and an explanation">
<caption><span>Constructors</span><span class="tabEnd">&nbsp;</span></caption>
<tr>
<th class="colOne" scope="col">Constructor and Description</th>
</tr>
<tr class="altColor">
<td class="colOne"><code><span class="memberNameLink"><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html#UnifiedMemoryManager-long-">UnifiedMemoryManager</a></span>(long&nbsp;capacity)</code>&nbsp;</td>
</tr>
</table>
</li>
</ul>
<!-- ========== METHOD SUMMARY =========== -->
<ul class="blockList">
<li class="blockList"><a name="method.summary">
<!-- -->
</a>
<h3>Method Summary</h3>
<table class="memberSummary" border="0" cellpadding="3" cellspacing="0" summary="Method Summary table, listing methods, and an explanation">
<caption><span id="t0" class="activeTableTab"><span>All Methods</span><span class="tabEnd">&nbsp;</span></span><span id="t2" class="tableTab"><span><a href="javascript:show(2);">Instance Methods</a></span><span class="tabEnd">&nbsp;</span></span><span id="t4" class="tableTab"><span><a href="javascript:show(8);">Concrete Methods</a></span><span class="tabEnd">&nbsp;</span></span></caption>
<tr>
<th class="colFirst" scope="col">Modifier and Type</th>
<th class="colLast" scope="col">Method and Description</th>
</tr>
<tr id="i0" class="altColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html#delete-java.lang.String-">delete</a></span>(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key)</code>
<div class="block">Removes a cache block associated with the given key from all memory
areas, and deletes evicted representations (files in local FS).</div>
</td>
</tr>
<tr id="i1" class="rowColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html#deleteAll--">deleteAll</a></span>()</code>
<div class="block">Removes all cache blocks from all memory areas and deletes all evicted
representations (files in local FS).</div>
</td>
</tr>
<tr id="i2" class="altColor">
<td class="colFirst"><code><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/CacheBlock.html" title="interface in org.apache.sysds.runtime.controlprogram.caching">CacheBlock</a></code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html#pin-java.lang.String-org.apache.sysds.runtime.controlprogram.caching.CacheBlock-boolean-">pin</a></span>(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key,
<a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/CacheBlock.html" title="interface in org.apache.sysds.runtime.controlprogram.caching">CacheBlock</a>&nbsp;block,
boolean&nbsp;dirty)</code>
<div class="block">Pins a cache block into operation memory.</div>
</td>
</tr>
<tr id="i3" class="rowColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html#pin-java.lang.String-long-boolean-">pin</a></span>(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key,
long&nbsp;size,
boolean&nbsp;dirty)</code>
<div class="block">Pins a virtual cache block into operation memory, by making a size reservation.</div>
</td>
</tr>
<tr id="i4" class="altColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html#unpin-java.lang.String-">unpin</a></span>(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key)</code>
<div class="block">Unpins (releases) a cache block from operation memory.</div>
</td>
</tr>
<tr id="i5" class="rowColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html#unpin-java.lang.String-org.apache.sysds.runtime.controlprogram.caching.CacheBlock-">unpin</a></span>(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key,
<a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/CacheBlock.html" title="interface in org.apache.sysds.runtime.controlprogram.caching">CacheBlock</a>&nbsp;block)</code>
<div class="block">Unpins (releases) a cache block from operation memory.</div>
</td>
</tr>
</table>
<ul class="blockList">
<li class="blockList"><a name="methods.inherited.from.class.java.lang.Object">
<!-- -->
</a>
<h3>Methods inherited from class&nbsp;java.lang.<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">Object</a></h3>
<code><a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-" title="class or interface in java.lang">equals</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#getClass--" title="class or interface in java.lang">getClass</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#hashCode--" title="class or interface in java.lang">hashCode</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#notify--" title="class or interface in java.lang">notify</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#notifyAll--" title="class or interface in java.lang">notifyAll</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#toString--" title="class or interface in java.lang">toString</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#wait--" title="class or interface in java.lang">wait</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#wait-long-" title="class or interface in java.lang">wait</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#wait-long-int-" title="class or interface in java.lang">wait</a></code></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<div class="details">
<ul class="blockList">
<li class="blockList">
<!-- ========= CONSTRUCTOR DETAIL ======== -->
<ul class="blockList">
<li class="blockList"><a name="constructor.detail">
<!-- -->
</a>
<h3>Constructor Detail</h3>
<a name="UnifiedMemoryManager-long-">
<!-- -->
</a>
<ul class="blockListLast">
<li class="blockList">
<h4>UnifiedMemoryManager</h4>
<pre>public&nbsp;UnifiedMemoryManager(long&nbsp;capacity)</pre>
</li>
</ul>
</li>
</ul>
<!-- ============ METHOD DETAIL ========== -->
<ul class="blockList">
<li class="blockList"><a name="method.detail">
<!-- -->
</a>
<h3>Method Detail</h3>
<a name="pin-java.lang.String-org.apache.sysds.runtime.controlprogram.caching.CacheBlock-boolean-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>pin</h4>
<pre>public&nbsp;<a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/CacheBlock.html" title="interface in org.apache.sysds.runtime.controlprogram.caching">CacheBlock</a>&nbsp;pin(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key,
<a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/CacheBlock.html" title="interface in org.apache.sysds.runtime.controlprogram.caching">CacheBlock</a>&nbsp;block,
boolean&nbsp;dirty)</pre>
<div class="block">Pins a cache block into operation memory.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>key</code> - unique identifier and local FS filename for eviction</dd>
<dd><code>block</code> - cache block if not under UMM control, null otherwise</dd>
<dd><code>dirty</code> - indicator if block is dirty (subject to buffer pool management)</dd>
<dt><span class="returnLabel">Returns:</span></dt>
<dd>pinned cache block, potentially restored from local FS</dd>
</dl>
</li>
</ul>
<a name="pin-java.lang.String-long-boolean-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>pin</h4>
<pre>public&nbsp;void&nbsp;pin(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key,
long&nbsp;size,
boolean&nbsp;dirty)</pre>
<div class="block">Pins a virtual cache block into operation memory, by making a size reservation.
The provided size is an upper bound of the actual object size, and can be
updated on unpin (once the actual cache block is provided).</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>key</code> - unique identifier and local FS filename for eviction</dd>
<dd><code>size</code> - memory reservation in operation area</dd>
<dd><code>dirty</code> - indicator if block is dirty (subject to buffer pool management)</dd>
</dl>
</li>
</ul>
<a name="unpin-java.lang.String-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>unpin</h4>
<pre>public&nbsp;void&nbsp;unpin(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key)</pre>
<div class="block">Unpins (releases) a cache block from operation memory. Dirty objects
are logically moved back to the buffer pool area.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>key</code> - unique identifier and local FS filename for eviction</dd>
</dl>
</li>
</ul>
<a name="unpin-java.lang.String-org.apache.sysds.runtime.controlprogram.caching.CacheBlock-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>unpin</h4>
<pre>public&nbsp;void&nbsp;unpin(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key,
<a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/CacheBlock.html" title="interface in org.apache.sysds.runtime.controlprogram.caching">CacheBlock</a>&nbsp;block)</pre>
<div class="block">Unpins (releases) a cache block from operation memory. If the size of
the provided cache block differs from the UMM meta data, the UMM meta
data is updated. Use cases include update-in-place operations and
size reservations via worst-case upper bound estimates.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>key</code> - unique identifier and local FS filename for eviction</dd>
<dd><code>block</code> - cache block which may be under UMM control, if null ignored</dd>
</dl>
</li>
</ul>
<a name="delete-java.lang.String-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>delete</h4>
<pre>public&nbsp;void&nbsp;delete(<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>&nbsp;key)</pre>
<div class="block">Removes a cache block associated with the given key from all memory
areas, and deletes evicted representations (files in local FS). The
local file system deletes can happen asynchronously.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>key</code> - unique identifier and local FS filename for eviction</dd>
</dl>
</li>
</ul>
<a name="deleteAll--">
<!-- -->
</a>
<ul class="blockListLast">
<li class="blockList">
<h4>deleteAll</h4>
<pre>public&nbsp;void&nbsp;deleteAll()</pre>
<div class="block">Removes all cache blocks from all memory areas and deletes all evicted
representations (files in local FS). All internally thread pools must be
shut down in a gracefully manner (e.g., wait for pending deletes).</div>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
</div>
<!-- ========= END OF CLASS DATA ========= -->
<!-- ======= START OF BOTTOM NAVBAR ====== -->
<div class="bottomNav"><a name="navbar.bottom">
<!-- -->
</a>
<div class="skipNav"><a href="#skip.navbar.bottom" title="Skip navigation links">Skip navigation links</a></div>
<a name="navbar.bottom.firstrow">
<!-- -->
</a>
<ul class="navList" title="Navigation">
<li><a href="../../../../../../overview-summary.html">Overview</a></li>
<li><a href="package-summary.html">Package</a></li>
<li class="navBarCell1Rev">Class</li>
<li><a href="class-use/UnifiedMemoryManager.html">Use</a></li>
<li><a href="package-tree.html">Tree</a></li>
<li><a href="../../../../../../deprecated-list.html">Deprecated</a></li>
<li><a href="../../../../../../index-all.html">Index</a></li>
<li><a href="../../../../../../help-doc.html">Help</a></li>
</ul>
</div>
<div class="subNav">
<ul class="navList">
<li><a href="../../../../../../org/apache/sysds/runtime/controlprogram/caching/TensorObject.html" title="class in org.apache.sysds.runtime.controlprogram.caching"><span class="typeNameLink">Prev&nbsp;Class</span></a></li>
<li>Next&nbsp;Class</li>
</ul>
<ul class="navList">
<li><a href="../../../../../../index.html?org/apache/sysds/runtime/controlprogram/caching/UnifiedMemoryManager.html" target="_top">Frames</a></li>
<li><a href="UnifiedMemoryManager.html" target="_top">No&nbsp;Frames</a></li>
</ul>
<ul class="navList" id="allclasses_navbar_bottom">
<li><a href="../../../../../../allclasses-noframe.html">All&nbsp;Classes</a></li>
</ul>
<div>
<script type="text/javascript"><!--
allClassesLink = document.getElementById("allclasses_navbar_bottom");
if(window==top) {
allClassesLink.style.display = "block";
}
else {
allClassesLink.style.display = "none";
}
//-->
</script>
</div>
<div>
<ul class="subNavList">
<li>Summary:&nbsp;</li>
<li>Nested&nbsp;|&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.summary">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.summary">Method</a></li>
</ul>
<ul class="subNavList">
<li>Detail:&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.detail">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.detail">Method</a></li>
</ul>
</div>
<a name="skip.navbar.bottom">
<!-- -->
</a></div>
<!-- ======== END OF BOTTOM NAVBAR ======= -->
<p class="legalCopy"><small>Copyright &#169; 2021 <a href="https://www.apache.org/">The Apache Software Foundation</a>. All rights reserved.</small></p>
</body>
</html>