Google Git
Sign in
apache / spark-website / 74943c8625877cef547882f29b430d612b6f6a45 / . / site / docs / 4.0.0-preview1 / api / java / org / apache / spark / SparkStatusTracker.html
blob: a49a8a4d3404e2b32b75478d14c0c7b002eba7c3 [file] [log] [blame]
<!DOCTYPE HTML>
<html lang="en">
<head>
<!-- Generated by javadoc (17) on Tue May 28 17:21:12 UTC 2024 -->
<title>SparkStatusTracker (Spark 4.0.0-preview1 JavaDoc)</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="dc.created" content="2024-05-28">
<meta name="description" content="declaration: package: org.apache.spark, class: SparkStatusTracker">
<meta name="generator" content="javadoc/ClassWriterImpl">
<link rel="stylesheet" type="text/css" href="../../../stylesheet.css" title="Style">
<link rel="stylesheet" type="text/css" href="../../../script-dir/jquery-ui.min.css" title="Style">
<link rel="stylesheet" type="text/css" href="../../../jquery-ui.overrides.css" title="Style">
<script type="text/javascript" src="../../../script.js"></script>
<script type="text/javascript" src="../../../script-dir/jquery-3.6.1.min.js"></script>
<script type="text/javascript" src="../../../script-dir/jquery-ui.min.js"></script>
</head>
<body class="class-declaration-page">
<script type="text/javascript">var evenRowColor = "even-row-color";
var oddRowColor = "odd-row-color";
var tableTab = "table-tab";
var activeTableTab = "active-table-tab";
var pathtoroot = "../../../";
loadScripts(document, 'script');</script>
<noscript>
<div>JavaScript is disabled on your browser.</div>
</noscript>
<div class="flex-box">
<header role="banner" class="flex-header">
<nav role="navigation">
<!-- ========= START OF TOP NAVBAR ======= -->
<div class="top-nav" id="navbar-top">
<div class="skip-nav"><a href="#skip-navbar-top" title="Skip navigation links">Skip navigation links</a></div>
<ul id="navbar-top-firstrow" class="nav-list" title="Navigation">
<li><a href="../../../index.html">Overview</a></li>
<li><a href="package-summary.html">Package</a></li>
<li class="nav-bar-cell1-rev">Class</li>
<li><a href="../../../deprecated-list.html">Deprecated</a></li>
<li><a href="../../../index-all.html">Index</a></li>
<li><a href="../../../help-doc.html#class">Help</a></li>
</ul>
</div>
<div class="sub-nav">
<div>
<ul class="sub-nav-list">
<li>Summary:&nbsp;</li>
<li>Nested&nbsp;|&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li>Constr&nbsp;|&nbsp;</li>
<li><a href="#method-summary">Method</a></li>
</ul>
<ul class="sub-nav-list">
<li>Detail:&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li>Constr&nbsp;|&nbsp;</li>
<li><a href="#method-detail">Method</a></li>
</ul>
</div>
<div class="nav-list-search"><label for="search-input">SEARCH:</label>
<input type="text" id="search-input" value="search" disabled="disabled">
<input type="reset" id="reset-button" value="reset" disabled="disabled">
</div>
</div>
<!-- ========= END OF TOP NAVBAR ========= -->
<span class="skip-nav" id="skip-navbar-top"></span></nav>
</header>
<div class="flex-content">
<main role="main">
<!-- ======== START OF CLASS DATA ======== -->
<div class="header">
<div class="sub-title"><span class="package-label-in-type">Package</span>&nbsp;<a href="package-summary.html">org.apache.spark</a></div>
<h1 title="Class SparkStatusTracker" class="title">Class SparkStatusTracker</h1>
</div>
<div class="inheritance" title="Inheritance Tree"><a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html" title="class or interface in java.lang" class="external-link">Object</a>
<div class="inheritance">org.apache.spark.SparkStatusTracker</div>
</div>
<section class="class-description" id="class-description">
<hr>
<div class="type-signature"><span class="modifiers">public class </span><span class="element-name type-name-label">SparkStatusTracker</span>
<span class="extends-implements">extends <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html" title="class or interface in java.lang" class="external-link">Object</a></span></div>
<div class="block">Low-level status reporting APIs for monitoring job and stage progress.
<p>
These APIs intentionally provide very weak consistency semantics; consumers of these APIs should
be prepared to handle empty / missing information. For example, a job's stage ids may be known
but the status API may not have any information about the details of those stages, so
<code>getStageInfo</code> could potentially return <code>None</code> for a valid stage id.
<p>
To limit memory usage, these APIs only provide information on recent jobs / stages. These APIs
will provide information for the last <code>spark.ui.retainedStages</code> stages and
<code>spark.ui.retainedJobs</code> jobs.
<p>
NOTE: this class's constructor should be considered private and may be subject to change.</div>
</section>
<section class="summary">
<ul class="summary-list">
<!-- ========== METHOD SUMMARY =========== -->
<li>
<section class="method-summary" id="method-summary">
<h2>Method Summary</h2>
<div id="method-summary-table">
<div class="table-tabs" role="tablist" aria-orientation="horizontal"><button id="method-summary-table-tab0" role="tab" aria-selected="true" aria-controls="method-summary-table.tabpanel" tabindex="0" onkeydown="switchTab(event)" onclick="show('method-summary-table', 'method-summary-table', 3)" class="active-table-tab">All Methods</button><button id="method-summary-table-tab2" role="tab" aria-selected="false" aria-controls="method-summary-table.tabpanel" tabindex="-1" onkeydown="switchTab(event)" onclick="show('method-summary-table', 'method-summary-table-tab2', 3)" class="table-tab">Instance Methods</button><button id="method-summary-table-tab4" role="tab" aria-selected="false" aria-controls="method-summary-table.tabpanel" tabindex="-1" onkeydown="switchTab(event)" onclick="show('method-summary-table', 'method-summary-table-tab4', 3)" class="table-tab">Concrete Methods</button></div>
<div id="method-summary-table.tabpanel" role="tabpanel">
<div class="summary-table three-column-summary" aria-labelledby="method-summary-table-tab0">
<div class="table-header col-first">Modifier and Type</div>
<div class="table-header col-second">Method</div>
<div class="table-header col-last">Description</div>
<div class="col-first even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code>int[]</code></div>
<div class="col-second even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="#getActiveJobIds()" class="member-name-link">getActiveJobIds</a>()</code></div>
<div class="col-last even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4">
<div class="block">Returns an array containing the ids of all active jobs.</div>
</div>
<div class="col-first odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code>int[]</code></div>
<div class="col-second odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="#getActiveStageIds()" class="member-name-link">getActiveStageIds</a>()</code></div>
<div class="col-last odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4">
<div class="block">Returns an array containing the ids of all active stages.</div>
</div>
<div class="col-first even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="SparkExecutorInfo.html" title="interface in org.apache.spark">SparkExecutorInfo</a>[]</code></div>
<div class="col-second even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="#getExecutorInfos()" class="member-name-link">getExecutorInfos</a>()</code></div>
<div class="col-last even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4">
<div class="block">Returns information of all known executors, including host, port, cacheSize, numRunningTasks
and memory metrics.</div>
</div>
<div class="col-first odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code>int[]</code></div>
<div class="col-second odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="#getJobIdsForGroup(java.lang.String)" class="member-name-link">getJobIdsForGroup</a><wbr>(<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>&nbsp;jobGroup)</code></div>
<div class="col-last odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4">
<div class="block">Return a list of all known jobs in a particular job group.</div>
</div>
<div class="col-first even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code>int[]</code></div>
<div class="col-second even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="#getJobIdsForTag(java.lang.String)" class="member-name-link">getJobIdsForTag</a><wbr>(<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>&nbsp;jobTag)</code></div>
<div class="col-last even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4">
<div class="block">Return a list of all known jobs with a particular tag.</div>
</div>
<div class="col-first odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code>scala.Option&lt;<a href="SparkJobInfo.html" title="interface in org.apache.spark">SparkJobInfo</a>&gt;</code></div>
<div class="col-second odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="#getJobInfo(int)" class="member-name-link">getJobInfo</a><wbr>(int&nbsp;jobId)</code></div>
<div class="col-last odd-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4">
<div class="block">Returns job information, or <code>None</code> if the job info could not be found or was garbage collected.</div>
</div>
<div class="col-first even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code>scala.Option&lt;<a href="SparkStageInfo.html" title="interface in org.apache.spark">SparkStageInfo</a>&gt;</code></div>
<div class="col-second even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4"><code><a href="#getStageInfo(int)" class="member-name-link">getStageInfo</a><wbr>(int&nbsp;stageId)</code></div>
<div class="col-last even-row-color method-summary-table method-summary-table-tab2 method-summary-table-tab4">
<div class="block">Returns stage information, or <code>None</code> if the stage info could not be found or was
garbage collected.</div>
</div>
</div>
</div>
</div>
<div class="inherited-list">
<h3 id="methods-inherited-from-class-Object">Methods inherited from class&nbsp;java.lang.<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html" title="class or interface in java.lang" class="external-link">Object</a></h3>
<code><a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#equals(java.lang.Object)" title="class or interface in java.lang" class="external-link">equals</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#getClass()" title="class or interface in java.lang" class="external-link">getClass</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#hashCode()" title="class or interface in java.lang" class="external-link">hashCode</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#notify()" title="class or interface in java.lang" class="external-link">notify</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#notifyAll()" title="class or interface in java.lang" class="external-link">notifyAll</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#toString()" title="class or interface in java.lang" class="external-link">toString</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#wait()" title="class or interface in java.lang" class="external-link">wait</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#wait(long)" title="class or interface in java.lang" class="external-link">wait</a>, <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#wait(long,int)" title="class or interface in java.lang" class="external-link">wait</a></code></div>
</section>
</li>
</ul>
</section>
<section class="details">
<ul class="details-list">
<!-- ============ METHOD DETAIL ========== -->
<li>
<section class="method-details" id="method-detail">
<h2>Method Details</h2>
<ul class="member-list">
<li>
<section class="detail" id="getActiveJobIds()">
<h3>getActiveJobIds</h3>
<div class="member-signature"><span class="modifiers">public</span>&nbsp;<span class="return-type">int[]</span>&nbsp;<span class="element-name">getActiveJobIds</span>()</div>
<div class="block">Returns an array containing the ids of all active jobs.
<p>
This method does not guarantee the order of the elements in its result.</div>
<dl class="notes">
<dt>Returns:</dt>
<dd>(undocumented)</dd>
</dl>
</section>
</li>
<li>
<section class="detail" id="getActiveStageIds()">
<h3>getActiveStageIds</h3>
<div class="member-signature"><span class="modifiers">public</span>&nbsp;<span class="return-type">int[]</span>&nbsp;<span class="element-name">getActiveStageIds</span>()</div>
<div class="block">Returns an array containing the ids of all active stages.
<p>
This method does not guarantee the order of the elements in its result.</div>
<dl class="notes">
<dt>Returns:</dt>
<dd>(undocumented)</dd>
</dl>
</section>
</li>
<li>
<section class="detail" id="getExecutorInfos()">
<h3>getExecutorInfos</h3>
<div class="member-signature"><span class="modifiers">public</span>&nbsp;<span class="return-type"><a href="SparkExecutorInfo.html" title="interface in org.apache.spark">SparkExecutorInfo</a>[]</span>&nbsp;<span class="element-name">getExecutorInfos</span>()</div>
<div class="block">Returns information of all known executors, including host, port, cacheSize, numRunningTasks
and memory metrics.
Note this include information for both the driver and executors.</div>
<dl class="notes">
<dt>Returns:</dt>
<dd>(undocumented)</dd>
</dl>
</section>
</li>
<li>
<section class="detail" id="getJobIdsForGroup(java.lang.String)">
<h3>getJobIdsForGroup</h3>
<div class="member-signature"><span class="modifiers">public</span>&nbsp;<span class="return-type">int[]</span>&nbsp;<span class="element-name">getJobIdsForGroup</span><wbr><span class="parameters">(<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>&nbsp;jobGroup)</span></div>
<div class="block">Return a list of all known jobs in a particular job group. If <code>jobGroup</code> is <code>null</code>, then
returns all known jobs that are not associated with a job group.
<p>
The returned list may contain running, failed, and completed jobs, and may vary across
invocations of this method. This method does not guarantee the order of the elements in
its result.</div>
<dl class="notes">
<dt>Parameters:</dt>
<dd><code>jobGroup</code> - (undocumented)</dd>
<dt>Returns:</dt>
<dd>(undocumented)</dd>
</dl>
</section>
</li>
<li>
<section class="detail" id="getJobIdsForTag(java.lang.String)">
<h3>getJobIdsForTag</h3>
<div class="member-signature"><span class="modifiers">public</span>&nbsp;<span class="return-type">int[]</span>&nbsp;<span class="element-name">getJobIdsForTag</span><wbr><span class="parameters">(<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>&nbsp;jobTag)</span></div>
<div class="block">Return a list of all known jobs with a particular tag.
<p>
The returned list may contain running, failed, and completed jobs, and may vary across
invocations of this method. This method does not guarantee the order of the elements in
its result.</div>
<dl class="notes">
<dt>Parameters:</dt>
<dd><code>jobTag</code> - (undocumented)</dd>
<dt>Returns:</dt>
<dd>(undocumented)</dd>
</dl>
</section>
</li>
<li>
<section class="detail" id="getJobInfo(int)">
<h3>getJobInfo</h3>
<div class="member-signature"><span class="modifiers">public</span>&nbsp;<span class="return-type">scala.Option&lt;<a href="SparkJobInfo.html" title="interface in org.apache.spark">SparkJobInfo</a>&gt;</span>&nbsp;<span class="element-name">getJobInfo</span><wbr><span class="parameters">(int&nbsp;jobId)</span></div>
<div class="block">Returns job information, or <code>None</code> if the job info could not be found or was garbage collected.</div>
<dl class="notes">
<dt>Parameters:</dt>
<dd><code>jobId</code> - (undocumented)</dd>
<dt>Returns:</dt>
<dd>(undocumented)</dd>
</dl>
</section>
</li>
<li>
<section class="detail" id="getStageInfo(int)">
<h3>getStageInfo</h3>
<div class="member-signature"><span class="modifiers">public</span>&nbsp;<span class="return-type">scala.Option&lt;<a href="SparkStageInfo.html" title="interface in org.apache.spark">SparkStageInfo</a>&gt;</span>&nbsp;<span class="element-name">getStageInfo</span><wbr><span class="parameters">(int&nbsp;stageId)</span></div>
<div class="block">Returns stage information, or <code>None</code> if the stage info could not be found or was
garbage collected.</div>
<dl class="notes">
<dt>Parameters:</dt>
<dd><code>stageId</code> - (undocumented)</dd>
<dt>Returns:</dt>
<dd>(undocumented)</dd>
</dl>
</section>
</li>
</ul>
</section>
</li>
</ul>
</section>
<!-- ========= END OF CLASS DATA ========= -->
</main>
</div>
</div>
<script defer="defer" type="text/javascript" src="../../../lib/jquery.js"></script><script defer="defer" type="text/javascript" src="../../../lib/api-javadocs.js"></script></body>
</html>