Google Git
Sign in
apache / hadoop-site / 952d547359cafef280f0a0324055741b2ed55be1 / . / content / docs / r3.3.1 / api / org / apache / hadoop / mapreduce / OutputCommitter.html
blob: 930070a96af5d055ab640c118c6903115516561b [file] [log] [blame]
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<!-- NewPage -->
<html lang="en">
<head>
<!-- Generated by javadoc (1.8.0_292) on Tue Jun 15 06:00:53 GMT 2021 -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>OutputCommitter (Apache Hadoop Main 3.3.1 API)</title>
<meta name="date" content="2021-06-15">
<link rel="stylesheet" type="text/css" href="../../../../stylesheet.css" title="Style">
<script type="text/javascript" src="../../../../script.js"></script>
</head>
<body>
<script type="text/javascript"><!--
try {
if (location.href.indexOf('is-external=true') == -1) {
parent.document.title="OutputCommitter (Apache Hadoop Main 3.3.1 API)";
}
}
catch(err) {
}
//-->
var methods = {"i0":10,"i1":6,"i2":42,"i3":10,"i4":6,"i5":10,"i6":42,"i7":10,"i8":6,"i9":10,"i10":6,"i11":6};
var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"],8:["t4","Concrete Methods"],32:["t6","Deprecated Methods"]};
var altColor = "altColor";
var rowColor = "rowColor";
var tableTab = "tableTab";
var activeTableTab = "activeTableTab";
</script>
<noscript>
<div>JavaScript is disabled on your browser.</div>
</noscript>
<!-- ========= START OF TOP NAVBAR ======= -->
<div class="topNav"><a name="navbar.top">
<!-- -->
</a>
<div class="skipNav"><a href="#skip.navbar.top" title="Skip navigation links">Skip navigation links</a></div>
<a name="navbar.top.firstrow">
<!-- -->
</a>
<ul class="navList" title="Navigation">
<li><a href="../../../../overview-summary.html">Overview</a></li>
<li><a href="package-summary.html">Package</a></li>
<li class="navBarCell1Rev">Class</li>
<li><a href="class-use/OutputCommitter.html">Use</a></li>
<li><a href="package-tree.html">Tree</a></li>
<li><a href="../../../../deprecated-list.html">Deprecated</a></li>
<li><a href="../../../../index-all.html">Index</a></li>
<li><a href="../../../../help-doc.html">Help</a></li>
</ul>
</div>
<div class="subNav">
<ul class="navList">
<li><a href="../../../../org/apache/hadoop/mapreduce/MarkableIterator.html" title="class in org.apache.hadoop.mapreduce"><span class="typeNameLink">Prev&nbsp;Class</span></a></li>
<li><a href="../../../../org/apache/hadoop/mapreduce/OutputFormat.html" title="class in org.apache.hadoop.mapreduce"><span class="typeNameLink">Next&nbsp;Class</span></a></li>
</ul>
<ul class="navList">
<li><a href="../../../../index.html?org/apache/hadoop/mapreduce/OutputCommitter.html" target="_top">Frames</a></li>
<li><a href="OutputCommitter.html" target="_top">No&nbsp;Frames</a></li>
</ul>
<ul class="navList" id="allclasses_navbar_top">
<li><a href="../../../../allclasses-noframe.html">All&nbsp;Classes</a></li>
</ul>
<div>
<script type="text/javascript"><!--
allClassesLink = document.getElementById("allclasses_navbar_top");
if(window==top) {
allClassesLink.style.display = "block";
}
else {
allClassesLink.style.display = "none";
}
//-->
</script>
</div>
<div>
<ul class="subNavList">
<li>Summary:&nbsp;</li>
<li>Nested&nbsp;|&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.summary">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.summary">Method</a></li>
</ul>
<ul class="subNavList">
<li>Detail:&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.detail">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.detail">Method</a></li>
</ul>
</div>
<a name="skip.navbar.top">
<!-- -->
</a></div>
<!-- ========= END OF TOP NAVBAR ========= -->
<!-- ======== START OF CLASS DATA ======== -->
<div class="header">
<div class="subTitle">org.apache.hadoop.mapreduce</div>
<h2 title="Class OutputCommitter" class="title">Class OutputCommitter</h2>
</div>
<div class="contentContainer">
<ul class="inheritance">
<li><a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">java.lang.Object</a></li>
<li>
<ul class="inheritance">
<li>org.apache.hadoop.mapreduce.OutputCommitter</li>
</ul>
</li>
</ul>
<div class="description">
<ul class="blockList">
<li class="blockList">
<dl>
<dt>Direct Known Subclasses:</dt>
<dd><a href="../../../../org/apache/hadoop/mapred/OutputCommitter.html" title="class in org.apache.hadoop.mapred">OutputCommitter</a>, <a href="../../../../org/apache/hadoop/mapreduce/lib/output/PathOutputCommitter.html" title="class in org.apache.hadoop.mapreduce.lib.output">PathOutputCommitter</a></dd>
</dl>
<hr>
<br>
<pre>@InterfaceAudience.Public
@InterfaceStability.Stable
public abstract class <span class="typeNameLabel">OutputCommitter</span>
extends <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">Object</a></pre>
<div class="block"><code>OutputCommitter</code> describes the commit of task output for a
Map-Reduce job.
<p>The Map-Reduce framework relies on the <code>OutputCommitter</code> of
the job to:<p>
<ol>
<li>
Setup the job during initialization. For example, create the temporary
output directory for the job during the initialization of the job.
</li>
<li>
Cleanup the job after the job completion. For example, remove the
temporary output directory after the job completion.
</li>
<li>
Setup the task temporary output.
</li>
<li>
Check whether a task needs a commit. This is to avoid the commit
procedure if a task does not need commit.
</li>
<li>
Commit of the task output.
</li>
<li>
Discard the task commit.
</li>
</ol>
The methods in this class can be called from several different processes and
from several different contexts. It is important to know which process and
which context each is called from. Each method should be marked accordingly
in its documentation. It is also important to note that not all methods are
guaranteed to be called once and only once. If a method is not guaranteed to
have this property the output committer needs to handle this appropriately.
Also note it will only be in rare situations where they may be called
multiple times for the same task.</div>
<dl>
<dt><span class="seeLabel">See Also:</span></dt>
<dd><a href="../../../../org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.html" title="class in org.apache.hadoop.mapreduce.lib.output"><code>FileOutputCommitter</code></a>,
<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce"><code>JobContext</code></a>,
<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce"><code>TaskAttemptContext</code></a></dd>
</dl>
</li>
</ul>
</div>
<div class="summary">
<ul class="blockList">
<li class="blockList">
<!-- ======== CONSTRUCTOR SUMMARY ======== -->
<ul class="blockList">
<li class="blockList"><a name="constructor.summary">
<!-- -->
</a>
<h3>Constructor Summary</h3>
<table class="memberSummary" border="0" cellpadding="3" cellspacing="0" summary="Constructor Summary table, listing constructors, and an explanation">
<caption><span>Constructors</span><span class="tabEnd">&nbsp;</span></caption>
<tr>
<th class="colOne" scope="col">Constructor and Description</th>
</tr>
<tr class="altColor">
<td class="colOne"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#OutputCommitter--">OutputCommitter</a></span>()</code>&nbsp;</td>
</tr>
</table>
</li>
</ul>
<!-- ========== METHOD SUMMARY =========== -->
<ul class="blockList">
<li class="blockList"><a name="method.summary">
<!-- -->
</a>
<h3>Method Summary</h3>
<table class="memberSummary" border="0" cellpadding="3" cellspacing="0" summary="Method Summary table, listing methods, and an explanation">
<caption><span id="t0" class="activeTableTab"><span>All Methods</span><span class="tabEnd">&nbsp;</span></span><span id="t2" class="tableTab"><span><a href="javascript:show(2);">Instance Methods</a></span><span class="tabEnd">&nbsp;</span></span><span id="t3" class="tableTab"><span><a href="javascript:show(4);">Abstract Methods</a></span><span class="tabEnd">&nbsp;</span></span><span id="t4" class="tableTab"><span><a href="javascript:show(8);">Concrete Methods</a></span><span class="tabEnd">&nbsp;</span></span><span id="t6" class="tableTab"><span><a href="javascript:show(32);">Deprecated Methods</a></span><span class="tabEnd">&nbsp;</span></span></caption>
<tr>
<th class="colFirst" scope="col">Modifier and Type</th>
<th class="colLast" scope="col">Method and Description</th>
</tr>
<tr id="i0" class="altColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#abortJob-org.apache.hadoop.mapreduce.JobContext-org.apache.hadoop.mapreduce.JobStatus.State-">abortJob</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext,
org.apache.hadoop.mapreduce.JobStatus.State&nbsp;state)</code>
<div class="block">For aborting an unsuccessful job's output.</div>
</td>
</tr>
<tr id="i1" class="rowColor">
<td class="colFirst"><code>abstract void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#abortTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">abortTask</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)</code>
<div class="block">Discard the task output.</div>
</td>
</tr>
<tr id="i2" class="altColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#cleanupJob-org.apache.hadoop.mapreduce.JobContext-">cleanupJob</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)</code>
<div class="block"><span class="deprecatedLabel">Deprecated.</span>&nbsp;
<div class="block"><span class="deprecationComment">Use <a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#commitJob-org.apache.hadoop.mapreduce.JobContext-"><code>commitJob(JobContext)</code></a> and
<a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#abortJob-org.apache.hadoop.mapreduce.JobContext-org.apache.hadoop.mapreduce.JobStatus.State-"><code>abortJob(JobContext, JobStatus.State)</code></a> instead.</span></div>
</div>
</td>
</tr>
<tr id="i3" class="rowColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#commitJob-org.apache.hadoop.mapreduce.JobContext-">commitJob</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)</code>
<div class="block">For committing job's output after successful job completion.</div>
</td>
</tr>
<tr id="i4" class="altColor">
<td class="colFirst"><code>abstract void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#commitTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">commitTask</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)</code>
<div class="block">To promote the task's temporary output to final output location.</div>
</td>
</tr>
<tr id="i5" class="rowColor">
<td class="colFirst"><code>boolean</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#isCommitJobRepeatable-org.apache.hadoop.mapreduce.JobContext-">isCommitJobRepeatable</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)</code>
<div class="block">Returns true if an in-progress job commit can be retried.</div>
</td>
</tr>
<tr id="i6" class="altColor">
<td class="colFirst"><code>boolean</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#isRecoverySupported--">isRecoverySupported</a></span>()</code>
<div class="block"><span class="deprecatedLabel">Deprecated.</span>&nbsp;
<div class="block"><span class="deprecationComment">Use <a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#isRecoverySupported-org.apache.hadoop.mapreduce.JobContext-"><code>isRecoverySupported(JobContext)</code></a> instead.</span></div>
</div>
</td>
</tr>
<tr id="i7" class="rowColor">
<td class="colFirst"><code>boolean</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#isRecoverySupported-org.apache.hadoop.mapreduce.JobContext-">isRecoverySupported</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)</code>
<div class="block">Is task output recovery supported for restarting jobs?
If task output recovery is supported, job restart can be done more
efficiently.</div>
</td>
</tr>
<tr id="i8" class="altColor">
<td class="colFirst"><code>abstract boolean</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#needsTaskCommit-org.apache.hadoop.mapreduce.TaskAttemptContext-">needsTaskCommit</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)</code>
<div class="block">Check whether task needs a commit.</div>
</td>
</tr>
<tr id="i9" class="rowColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#recoverTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">recoverTask</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)</code>
<div class="block">Recover the task output.</div>
</td>
</tr>
<tr id="i10" class="altColor">
<td class="colFirst"><code>abstract void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#setupJob-org.apache.hadoop.mapreduce.JobContext-">setupJob</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)</code>
<div class="block">For the framework to setup the job output during initialization.</div>
</td>
</tr>
<tr id="i11" class="rowColor">
<td class="colFirst"><code>abstract void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#setupTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">setupTask</a></span>(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)</code>
<div class="block">Sets up output for the task.</div>
</td>
</tr>
</table>
<ul class="blockList">
<li class="blockList"><a name="methods.inherited.from.class.java.lang.Object">
<!-- -->
</a>
<h3>Methods inherited from class&nbsp;java.lang.<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">Object</a></h3>
<code><a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#clone--" title="class or interface in java.lang">clone</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-" title="class or interface in java.lang">equals</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#finalize--" title="class or interface in java.lang">finalize</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#getClass--" title="class or interface in java.lang">getClass</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#hashCode--" title="class or interface in java.lang">hashCode</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#notify--" title="class or interface in java.lang">notify</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#notifyAll--" title="class or interface in java.lang">notifyAll</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#toString--" title="class or interface in java.lang">toString</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#wait--" title="class or interface in java.lang">wait</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#wait-long-" title="class or interface in java.lang">wait</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html?is-external=true#wait-long-int-" title="class or interface in java.lang">wait</a></code></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<div class="details">
<ul class="blockList">
<li class="blockList">
<!-- ========= CONSTRUCTOR DETAIL ======== -->
<ul class="blockList">
<li class="blockList"><a name="constructor.detail">
<!-- -->
</a>
<h3>Constructor Detail</h3>
<a name="OutputCommitter--">
<!-- -->
</a>
<ul class="blockListLast">
<li class="blockList">
<h4>OutputCommitter</h4>
<pre>public&nbsp;OutputCommitter()</pre>
</li>
</ul>
</li>
</ul>
<!-- ============ METHOD DETAIL ========== -->
<ul class="blockList">
<li class="blockList"><a name="method.detail">
<!-- -->
</a>
<h3>Method Detail</h3>
<a name="setupJob-org.apache.hadoop.mapreduce.JobContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>setupJob</h4>
<pre>public abstract&nbsp;void&nbsp;setupJob(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">For the framework to setup the job output during initialization. This is
called from the application master process for the entire job. This will be
called multiple times, once per job attempt.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>jobContext</code> - Context of the job whose output is being written.</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code> - if temporary output could not be created</dd>
</dl>
</li>
</ul>
<a name="cleanupJob-org.apache.hadoop.mapreduce.JobContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>cleanupJob</h4>
<pre><a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Deprecated.html?is-external=true" title="class or interface in java.lang">@Deprecated</a>
public&nbsp;void&nbsp;cleanupJob(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block"><span class="deprecatedLabel">Deprecated.</span>&nbsp;<span class="deprecationComment">Use <a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#commitJob-org.apache.hadoop.mapreduce.JobContext-"><code>commitJob(JobContext)</code></a> and
<a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#abortJob-org.apache.hadoop.mapreduce.JobContext-org.apache.hadoop.mapreduce.JobStatus.State-"><code>abortJob(JobContext, JobStatus.State)</code></a> instead.</span></div>
<div class="block">For cleaning up the job's output after job completion. This is called
from the application master process for the entire job. This may be called
multiple times.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>jobContext</code> - Context of the job whose output is being written.</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
<a name="commitJob-org.apache.hadoop.mapreduce.JobContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>commitJob</h4>
<pre>public&nbsp;void&nbsp;commitJob(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">For committing job's output after successful job completion. Note that this
is invoked for jobs with final runstate as SUCCESSFUL. This is called
from the application master process for the entire job. This is guaranteed
to only be called once. If it throws an exception the entire job will
fail.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>jobContext</code> - Context of the job whose output is being written.</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
<a name="abortJob-org.apache.hadoop.mapreduce.JobContext-org.apache.hadoop.mapreduce.JobStatus.State-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>abortJob</h4>
<pre>public&nbsp;void&nbsp;abortJob(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext,
org.apache.hadoop.mapreduce.JobStatus.State&nbsp;state)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">For aborting an unsuccessful job's output. Note that this is invoked for
jobs with final runstate as <code>JobStatus.State.FAILED</code> or
<code>JobStatus.State.KILLED</code>. This is called from the application
master process for the entire job. This may be called multiple times.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>jobContext</code> - Context of the job whose output is being written.</dd>
<dd><code>state</code> - final runstate of the job</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
<a name="setupTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>setupTask</h4>
<pre>public abstract&nbsp;void&nbsp;setupTask(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">Sets up output for the task. This is called from each individual task's
process that will output to HDFS, and it is called just for that task. This
may be called multiple times for the same task, but for different task
attempts.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>taskContext</code> - Context of the task whose output is being written.</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
<a name="needsTaskCommit-org.apache.hadoop.mapreduce.TaskAttemptContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>needsTaskCommit</h4>
<pre>public abstract&nbsp;boolean&nbsp;needsTaskCommit(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">Check whether task needs a commit. This is called from each individual
task's process that will output to HDFS, and it is called just for that
task.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>taskContext</code> - </dd>
<dt><span class="returnLabel">Returns:</span></dt>
<dd>true/false</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
<a name="commitTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>commitTask</h4>
<pre>public abstract&nbsp;void&nbsp;commitTask(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">To promote the task's temporary output to final output location.
If <a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#needsTaskCommit-org.apache.hadoop.mapreduce.TaskAttemptContext-"><code>needsTaskCommit(TaskAttemptContext)</code></a> returns true and this
task is the task that the AM determines finished first, this method
is called to commit an individual task's output. This is to mark
that tasks output as complete, as <a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#commitJob-org.apache.hadoop.mapreduce.JobContext-"><code>commitJob(JobContext)</code></a> will
also be called later on if the entire job finished successfully. This
is called from a task's process. This may be called multiple times for the
same task, but different task attempts. It should be very rare for this to
be called multiple times and requires odd networking failures to make this
happen. In the future the Hadoop framework may eliminate this race.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>taskContext</code> - Context of the task whose output is being written.</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code> - if commit is not successful.</dd>
</dl>
</li>
</ul>
<a name="abortTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>abortTask</h4>
<pre>public abstract&nbsp;void&nbsp;abortTask(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">Discard the task output. This is called from a task's process to clean
up a single task's output that can not yet been committed. This may be
called multiple times for the same task, but for different task attempts.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>taskContext</code> - </dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
<a name="isRecoverySupported--">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>isRecoverySupported</h4>
<pre><a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Deprecated.html?is-external=true" title="class or interface in java.lang">@Deprecated</a>
public&nbsp;boolean&nbsp;isRecoverySupported()</pre>
<div class="block"><span class="deprecatedLabel">Deprecated.</span>&nbsp;<span class="deprecationComment">Use <a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#isRecoverySupported-org.apache.hadoop.mapreduce.JobContext-"><code>isRecoverySupported(JobContext)</code></a> instead.</span></div>
<div class="block">Is task output recovery supported for restarting jobs?
If task output recovery is supported, job restart can be done more
efficiently.</div>
<dl>
<dt><span class="returnLabel">Returns:</span></dt>
<dd><code>true</code> if task output recovery is supported,
<code>false</code> otherwise</dd>
<dt><span class="seeLabel">See Also:</span></dt>
<dd><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#recoverTask-org.apache.hadoop.mapreduce.TaskAttemptContext-"><code>recoverTask(TaskAttemptContext)</code></a></dd>
</dl>
</li>
</ul>
<a name="isCommitJobRepeatable-org.apache.hadoop.mapreduce.JobContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>isCommitJobRepeatable</h4>
<pre>public&nbsp;boolean&nbsp;isCommitJobRepeatable(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">Returns true if an in-progress job commit can be retried. If the MR AM is
re-run then it will check this value to determine if it can retry an
in-progress commit that was started by a previous version.
Note that in rare scenarios, the previous AM version might still be running
at that time, due to system anomalies. Hence if this method returns true
then the retry commit operation should be able to run concurrently with
the previous operation.
If repeatable job commit is supported, job restart can tolerate previous
AM failures during job commit.
By default, it is not supported. Extended classes (like:
FileOutputCommitter) should explicitly override it if provide support.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>jobContext</code> - Context of the job whose output is being written.</dd>
<dt><span class="returnLabel">Returns:</span></dt>
<dd><code>true</code> repeatable job commit is supported,
<code>false</code> otherwise</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
<a name="isRecoverySupported-org.apache.hadoop.mapreduce.JobContext-">
<!-- -->
</a>
<ul class="blockList">
<li class="blockList">
<h4>isRecoverySupported</h4>
<pre>public&nbsp;boolean&nbsp;isRecoverySupported(<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html" title="interface in org.apache.hadoop.mapreduce">JobContext</a>&nbsp;jobContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">Is task output recovery supported for restarting jobs?
If task output recovery is supported, job restart can be done more
efficiently.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>jobContext</code> - Context of the job whose output is being written.</dd>
<dt><span class="returnLabel">Returns:</span></dt>
<dd><code>true</code> if task output recovery is supported,
<code>false</code> otherwise</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
<dt><span class="seeLabel">See Also:</span></dt>
<dd><a href="../../../../org/apache/hadoop/mapreduce/OutputCommitter.html#recoverTask-org.apache.hadoop.mapreduce.TaskAttemptContext-"><code>recoverTask(TaskAttemptContext)</code></a></dd>
</dl>
</li>
</ul>
<a name="recoverTask-org.apache.hadoop.mapreduce.TaskAttemptContext-">
<!-- -->
</a>
<ul class="blockListLast">
<li class="blockList">
<h4>recoverTask</h4>
<pre>public&nbsp;void&nbsp;recoverTask(<a href="../../../../org/apache/hadoop/mapreduce/TaskAttemptContext.html" title="interface in org.apache.hadoop.mapreduce">TaskAttemptContext</a>&nbsp;taskContext)
throws <a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></pre>
<div class="block">Recover the task output.
The retry-count for the job will be passed via the
<code>MRJobConfig.APPLICATION_ATTEMPT_ID</code> key in
<a href="../../../../org/apache/hadoop/mapreduce/JobContext.html#getConfiguration--"><code>JobContext.getConfiguration()</code></a> for the
<code>OutputCommitter</code>. This is called from the application master
process, but it is called individually for each task.
If an exception is thrown the task will be attempted again.
This may be called multiple times for the same task. But from different
application attempts.</div>
<dl>
<dt><span class="paramLabel">Parameters:</span></dt>
<dd><code>taskContext</code> - Context of the task whose output is being recovered</dd>
<dt><span class="throwsLabel">Throws:</span></dt>
<dd><code><a href="https://docs.oracle.com/javase/8/docs/api/java/io/IOException.html?is-external=true" title="class or interface in java.io">IOException</a></code></dd>
</dl>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
</div>
<!-- ========= END OF CLASS DATA ========= -->
<!-- ======= START OF BOTTOM NAVBAR ====== -->
<div class="bottomNav"><a name="navbar.bottom">
<!-- -->
</a>
<div class="skipNav"><a href="#skip.navbar.bottom" title="Skip navigation links">Skip navigation links</a></div>
<a name="navbar.bottom.firstrow">
<!-- -->
</a>
<ul class="navList" title="Navigation">
<li><a href="../../../../overview-summary.html">Overview</a></li>
<li><a href="package-summary.html">Package</a></li>
<li class="navBarCell1Rev">Class</li>
<li><a href="class-use/OutputCommitter.html">Use</a></li>
<li><a href="package-tree.html">Tree</a></li>
<li><a href="../../../../deprecated-list.html">Deprecated</a></li>
<li><a href="../../../../index-all.html">Index</a></li>
<li><a href="../../../../help-doc.html">Help</a></li>
</ul>
</div>
<div class="subNav">
<ul class="navList">
<li><a href="../../../../org/apache/hadoop/mapreduce/MarkableIterator.html" title="class in org.apache.hadoop.mapreduce"><span class="typeNameLink">Prev&nbsp;Class</span></a></li>
<li><a href="../../../../org/apache/hadoop/mapreduce/OutputFormat.html" title="class in org.apache.hadoop.mapreduce"><span class="typeNameLink">Next&nbsp;Class</span></a></li>
</ul>
<ul class="navList">
<li><a href="../../../../index.html?org/apache/hadoop/mapreduce/OutputCommitter.html" target="_top">Frames</a></li>
<li><a href="OutputCommitter.html" target="_top">No&nbsp;Frames</a></li>
</ul>
<ul class="navList" id="allclasses_navbar_bottom">
<li><a href="../../../../allclasses-noframe.html">All&nbsp;Classes</a></li>
</ul>
<div>
<script type="text/javascript"><!--
allClassesLink = document.getElementById("allclasses_navbar_bottom");
if(window==top) {
allClassesLink.style.display = "block";
}
else {
allClassesLink.style.display = "none";
}
//-->
</script>
</div>
<div>
<ul class="subNavList">
<li>Summary:&nbsp;</li>
<li>Nested&nbsp;|&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.summary">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.summary">Method</a></li>
</ul>
<ul class="subNavList">
<li>Detail:&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor.detail">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method.detail">Method</a></li>
</ul>
</div>
<a name="skip.navbar.bottom">
<!-- -->
</a></div>
<!-- ======== END OF BOTTOM NAVBAR ======= -->
<p class="legalCopy"><small>Copyright &#169; 2021 <a href="https://www.apache.org">Apache Software Foundation</a>. All rights reserved.</small></p>
</body>
</html>