Google Git
Sign in
apache / spark-website / 74943c8625877cef547882f29b430d612b6f6a45 / . / site / docs / 4.0.0-preview1 / api / R / reference / column_nonaggregate_functions.html
blob: 21efee2ae3d07111659f9e574a929161cbd65394 [file] [log] [blame]
<!DOCTYPE html>
<!-- Generated by pkgdown: do not edit by hand --><html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><meta name="description" content="Non-aggregate functions defined for Column."><!-- Inform modern browsers that this page supports both dark and light color schemes,
and the page author prefers light. --><meta name="color-scheme" content="dark light"><script>
// If `prefers-color-scheme` is not supported, fall back to light mode.
// i.e. In this case, inject the `light` CSS before the others, with
// no media filter so that it will be downloaded with highest priority.
if (window.matchMedia("(prefers-color-scheme: dark)").media === "not all") {
document.documentElement.style.display = "none";
document.head.insertAdjacentHTML(
"beforeend",
"<link id=\"css\" rel=\"stylesheet\" href=\"https://bootswatch.com/5/flatly/bootstrap.css\" onload=\"document.documentElement.style.display = ''\">"
);
}
</script><title>Non-aggregate functions for Column operations — column_nonaggregate_functions • SparkR</title><script src="../deps/jquery-3.6.0/jquery-3.6.0.min.js"></script><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><link href="../deps/bootstrap-5.3.1/bootstrap.min.css" rel="stylesheet"><script src="../deps/bootstrap-5.3.1/bootstrap.bundle.min.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous"><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous"><!-- bootstrap-toc --><script src="https://cdn.rawgit.com/afeld/bootstrap-toc/v1.0.1/dist/bootstrap-toc.min.js"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- search --><script src="https://cdnjs.cloudflare.com/ajax/libs/fuse.js/6.4.6/fuse.js" integrity="sha512-zv6Ywkjyktsohkbp9bb45V6tEMoWhzFzXis+LrMehmJZZSys19Yxf1dopHx7WzIKxr5tK2dVcYmaCk2uqdjF4A==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/autocomplete.js/0.38.0/autocomplete.jquery.min.js" integrity="sha512-GU9ayf+66Xx2TmpxqJpliWbT5PiGYxpaG8rfnBEk1LL8l1KGkRShhngwdXK1UgqhAzWpZHSiYPc09/NwDQIGyg==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mark.js/8.11.1/mark.min.js" integrity="sha512-5CYOlHXGh6QpOFA/TeTylKLWfB3ftPsde7AnmhuitiTX4K5SqCLBeKro6sPS8ilsz1Q4NRx3v8Ko2IBiszzdww==" crossorigin="anonymous"></script><!-- pkgdown --><script src="../pkgdown.js"></script><link href="../extra.css" rel="stylesheet"><meta property="og:title" content="Non-aggregate functions for Column operations — column_nonaggregate_functions"><meta property="og:description" content="Non-aggregate functions defined for Column."><!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]--><!-- Flatly Theme - Light --><link id="css-light" rel="stylesheet" href="https://bootswatch.com/5/flatly/bootstrap.css" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)"><!-- Darkly Theme - Dark --><link id="css-dark" rel="stylesheet" href="https://bootswatch.com/5/darkly/bootstrap.css" media="(prefers-color-scheme: dark)"><!-- preferably CSS --><link rel="stylesheet" href="../preferably.css"><link id="css-code-light" rel="stylesheet" href="../code-color-scheme-light.css" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)"><link id="css-code-dark" rel="stylesheet" href="../code-color-scheme-dark.css" media="(prefers-color-scheme: dark)"><script src="../darkswitch.js"></script></head><body>
<a href="#main" class="visually-hidden-focusable">Skip to contents</a>
<nav class="navbar fixed-top navbar-dark navbar-expand-lg bg-primary"><div class="container">
<a class="external-link navbar-brand" href="https://spark.apache.org/">
<img src="https://spark.apache.org/images/spark-logo-rev.svg" alt="" max-height="100%"></a>
<a class="navbar-brand me-2" href="../index.html">SparkR</a>
<small class="nav-text text-muted me-auto" data-bs-toggle="tooltip" data-bs-placement="bottom" title="">4.0.0</small>
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbar" aria-controls="navbar" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div id="navbar" class="collapse navbar-collapse ms-2">
<ul class="navbar-nav me-auto"><li class="active nav-item">
<a class="nav-link" href="../reference/index.html">Reference</a>
</li>
<li class="nav-item dropdown">
<a href="#" class="nav-link dropdown-toggle" data-bs-toggle="dropdown" role="button" aria-expanded="false" aria-haspopup="true" id="dropdown-articles">Articles</a>
<div class="dropdown-menu" aria-labelledby="dropdown-articles">
<a class="dropdown-item" href="../articles/sparkr-vignettes.html">SparkR - Practical Guide</a>
</div>
</li>
</ul><form class="form-inline my-2 my-lg-0" role="search">
<input type="search" class="form-control me-sm-2" aria-label="Toggle navigation" name="search-input" data-search-index="../search.json" id="search-input" placeholder="Search for" autocomplete="off"></form>
<ul class="navbar-nav"><li>
<a class="external-link nav-link" id="css-toggle-btn" aria-label="github">
<span class="fas fa fas fa-adjust fa-lg"></span>
</a>
</li>
</ul></div>
</div>
</nav><div class="container template-reference-topic">
<div class="row">
<main id="main" class="col-md-9"><div class="page-header">
<img src="" class="logo" alt=""><h1>Non-aggregate functions for Column operations</h1>
<div class="d-none name"><code>column_nonaggregate_functions.Rd</code></div>
</div>
<div class="ref-description section level2">
<p>Non-aggregate functions defined for <code>Column</code>.</p>
</div>
<div class="section level2">
<h2 id="ref-usage">Usage<a class="anchor" aria-label="anchor" href="#ref-usage"></a></h2>
<div class="sourceCode"><pre class="sourceCode r"><code><span><span class="fu">when</span><span class="op">(</span><span class="va">condition</span>, <span class="va">value</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">bitwise_not</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">bitwiseNOT</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">create_array</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">create_map</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">expr</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">greatest</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">input_file_name</span><span class="op">(</span>x <span class="op">=</span> <span class="st">"missing"</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">isnan</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">least</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">lit</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">monotonically_increasing_id</span><span class="op">(</span>x <span class="op">=</span> <span class="st">"missing"</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">nanvl</span><span class="op">(</span><span class="va">y</span>, <span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">negate</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">negative</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">positive</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">rand</span><span class="op">(</span><span class="va">seed</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">randn</span><span class="op">(</span><span class="va">seed</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">spark_partition_id</span><span class="op">(</span>x <span class="op">=</span> <span class="st">"missing"</span><span class="op">)</span></span>
<span></span>
<span><span class="fu">struct</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for ANY</span></span>
<span><span class="fu">lit</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">bitwise_not</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">bitwiseNOT</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu"><a href="coalesce.html">coalesce</a></span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">isnan</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">is.nan</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for missing</span></span>
<span><span class="fu">monotonically_increasing_id</span><span class="op">(</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">negate</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">negative</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">positive</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for missing</span></span>
<span><span class="fu">spark_partition_id</span><span class="op">(</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for characterOrColumn</span></span>
<span><span class="fu">struct</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">nanvl</span><span class="op">(</span><span class="va">y</span>, <span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">greatest</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">least</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for character</span></span>
<span><span class="fu">expr</span><span class="op">(</span><span class="va">x</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for missing</span></span>
<span><span class="fu">rand</span><span class="op">(</span><span class="va">seed</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for numeric</span></span>
<span><span class="fu">rand</span><span class="op">(</span><span class="va">seed</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for missing</span></span>
<span><span class="fu">randn</span><span class="op">(</span><span class="va">seed</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for numeric</span></span>
<span><span class="fu">randn</span><span class="op">(</span><span class="va">seed</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">when</span><span class="op">(</span><span class="va">condition</span>, <span class="va">value</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">ifelse</span><span class="op">(</span><span class="va">test</span>, <span class="va">yes</span>, <span class="va">no</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">create_array</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for Column</span></span>
<span><span class="fu">create_map</span><span class="op">(</span><span class="va">x</span>, <span class="va">...</span><span class="op">)</span></span>
<span></span>
<span><span class="co"># S4 method for missing</span></span>
<span><span class="fu">input_file_name</span><span class="op">(</span><span class="op">)</span></span></code></pre></div>
</div>
<div class="section level2">
<h2 id="arguments">Arguments<a class="anchor" aria-label="anchor" href="#arguments"></a></h2>
<dl><dt>condition</dt>
<dd><p>the condition to test on. Must be a Column expression.</p></dd>
<dt>value</dt>
<dd><p>result expression.</p></dd>
<dt>x</dt>
<dd><p>Column to compute on. In <code>lit</code>, it is a literal value or a Column.
In <code>expr</code>, it contains an expression character object to be parsed.</p></dd>
<dt>...</dt>
<dd><p>additional Columns.</p></dd>
<dt>y</dt>
<dd><p>Column to compute on.</p></dd>
<dt>seed</dt>
<dd><p>a random seed. Can be missing.</p></dd>
<dt>test</dt>
<dd><p>a Column expression that describes the condition.</p></dd>
<dt>yes</dt>
<dd><p>return values for <code>TRUE</code> elements of test.</p></dd>
<dt>no</dt>
<dd><p>return values for <code>FALSE</code> elements of test.</p></dd>
</dl></div>
<div class="section level2">
<h2 id="details">Details<a class="anchor" aria-label="anchor" href="#details"></a></h2>
<p><code>lit</code>: A new Column is created to represent the literal value.
If the parameter is a Column, it is returned unchanged.</p>
<p><code>bitwise_not</code>: Computes bitwise NOT.</p>
<p><code>bitwiseNOT</code>: Computes bitwise NOT.</p>
<p><code>coalesce</code>: Returns the first column that is not NA, or NA if all inputs are.</p>
<p><code>isnan</code>: Returns true if the column is NaN.</p>
<p><code>is.nan</code>: Alias for isnan.</p>
<p><code>monotonically_increasing_id</code>: Returns a column that generates monotonically increasing
64-bit integers. The generated ID is guaranteed to be monotonically increasing and unique,
but not consecutive. The current implementation puts the partition ID in the upper 31 bits,
and the record number within each partition in the lower 33 bits. The assumption is that the
SparkDataFrame has less than 1 billion partitions, and each partition has less than 8 billion
records. As an example, consider a SparkDataFrame with two partitions, each with 3 records.
This expression would return the following IDs:
0, 1, 2, 8589934592 (1L &lt;&lt; 33), 8589934593, 8589934594.
This is equivalent to the MONOTONICALLY_INCREASING_ID function in SQL.
The method should be used with no argument.
Note: the function is non-deterministic because its result depends on partition IDs.</p>
<p><code>negate</code>: Unary minus, i.e. negate the expression.</p>
<p><code>negative</code>: Alias for <code>negate</code>.</p>
<p><code>positive</code>: Unary plus, i.e. return the expression.</p>
<p><code>spark_partition_id</code>: Returns the partition ID as a SparkDataFrame column.
Note that this is nondeterministic because it depends on data partitioning and
task scheduling.
This is equivalent to the <code>SPARK_PARTITION_ID</code> function in SQL.</p>
<p><code>struct</code>: Creates a new struct column that composes multiple input columns.</p>
<p><code>nanvl</code>: Returns the first column (<code>y</code>) if it is not NaN, or the second column
(<code>x</code>) if the first column is NaN. Both inputs should be floating point columns
(DoubleType or FloatType).</p>
<p><code>greatest</code>: Returns the greatest value of the list of column names, skipping null values.
This function takes at least 2 parameters. It will return null if all parameters are null.</p>
<p><code>least</code>: Returns the least value of the list of column names, skipping null values.
This function takes at least 2 parameters. It will return null if all parameters are null.</p>
<p><code>expr</code>: Parses the expression string into the column that it represents, similar to
<code>SparkDataFrame.selectExpr</code></p>
<p><code>rand</code>: Generates a random column with independent and identically distributed (i.i.d.)
samples uniformly distributed in [0.0, 1.0).
Note: the function is non-deterministic in general case.</p>
<p><code>randn</code>: Generates a column with independent and identically distributed (i.i.d.) samples
from the standard normal distribution.
Note: the function is non-deterministic in general case.</p>
<p><code>when</code>: Evaluates a list of conditions and returns one of multiple possible result
expressions. For unmatched expressions null is returned.</p>
<p><code>ifelse</code>: Evaluates a list of conditions and returns <code>yes</code> if the conditions are
satisfied. Otherwise <code>no</code> is returned for unmatched conditions.</p>
<p><code>create_array</code>: Creates a new array column. The input columns must all have the same data
type.</p>
<p><code>create_map</code>: Creates a new map column. The input columns must be grouped as key-value
pairs, e.g. (key1, value1, key2, value2, ...).
The key columns must all have the same data type, and can't be null.
The value columns must all have the same data type.</p>
<p><code>input_file_name</code>: Creates a string column with the input file name for a given row.
The method should be used with no argument.</p>
</div>
<div class="section level2">
<h2 id="note">Note<a class="anchor" aria-label="anchor" href="#note"></a></h2>
<p>lit since 1.5.0</p>
<p>bitwise_not since 3.2.0</p>
<p>bitwiseNOT since 1.5.0</p>
<p>coalesce(Column) since 2.1.1</p>
<p>isnan since 2.0.0</p>
<p>is.nan since 2.0.0</p>
<p>negate since 1.5.0</p>
<p>negative since 3.5.0</p>
<p>positive since 3.5.0</p>
<p>spark_partition_id since 2.0.0</p>
<p>struct since 1.6.0</p>
<p>nanvl since 1.5.0</p>
<p>greatest since 1.5.0</p>
<p>least since 1.5.0</p>
<p>expr since 1.5.0</p>
<p>rand since 1.5.0</p>
<p>rand(numeric) since 1.5.0</p>
<p>randn since 1.5.0</p>
<p>randn(numeric) since 1.5.0</p>
<p>when since 1.5.0</p>
<p>ifelse since 1.5.0</p>
<p>create_array since 2.3.0</p>
<p>create_map since 2.3.0</p>
<p>input_file_name since 2.3.0</p>
</div>
<div class="section level2">
<h2 id="see-also">See also<a class="anchor" aria-label="anchor" href="#see-also"></a></h2>
<div class="dont-index"><p>coalesce,SparkDataFrame-method</p>
<p>Other non-aggregate functions:
<code><a href="column.html">column</a>()</code>,
<code><a href="not.html">not</a>()</code></p></div>
</div>
<div class="section level2">
<h2 id="ref-examples">Examples<a class="anchor" aria-label="anchor" href="#ref-examples"></a></h2>
<div class="sourceCode"><pre class="sourceCode r"><code><span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="op">{</span></span></span>
<span class="r-in"><span><span class="co"># Dataframe used throughout this doc</span></span></span>
<span class="r-in"><span><span class="va">df</span> <span class="op">&lt;-</span> <span class="fu"><a href="createDataFrame.html">createDataFrame</a></span><span class="op">(</span><span class="fu"><a href="https://rdrr.io/r/base/cbind.html" class="external-link">cbind</a></span><span class="op">(</span>model <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/colnames.html" class="external-link">rownames</a></span><span class="op">(</span><span class="va">mtcars</span><span class="op">)</span>, <span class="va">mtcars</span><span class="op">)</span><span class="op">)</span><span class="op">}</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="op">{</span></span></span>
<span class="r-in"><span><span class="va">tmp</span> <span class="op">&lt;-</span> <span class="fu"><a href="mutate.html">mutate</a></span><span class="op">(</span><span class="va">df</span>, v1 <span class="op">=</span> <span class="fu">lit</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">mpg</span><span class="op">)</span>, v2 <span class="op">=</span> <span class="fu">lit</span><span class="op">(</span><span class="st">"x"</span><span class="op">)</span>, v3 <span class="op">=</span> <span class="fu">lit</span><span class="op">(</span><span class="st">"2015-01-01"</span><span class="op">)</span>,</span></span>
<span class="r-in"><span> v4 <span class="op">=</span> <span class="fu">negate</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">mpg</span><span class="op">)</span>, v5 <span class="op">=</span> <span class="fu">expr</span><span class="op">(</span><span class="st">'length(model)'</span><span class="op">)</span>,</span></span>
<span class="r-in"><span> v6 <span class="op">=</span> <span class="fu">greatest</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">vs</span>, <span class="va">df</span><span class="op">$</span><span class="va">am</span><span class="op">)</span>, v7 <span class="op">=</span> <span class="fu">least</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">vs</span>, <span class="va">df</span><span class="op">$</span><span class="va">am</span><span class="op">)</span>,</span></span>
<span class="r-in"><span> v8 <span class="op">=</span> <span class="fu"><a href="column.html">column</a></span><span class="op">(</span><span class="st">"mpg"</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="va">tmp</span><span class="op">)</span><span class="op">}</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="op">{</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="fu"><a href="select.html">select</a></span><span class="op">(</span><span class="va">df</span>, <span class="fu">bitwise_not</span><span class="op">(</span><span class="fu"><a href="cast.html">cast</a></span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">vs</span>, <span class="st">"int"</span><span class="op">)</span><span class="op">)</span><span class="op">)</span><span class="op">)</span><span class="op">}</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="fu"><a href="select.html">select</a></span><span class="op">(</span><span class="va">df</span>, <span class="fu">monotonically_increasing_id</span><span class="op">(</span><span class="op">)</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="fu"><a href="select.html">select</a></span><span class="op">(</span><span class="va">df</span>, <span class="fu">spark_partition_id</span><span class="op">(</span><span class="op">)</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="op">{</span></span></span>
<span class="r-in"><span><span class="va">tmp</span> <span class="op">&lt;-</span> <span class="fu"><a href="mutate.html">mutate</a></span><span class="op">(</span><span class="va">df</span>, v1 <span class="op">=</span> <span class="fu">struct</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">mpg</span>, <span class="va">df</span><span class="op">$</span><span class="va">cyl</span><span class="op">)</span>, v2 <span class="op">=</span> <span class="fu">struct</span><span class="op">(</span><span class="st">"hp"</span>, <span class="st">"wt"</span>, <span class="st">"vs"</span><span class="op">)</span>,</span></span>
<span class="r-in"><span> v3 <span class="op">=</span> <span class="fu">create_array</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">mpg</span>, <span class="va">df</span><span class="op">$</span><span class="va">cyl</span>, <span class="va">df</span><span class="op">$</span><span class="va">hp</span><span class="op">)</span>,</span></span>
<span class="r-in"><span> v4 <span class="op">=</span> <span class="fu">create_map</span><span class="op">(</span><span class="fu">lit</span><span class="op">(</span><span class="st">"x"</span><span class="op">)</span>, <span class="fu">lit</span><span class="op">(</span><span class="fl">1.0</span><span class="op">)</span>, <span class="fu">lit</span><span class="op">(</span><span class="st">"y"</span><span class="op">)</span>, <span class="fu">lit</span><span class="op">(</span><span class="op">-</span><span class="fl">1.0</span><span class="op">)</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="va">tmp</span><span class="op">)</span><span class="op">}</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="op">{</span></span></span>
<span class="r-in"><span><span class="va">tmp</span> <span class="op">&lt;-</span> <span class="fu"><a href="mutate.html">mutate</a></span><span class="op">(</span><span class="va">df</span>, r1 <span class="op">=</span> <span class="fu">rand</span><span class="op">(</span><span class="op">)</span>, r2 <span class="op">=</span> <span class="fu">rand</span><span class="op">(</span><span class="fl">10</span><span class="op">)</span>, r3 <span class="op">=</span> <span class="fu">randn</span><span class="op">(</span><span class="op">)</span>, r4 <span class="op">=</span> <span class="fu">randn</span><span class="op">(</span><span class="fl">10</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="va">tmp</span><span class="op">)</span><span class="op">}</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="op">{</span></span></span>
<span class="r-in"><span><span class="va">tmp</span> <span class="op">&lt;-</span> <span class="fu"><a href="mutate.html">mutate</a></span><span class="op">(</span><span class="va">df</span>, mpg_na <span class="op">=</span> <span class="fu"><a href="otherwise.html">otherwise</a></span><span class="op">(</span><span class="fu">when</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">mpg</span> <span class="op">&gt;</span> <span class="fl">20</span>, <span class="va">df</span><span class="op">$</span><span class="va">mpg</span><span class="op">)</span>, <span class="fu">lit</span><span class="op">(</span><span class="cn">NaN</span><span class="op">)</span><span class="op">)</span>,</span></span>
<span class="r-in"><span> mpg2 <span class="op">=</span> <span class="fu">ifelse</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">mpg</span> <span class="op">&gt;</span> <span class="fl">20</span> <span class="op">&amp;</span> <span class="va">df</span><span class="op">$</span><span class="va">am</span> <span class="op">&gt;</span> <span class="fl">0</span>, <span class="fl">0</span>, <span class="fl">1</span><span class="op">)</span>,</span></span>
<span class="r-in"><span> mpg3 <span class="op">=</span> <span class="fu">ifelse</span><span class="op">(</span><span class="va">df</span><span class="op">$</span><span class="va">mpg</span> <span class="op">&gt;</span> <span class="fl">20</span>, <span class="va">df</span><span class="op">$</span><span class="va">mpg</span>, <span class="fl">20.0</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="va">tmp</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="va">tmp</span> <span class="op">&lt;-</span> <span class="fu"><a href="mutate.html">mutate</a></span><span class="op">(</span><span class="va">tmp</span>, ind_na1 <span class="op">=</span> <span class="fu">is.nan</span><span class="op">(</span><span class="va">tmp</span><span class="op">$</span><span class="va">mpg_na</span><span class="op">)</span>, ind_na2 <span class="op">=</span> <span class="fu">isnan</span><span class="op">(</span><span class="va">tmp</span><span class="op">$</span><span class="va">mpg_na</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="fu"><a href="select.html">select</a></span><span class="op">(</span><span class="va">tmp</span>, <span class="fu"><a href="coalesce.html">coalesce</a></span><span class="op">(</span><span class="va">tmp</span><span class="op">$</span><span class="va">mpg_na</span>, <span class="va">tmp</span><span class="op">$</span><span class="va">mpg</span><span class="op">)</span><span class="op">)</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="fu"><a href="select.html">select</a></span><span class="op">(</span><span class="va">tmp</span>, <span class="fu">nanvl</span><span class="op">(</span><span class="va">tmp</span><span class="op">$</span><span class="va">mpg_na</span>, <span class="va">tmp</span><span class="op">$</span><span class="va">hp</span><span class="op">)</span><span class="op">)</span><span class="op">)</span><span class="op">}</span></span></span>
<span class="r-in"><span></span></span>
<span class="r-in"><span><span class="kw">if</span> <span class="op">(</span><span class="cn">FALSE</span><span class="op">)</span> <span class="op">{</span></span></span>
<span class="r-in"><span><span class="va">tmp</span> <span class="op">&lt;-</span> <span class="fu"><a href="read.text.html">read.text</a></span><span class="op">(</span><span class="st">"README.md"</span><span class="op">)</span></span></span>
<span class="r-in"><span><span class="fu"><a href="head.html">head</a></span><span class="op">(</span><span class="fu"><a href="select.html">select</a></span><span class="op">(</span><span class="va">tmp</span>, <span class="fu">input_file_name</span><span class="op">(</span><span class="op">)</span><span class="op">)</span><span class="op">)</span><span class="op">}</span></span></span>
</code></pre></div>
</div>
</main><aside class="col-md-3"><nav id="toc"><h2>On this page</h2>
</nav></aside></div>
<footer><div class="copyright">
<p></p><p>Developed by <a href="https://www.apache.org/" class="external-link"> The Apache Software Foundation</a>.</p>
</div>
<div class="pkgdown">
<p></p><p>Site built with <a href="https://pkgdown.r-lib.org/" class="external-link">pkgdown</a> 2.0.9.</p>
<p class="preferably">Using <a href="https://preferably.amirmasoudabdol.name/?source=footer" class="external-link">preferably</a> template.</p>
</div>
</footer></div>
</body></html>