| <!DOCTYPE html> |
| <!-- Generated by pkgdown: do not edit by hand --><html lang="en"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <title>Connecting to Flight RPC Servers • Arrow R Package</title> |
| <!-- jquery --><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script><!-- Bootstrap --><link href="https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.4.0/cosmo/bootstrap.min.css" rel="stylesheet" crossorigin="anonymous"> |
| <script src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/js/bootstrap.min.js" integrity="sha256-nuL8/2cJ5NDSSwnKD8VqreErSWHtnEP9E7AySL+1ev4=" crossorigin="anonymous"></script><!-- bootstrap-toc --><link rel="stylesheet" href="../bootstrap-toc.css"> |
| <script src="../bootstrap-toc.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous"> |
| <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous"> |
| <!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- pkgdown --><link href="../pkgdown.css" rel="stylesheet"> |
| <script src="../pkgdown.js"></script><meta property="og:title" content="Connecting to Flight RPC Servers"> |
| <meta property="og:description" content="arrow"> |
| <!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]-->
|
|
|
| <!-- Matomo -->
|
| <script>
|
| var _paq = window._paq = window._paq || [];
|
| /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
|
| _paq.push(["setDoNotTrack", true]);
|
| _paq.push(["disableCookies"]);
|
| _paq.push(['trackPageView']);
|
| _paq.push(['enableLinkTracking']);
|
| (function() {
|
| var u="https://analytics.apache.org/";
|
| _paq.push(['setTrackerUrl', u+'matomo.php']);
|
| _paq.push(['setSiteId', '20']);
|
| var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
|
| g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
|
| })();
|
| </script>
|
| <!-- End Matomo Code -->
|
| |
| </head> |
| <body data-spy="scroll" data-target="#toc"> |
| <div class="container template-article"> |
| <header><div class="navbar navbar-default navbar-fixed-top" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <span class="navbar-brand"> |
| <a class="navbar-link" href="../index.html">Arrow R Package</a> |
| <span class="version label label-default" data-toggle="tooltip" data-placement="bottom" title="Released version">2.0.0</span> |
| </span> |
| </div> |
| |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav"> |
| <li> |
| <a href="https://arrow.apache.org/">❯❯❯</a> |
| </li> |
| <li> |
| <a href="../articles/arrow.html">Get started</a> |
| </li> |
| <li> |
| <a href="../reference/index.html">Reference</a> |
| </li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"> |
| Articles |
| |
| <span class="caret"></span> |
| </a> |
| <ul class="dropdown-menu" role="menu"> |
| <li> |
| <a href="../articles/dataset.html">Working with Arrow Datasets and dplyr</a> |
| </li> |
| <li> |
| <a href="../articles/flight.html">Connecting to Flight RPC Servers</a> |
| </li> |
| <li> |
| <a href="../articles/fs.html">Working with Cloud Storage (S3)</a> |
| </li> |
| <li> |
| <a href="../articles/install.html">Installing the Arrow Package on Linux</a> |
| </li> |
| <li> |
| <a href="../articles/python.html">Apache Arrow in Python and R with reticulate</a> |
| </li> |
| </ul> |
| </li> |
| <li> |
| <a href="../news/index.html">Changelog</a> |
| </li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"> |
| Project docs |
| |
| <span class="caret"></span> |
| </a> |
| <ul class="dropdown-menu" role="menu"> |
| <li> |
| <a href="https://arrow.apache.org/docs/format/README.html">Specification</a> |
| </li> |
| <li> |
| <a href="https://arrow.apache.org/docs/c_glib">C GLib</a> |
| </li> |
| <li> |
| <a href="https://arrow.apache.org/docs/cpp">C++</a> |
| </li> |
| <li> |
| <a href="https://arrow.apache.org/docs/java">Java</a> |
| </li> |
| <li> |
| <a href="https://arrow.apache.org/docs/js">JavaScript</a> |
| </li> |
| <li> |
| <a href="https://arrow.apache.org/docs/python">Python</a> |
| </li> |
| <li> |
| <a href="../index.html">R</a> |
| </li> |
| </ul> |
| </li> |
| </ul> |
| <ul class="nav navbar-nav navbar-right"> |
| <li> |
| <a href="https://github.com/apache/arrow/"> |
| <span class="fab fa fab fa-github fa-lg"></span> |
| |
| </a> |
| </li> |
| </ul> |
| </div> |
| <!--/.nav-collapse --> |
| </div> |
| <!--/.container --> |
| </div> |
| <!--/.navbar --> |
| |
| |
| |
| </header><div class="row"> |
| <div class="col-md-9 contents"> |
| <div class="page-header toc-ignore"> |
| <h1 data-toc-skip>Connecting to Flight RPC Servers</h1> |
| |
| |
| <small class="dont-index">Source: <a href="https://github.com/apache/arrow/blob/master/vignettes/flight.Rmd"><code>vignettes/flight.Rmd</code></a></small> |
| <div class="hidden name"><code>flight.Rmd</code></div> |
| |
| </div> |
| |
| |
| |
| <p><a href="https://arrow.apache.org/blog/2019/10/13/introducing-arrow-flight/"><strong>Flight</strong></a> is a general-purpose client-server framework for high performance transport of large datasets over network interfaces, built as part of the <a href="https://arrow.apache.org">Apache Arrow</a> project. The <code>arrow</code> package provides methods for connecting to Flight RPC servers to send and receive data.</p> |
| <div id="getting-started" class="section level2"> |
| <h2 class="hasAnchor"> |
| <a href="#getting-started" class="anchor"></a>Getting Started</h2> |
| <p>The <code>flight</code> functions in the package use <code>reticulate</code> to call methods in the <code>pyarrow</code> Python package. Before using them for the first time, you'll need to be sure you have <code>reticulate</code>, and you'll also need to install <code>pyarrow</code>:</p> |
| <div class="sourceCode"><pre class="downlit"> |
| <span class="fu"><a href="https://rdrr.io/r/utils/install.packages.html">install.packages</a></span><span class="op">(</span><span class="st">"reticulate"</span><span class="op">)</span> |
| <span class="fu">arrow</span><span class="fu">::</span><span class="fu"><a href="../reference/install_pyarrow.html">install_pyarrow</a></span><span class="op">(</span><span class="op">)</span></pre></div> |
| <p>See <code><a href="../articles/python.html">vignette("python", package = "arrow")</a></code> for more details on setting up <code>pyarrow</code>.</p> |
| </div> |
| <div id="example" class="section level2"> |
| <h2 class="hasAnchor"> |
| <a href="#example" class="anchor"></a>Example</h2> |
| <p>The package includes methods for starting a Python-based Flight server, as well as methods for connecting to a Flight server running elsewhere.</p> |
| <p>To illustrate both sides, in one process let's start a demo server:</p> |
| <div class="sourceCode"><pre class="downlit"> |
| <span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="https://github.com/apache/arrow/">arrow</a></span><span class="op">)</span> |
| <span class="va">demo_server</span> <span class="op"><-</span> <span class="fu"><a href="../reference/load_flight_server.html">load_flight_server</a></span><span class="op">(</span><span class="st">"demo_flight_server"</span><span class="op">)</span> |
| <span class="va">server</span> <span class="op"><-</span> <span class="va">demo_server</span><span class="op">$</span><span class="fu">DemoFlightServer</span><span class="op">(</span>port <span class="op">=</span> <span class="fl">8089</span><span class="op">)</span> |
| <span class="va">server</span><span class="op">$</span><span class="fu">serve</span><span class="op">(</span><span class="op">)</span></pre></div> |
| <p>We'll leave that one running.</p> |
| <p>In a different R process, let's connect to it and put some data in it.</p> |
| <div class="sourceCode"><pre class="downlit"> |
| <span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="https://github.com/apache/arrow/">arrow</a></span><span class="op">)</span> |
| <span class="va">client</span> <span class="op"><-</span> <span class="fu"><a href="../reference/flight_connect.html">flight_connect</a></span><span class="op">(</span>port <span class="op">=</span> <span class="fl">8089</span><span class="op">)</span> |
| <span class="co"># Upload some data to our server so there's something to demo</span> |
| <span class="fu"><a href="../reference/push_data.html">push_data</a></span><span class="op">(</span><span class="va">client</span>, <span class="va">iris</span>, path <span class="op">=</span> <span class="st">"test_data/iris"</span><span class="op">)</span></pre></div> |
| <p>Now, in a new R process, let's connect to the server and pull the data we put there:</p> |
| <div class="sourceCode"><pre class="downlit"> |
| <span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="https://github.com/apache/arrow/">arrow</a></span><span class="op">)</span> |
| <span class="kw"><a href="https://rdrr.io/r/base/library.html">library</a></span><span class="op">(</span><span class="va"><a href="https://dplyr.tidyverse.org">dplyr</a></span><span class="op">)</span> |
| <span class="va">client</span> <span class="op"><-</span> <span class="fu"><a href="../reference/flight_connect.html">flight_connect</a></span><span class="op">(</span>port <span class="op">=</span> <span class="fl">8089</span><span class="op">)</span> |
| <span class="va">client</span> <span class="op">%>%</span> |
| <span class="fu"><a href="../reference/flight_get.html">flight_get</a></span><span class="op">(</span><span class="st">"test_data/iris"</span><span class="op">)</span> <span class="op">%>%</span> |
| <span class="fu"><a href="https://dplyr.tidyverse.org/reference/group_by.html">group_by</a></span><span class="op">(</span><span class="va">Species</span><span class="op">)</span> <span class="op">%>%</span> |
| <span class="fu"><a href="https://dplyr.tidyverse.org/reference/summarise.html">summarize</a></span><span class="op">(</span>max_petal <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/base/Extremes.html">max</a></span><span class="op">(</span><span class="va">Petal.Length</span><span class="op">)</span><span class="op">)</span> |
| |
| <span class="co">## # A tibble: 3 x 2</span> |
| <span class="co">## Species max_petal</span> |
| <span class="co">## <fct> <dbl></span> |
| <span class="co">## 1 setosa 1.9</span> |
| <span class="co">## 2 versicolor 5.1</span> |
| <span class="co">## 3 virginica 6.9</span></pre></div> |
| <p>Because <code><a href="../reference/flight_get.html">flight_get()</a></code> returns an Arrow data structure, we can directly pipe its result into a <code>dplyr</code> workflow.</p> |
| </div> |
| </div> |
| |
| <div class="col-md-3 hidden-xs hidden-sm" id="pkgdown-sidebar"> |
| |
| <nav id="toc" data-toggle="toc"><h2 data-toc-skip>Contents</h2> |
| </nav> |
| </div> |
| |
| </div> |
| |
| |
| |
| <footer><div class="copyright"> |
| <p>Developed by Romain François, Jeroen Ooms, Neal Richardson, Apache Arrow.</p> |
| </div> |
| |
| <div class="pkgdown"> |
| <p>Site built with <a href="https://pkgdown.r-lib.org/">pkgdown</a> 1.6.1.</p> |
| </div> |
| |
| </footer> |
| </div> |
| |
| |
| |
| |
| <script type="text/javascript" src="/docs/_static/versionwarning.js"></script> </body> |
| </html> |
