docs/11.0/r/articles/developers/docker.html - arrow-site - Git at Google

 <!DOCTYPE html>
 <!-- Generated by pkgdown: do not edit by hand --><html lang="en">
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <meta charset="utf-8">
 <meta http-equiv="X-UA-Compatible" content="IE=edge">
 <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
 <meta name="description" content="A guide for arrow developers wanting to use docker
 ">
 <title>Using docker containers • Arrow R Package</title>
 <!-- favicons --><link rel="icon" type="image/png" sizes="16x16" href="../../favicon-16x16.png">
 <link rel="icon" type="image/png" sizes="32x32" href="../../favicon-32x32.png">
 <link rel="apple-touch-icon" type="image/png" sizes="180x180" href="../../apple-touch-icon.png">
 <link rel="apple-touch-icon" type="image/png" sizes="120x120" href="../../apple-touch-icon-120x120.png">
 <link rel="apple-touch-icon" type="image/png" sizes="76x76" href="../../apple-touch-icon-76x76.png">
 <link rel="apple-touch-icon" type="image/png" sizes="60x60" href="../../apple-touch-icon-60x60.png">
 <script src="../../deps/jquery-3.6.0/jquery-3.6.0.min.js"></script><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
 <link href="../../deps/bootstrap-5.2.2/bootstrap.min.css" rel="stylesheet">
 <script src="../../deps/bootstrap-5.2.2/bootstrap.bundle.min.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous">
 <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous">
 <!-- bootstrap-toc --><script src="https://cdn.jsdelivr.net/gh/afeld/bootstrap-toc@v1.0.1/dist/bootstrap-toc.min.js" integrity="sha256-4veVQbu7//Lk5TSmc7YV48MxtMy98e26cf5MrgZYnwo=" crossorigin="anonymous"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- search --><script src="https://cdnjs.cloudflare.com/ajax/libs/fuse.js/6.4.6/fuse.js" integrity="sha512-zv6Ywkjyktsohkbp9bb45V6tEMoWhzFzXis+LrMehmJZZSys19Yxf1dopHx7WzIKxr5tK2dVcYmaCk2uqdjF4A==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/autocomplete.js/0.38.0/autocomplete.jquery.min.js" integrity="sha512-GU9ayf+66Xx2TmpxqJpliWbT5PiGYxpaG8rfnBEk1LL8l1KGkRShhngwdXK1UgqhAzWpZHSiYPc09/NwDQIGyg==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mark.js/8.11.1/mark.min.js" integrity="sha512-5CYOlHXGh6QpOFA/TeTylKLWfB3ftPsde7AnmhuitiTX4K5SqCLBeKro6sPS8ilsz1Q4NRx3v8Ko2IBiszzdww==" crossorigin="anonymous"></script><!-- pkgdown --><script src="../../pkgdown.js"></script><script src="../../extra.js"></script><meta property="og:title" content="Using docker containers">
 <meta property="og:description" content="A guide for arrow developers wanting to use docker
 ">
 <meta property="og:image" content="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png">
 <meta property="og:image:alt" content="Apache Arrow logo, displaying the triple chevron image adjacent to the text">
 <meta name="twitter:card" content="summary_large_image">
 <meta name="twitter:creator" content="@apachearrow">
 <meta name="twitter:site" content="@apachearrow">
 <!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
 <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
 <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
 <![endif]--><!-- Matomo --><script>
   var _paq = window._paq = window._paq || [];
   /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
   /* We explicitly disable cookie tracking to avoid privacy issues */
   _paq.push(['disableCookies']);
   _paq.push(['trackPageView']);
   _paq.push(['enableLinkTracking']);
   (function() {
     var u="https://analytics.apache.org/";
     _paq.push(['setTrackerUrl', u+'matomo.php']);
     _paq.push(['setSiteId', '20']);
     var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
     g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
   })();
 </script><!-- End Matomo Code -->
 </head>
 <body>
     <a href="#main" class="visually-hidden-focusable">Skip to contents</a>


     <nav class="navbar fixed-top navbar-dark navbar-expand-lg bg-black"><div class="container">

     <a class="navbar-brand me-2" href="../../index.html">Arrow R Package</a>

     <span class="version">
       <small class="nav-text text-muted me-auto" data-bs-toggle="tooltip" data-bs-placement="bottom" title="">11.0.0</small>
     </span>


     <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbar" aria-controls="navbar" aria-expanded="false" aria-label="Toggle navigation">
       <span class="navbar-toggler-icon"></span>
     </button>

     <div id="navbar" class="collapse navbar-collapse ms-3">
       <ul class="navbar-nav me-auto">
 <li class="nav-item">
   <a class="nav-link" href="../../articles/arrow.html">Get started</a>
 </li>
 <li class="nav-item">
   <a class="nav-link" href="../../reference/index.html">Reference</a>
 </li>
 <li class="active nav-item dropdown">
   <a href="#" class="nav-link dropdown-toggle" data-bs-toggle="dropdown" role="button" aria-expanded="false" aria-haspopup="true" id="dropdown-articles">Articles</a>
   <div class="dropdown-menu" aria-labelledby="dropdown-articles">
     <h6 class="dropdown-header" data-toc-skip>Using the package</h6>
     <a class="dropdown-item" href="../../articles/read_write.html">Reading and writing data files</a>
     <a class="dropdown-item" href="../../articles/data_wrangling.html">Data analysis with dplyr syntax</a>
     <a class="dropdown-item" href="../../articles/dataset.html">Working with multi-file data sets</a>
     <a class="dropdown-item" href="../../articles/python.html">Integrating Arrow, Python, and R</a>
     <a class="dropdown-item" href="../../articles/fs.html">Using cloud storage (S3, GCS)</a>
     <a class="dropdown-item" href="../../articles/flight.html">Connecting to a Flight server</a>
     <div class="dropdown-divider"></div>
     <h6 class="dropdown-header" data-toc-skip>Arrow concepts</h6>
     <a class="dropdown-item" href="../../articles/data_objects.html">Data objects</a>
     <a class="dropdown-item" href="../../articles/data_types.html">Data types</a>
     <a class="dropdown-item" href="../../articles/metadata.html">Metadata</a>
     <div class="dropdown-divider"></div>
     <h6 class="dropdown-header" data-toc-skip>Installation</h6>
     <a class="dropdown-item" href="../../articles/install.html">Installing on Linux</a>
     <a class="dropdown-item" href="../../articles/install_nightly.html">Installing development versions</a>
     <div class="dropdown-divider"></div>
     <a class="dropdown-item" href="../../articles/index.html">More articles...</a>
   </div>
 </li>
 <li class="nav-item">
   <a class="nav-link" href="../../news/index.html">Changelog</a>
 </li>
       </ul>
 <form class="form-inline my-2 my-lg-0" role="search">
         <input type="search" class="form-control me-sm-2" aria-label="Toggle navigation" name="search-input" data-search-index="../../search.json" id="search-input" placeholder="Search for" autocomplete="off">
 </form>

       <ul class="navbar-nav"></ul>
 </div>


   </div>
 </nav><div class="container template-article">

 <div class="row">
   <main id="main" class="col-md-9"><div class="page-header">
       <img src="" class="logo" alt=""><h1>Using docker containers</h1>


       <small class="dont-index">Source: <a href="https://github.com/apache/arrow/blob/master/r/vignettes/developers/docker.Rmd" class="external-link"><code>vignettes/developers/docker.Rmd</code></a></small>
       <div class="d-none name"><code>docker.Rmd</code></div>
     </div>


 <p>Arrow is compatible with a huge number of combinations of OSs, OS versions, compilers, R versions, and other variables. Sometimes these combinations of variables means that behaviours are found in some environments which cannot be replicated in others. In addition, there are different ways of building Arrow, for example, using environment variables to specify the building of optional components.</p>
 <p>What all this means is that you may need to use a different setup to the one in which you are working, when diagnosing a bug or testing out a new feature which you have reason to believe may be affected by these variables. One way to do this is so spin up a Docker image containing the desired setup.</p>
 <p>This article provides a basic guide to using Docker in your R development.</p>
 <div class="section level2">
 <h2 id="how-do-i-run-a-docker-container">How do I run a Docker container?<a class="anchor" aria-label="anchor" href="#how-do-i-run-a-docker-container"></a>
 </h2>
 <p>There are a number of images which have been created for the convenience of Arrow devs and you can find them on <a href="https://hub.docker.com/r/apache/arrow-dev/tags" class="external-link">the DockerHub repo</a>.</p>
 <p>The code below shows an example command you could use to run a Docker container.</p>
 <p>This should be run in the root directory of a checkout of the arrow repo.</p>
 <pre class="shell"><code>docker run -it -e ARROW_DEPENDENCY_SOURCE=AUTO -v $(pwd):/arrow apache/arrow-dev:r-rhub-ubuntu-gcc-release-latest</code></pre>
 <p>Components:</p>
 <ul>
 <li>
 <code>docker run</code> - command to run the container</li>
 <li>
 <code>-it</code> - run with an interactive terminal so you can run commands on the containers</li>
 <li>
 <code>-e ARROW_DEPENDENCY_SOURCE=AUTO</code> - set the environment variable <code>ARROW_DEPENDENCY_SOURCE</code> to the value <code>AUTO</code>
 </li>
 <li>
 <code>-v $(pwd):/arrow</code> - mount the current directory at <code>/arrow</code> in the container</li>
 <li>
 <code>apache/arrow-dev</code> - the DockerHub repo to get this container from</li>
 <li>
 <code>r-rhub-ubuntu-gcc-release-latest</code> - the image tag</li>
 </ul>
 <p>Once you run this command, if you don’t have a copy of that particular image saved locally, it will first be downloaded before a container is spun up.</p>
 <p>In the example above, mounting the directory in which the Arrow repo was stored on the local machine, meant that that code could be built and tested on the container.</p>
 </div>
 <div class="section level2">
 <h2 id="how-do-i-exit-this-image">How do I exit this image?<a class="anchor" aria-label="anchor" href="#how-do-i-exit-this-image"></a>
 </h2>
 <p>On Linux, press Ctrl+D.</p>
 </div>
 <div class="section level2">
 <h2 id="how-do-i-show-all-images-saved">How do I show all images saved?<a class="anchor" aria-label="anchor" href="#how-do-i-show-all-images-saved"></a>
 </h2>
 <pre class="shell"><code>docker images</code></pre>
 </div>
 <div class="section level2">
 <h2 id="how-do-i-show-all-running-containers">How do I show all running containers?<a class="anchor" aria-label="anchor" href="#how-do-i-show-all-running-containers"></a>
 </h2>
 <pre class="shell"><code>docker ps</code></pre>
 </div>
 <div class="section level2">
 <h2 id="how-do-i-show-all-containers">How do I show all containers?<a class="anchor" aria-label="anchor" href="#how-do-i-show-all-containers"></a>
 </h2>
 <pre class="shell"><code>sudo docker ps -a</code></pre>
 </div>
 <div class="section level2">
 <h2 id="running-existing-workflows-from-docker-compose-yml">Running existing workflows from docker-compose.yml<a class="anchor" aria-label="anchor" href="#running-existing-workflows-from-docker-compose-yml"></a>
 </h2>
 <p>There are a number of workflows outlined in the file <code>docker-compose.yml</code> in the arrow repo root directory. For example, you can use the workflow called <code>r</code> to test building and installing the R package. This is advantageous as you can use existing utility scripts and install it onto a container which already has R on it.</p>
 <p>These workflows are also parameterized, which means you can specify different options (or just use the defaults, which can be found in <code>.env</code>)</p>
 <div class="section level3">
 <h3 id="example---the-manual-way">Example - The manual way<a class="anchor" aria-label="anchor" href="#example---the-manual-way"></a>
 </h3>
 <p>If you wanted to run <a href="https://hub.docker.com/r/rhub/ubuntu-gcc-release" class="external-link">RHub’s latest <code>ubuntu-gcc-release</code> image</a>, you could run:</p>
 <pre class="shell"><code>R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest docker-compose build r
 R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest docker-compose run r</code></pre>
 </div>
 <div class="section level3">
 <h3 id="example---using-archery">Example - Using Archery<a class="anchor" aria-label="anchor" href="#example---using-archery"></a>
 </h3>
 <p>Alternatively, you may prefer to use the <a href="https://arrow.apache.org/docs/developers/docker.html" class="external-link">Archery tool to run docker images</a>. This has the advantage of making it simpler to build some of the existing Arrow CI jobs which have hierarchical dependencies, and so for example, you could build the R package on a container which already has the C++ code pre-built.</p>
 <p>This is the same tool which our CI uses - via a tool called <a href="https://arrow.apache.org/docs/developers/crossbow.html" class="external-link">Crossbow</a>.</p>
 <p>If you want to run the <code>r</code> workflow discussed above, you could run:</p>
 <pre class="shell"><code>R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest archery docker run r</code></pre>
 </div>
 </div>
   </main><aside class="col-md-3"><nav id="toc"><h2>On this page</h2>
     </nav></aside>
 </div>


     <footer><div class="pkgdown-footer-left">
   <p></p>
 <p>Developed by Neal Richardson, Ian Cook, Nic Crane, Dewey Dunnington, Romain François, Jonathan Keane, Dragoș Moldovan-Grünfeld, Jeroen Ooms, Apache Arrow.</p>
 </div>

 <div class="pkgdown-footer-right">
   <p></p>
 <p>Site built with <a href="https://pkgdown.r-lib.org/" class="external-link">pkgdown</a> 2.0.7.</p>
 </div>

     </footer>
 </div>


   </body>
 </html>
	<!DOCTYPE html>
	<!-- Generated by pkgdown: do not edit by hand --><html lang="en">
	<head>
	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
	<meta charset="utf-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
	<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
	<meta name="description" content="A guide for arrow developers wanting to use docker
	">
	<title>Using docker containers • Arrow R Package</title>
	<!-- favicons --><link rel="icon" type="image/png" sizes="16x16" href="../../favicon-16x16.png">
	<link rel="icon" type="image/png" sizes="32x32" href="../../favicon-32x32.png">
	<link rel="apple-touch-icon" type="image/png" sizes="180x180" href="../../apple-touch-icon.png">
	<link rel="apple-touch-icon" type="image/png" sizes="120x120" href="../../apple-touch-icon-120x120.png">
	<link rel="apple-touch-icon" type="image/png" sizes="76x76" href="../../apple-touch-icon-76x76.png">
	<link rel="apple-touch-icon" type="image/png" sizes="60x60" href="../../apple-touch-icon-60x60.png">
	<script src="../../deps/jquery-3.6.0/jquery-3.6.0.min.js"></script><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
	<link href="../../deps/bootstrap-5.2.2/bootstrap.min.css" rel="stylesheet">
	<script src="../../deps/bootstrap-5.2.2/bootstrap.bundle.min.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous">
	<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous">
	<!-- bootstrap-toc --><script src="https://cdn.jsdelivr.net/gh/afeld/bootstrap-toc@v1.0.1/dist/bootstrap-toc.min.js" integrity="sha256-4veVQbu7//Lk5TSmc7YV48MxtMy98e26cf5MrgZYnwo=" crossorigin="anonymous"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- search --><script src="https://cdnjs.cloudflare.com/ajax/libs/fuse.js/6.4.6/fuse.js" integrity="sha512-zv6Ywkjyktsohkbp9bb45V6tEMoWhzFzXis+LrMehmJZZSys19Yxf1dopHx7WzIKxr5tK2dVcYmaCk2uqdjF4A==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/autocomplete.js/0.38.0/autocomplete.jquery.min.js" integrity="sha512-GU9ayf+66Xx2TmpxqJpliWbT5PiGYxpaG8rfnBEk1LL8l1KGkRShhngwdXK1UgqhAzWpZHSiYPc09/NwDQIGyg==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mark.js/8.11.1/mark.min.js" integrity="sha512-5CYOlHXGh6QpOFA/TeTylKLWfB3ftPsde7AnmhuitiTX4K5SqCLBeKro6sPS8ilsz1Q4NRx3v8Ko2IBiszzdww==" crossorigin="anonymous"></script><!-- pkgdown --><script src="../../pkgdown.js"></script><script src="../../extra.js"></script><meta property="og:title" content="Using docker containers">
	<meta property="og:description" content="A guide for arrow developers wanting to use docker
	">
	<meta property="og:image" content="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png">
	<meta property="og:image:alt" content="Apache Arrow logo, displaying the triple chevron image adjacent to the text">
	<meta name="twitter:card" content="summary_large_image">
	<meta name="twitter:creator" content="@apachearrow">
	<meta name="twitter:site" content="@apachearrow">
	<!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
	<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
	<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
	<![endif]--><!-- Matomo --><script>
	var _paq = window._paq = window._paq \|\| [];
	/* tracker methods like "setCustomDimension" should be called before "trackPageView" */
	/* We explicitly disable cookie tracking to avoid privacy issues */
	_paq.push(['disableCookies']);
	_paq.push(['trackPageView']);
	_paq.push(['enableLinkTracking']);
	(function() {
	var u="https://analytics.apache.org/";
	_paq.push(['setTrackerUrl', u+'matomo.php']);
	_paq.push(['setSiteId', '20']);
	var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
	g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
	})();
	</script><!-- End Matomo Code -->
	</head>
	<body>
	<a href="#main" class="visually-hidden-focusable">Skip to contents</a>


	<nav class="navbar fixed-top navbar-dark navbar-expand-lg bg-black"><div class="container">

	<a class="navbar-brand me-2" href="../../index.html">Arrow R Package</a>

	<span class="version">
	<small class="nav-text text-muted me-auto" data-bs-toggle="tooltip" data-bs-placement="bottom" title="">11.0.0</small>
	</span>


	<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbar" aria-controls="navbar" aria-expanded="false" aria-label="Toggle navigation">
	<span class="navbar-toggler-icon"></span>
	</button>

	<div id="navbar" class="collapse navbar-collapse ms-3">
	<ul class="navbar-nav me-auto">
	<li class="nav-item">
	<a class="nav-link" href="../../articles/arrow.html">Get started</a>
	</li>
	<li class="nav-item">
	<a class="nav-link" href="../../reference/index.html">Reference</a>
	</li>
	<li class="active nav-item dropdown">
	<a href="#" class="nav-link dropdown-toggle" data-bs-toggle="dropdown" role="button" aria-expanded="false" aria-haspopup="true" id="dropdown-articles">Articles</a>
	<div class="dropdown-menu" aria-labelledby="dropdown-articles">
	<h6 class="dropdown-header" data-toc-skip>Using the package</h6>
	<a class="dropdown-item" href="../../articles/read_write.html">Reading and writing data files</a>
	<a class="dropdown-item" href="../../articles/data_wrangling.html">Data analysis with dplyr syntax</a>
	<a class="dropdown-item" href="../../articles/dataset.html">Working with multi-file data sets</a>
	<a class="dropdown-item" href="../../articles/python.html">Integrating Arrow, Python, and R</a>
	<a class="dropdown-item" href="../../articles/fs.html">Using cloud storage (S3, GCS)</a>
	<a class="dropdown-item" href="../../articles/flight.html">Connecting to a Flight server</a>
	<div class="dropdown-divider"></div>
	<h6 class="dropdown-header" data-toc-skip>Arrow concepts</h6>
	<a class="dropdown-item" href="../../articles/data_objects.html">Data objects</a>
	<a class="dropdown-item" href="../../articles/data_types.html">Data types</a>
	<a class="dropdown-item" href="../../articles/metadata.html">Metadata</a>
	<div class="dropdown-divider"></div>
	<h6 class="dropdown-header" data-toc-skip>Installation</h6>
	<a class="dropdown-item" href="../../articles/install.html">Installing on Linux</a>
	<a class="dropdown-item" href="../../articles/install_nightly.html">Installing development versions</a>
	<div class="dropdown-divider"></div>
	<a class="dropdown-item" href="../../articles/index.html">More articles...</a>
	</div>
	</li>
	<li class="nav-item">
	<a class="nav-link" href="../../news/index.html">Changelog</a>
	</li>
	</ul>
	<form class="form-inline my-2 my-lg-0" role="search">
	<input type="search" class="form-control me-sm-2" aria-label="Toggle navigation" name="search-input" data-search-index="../../search.json" id="search-input" placeholder="Search for" autocomplete="off">
	</form>

	<ul class="navbar-nav"></ul>
	</div>


	</div>
	</nav><div class="container template-article">

	<div class="row">
	<main id="main" class="col-md-9"><div class="page-header">
	<img src="" class="logo" alt=""><h1>Using docker containers</h1>


	<small class="dont-index">Source: <a href="https://github.com/apache/arrow/blob/master/r/vignettes/developers/docker.Rmd" class="external-link"><code>vignettes/developers/docker.Rmd</code></a></small>
	<div class="d-none name"><code>docker.Rmd</code></div>
	</div>



	<p>Arrow is compatible with a huge number of combinations of OSs, OS versions, compilers, R versions, and other variables. Sometimes these combinations of variables means that behaviours are found in some environments which cannot be replicated in others. In addition, there are different ways of building Arrow, for example, using environment variables to specify the building of optional components.</p>
	<p>What all this means is that you may need to use a different setup to the one in which you are working, when diagnosing a bug or testing out a new feature which you have reason to believe may be affected by these variables. One way to do this is so spin up a Docker image containing the desired setup.</p>
	<p>This article provides a basic guide to using Docker in your R development.</p>
	<div class="section level2">
	<h2 id="how-do-i-run-a-docker-container">How do I run a Docker container?<a class="anchor" aria-label="anchor" href="#how-do-i-run-a-docker-container"></a>
	</h2>
	<p>There are a number of images which have been created for the convenience of Arrow devs and you can find them on <a href="https://hub.docker.com/r/apache/arrow-dev/tags" class="external-link">the DockerHub repo</a>.</p>
	<p>The code below shows an example command you could use to run a Docker container.</p>
	<p>This should be run in the root directory of a checkout of the arrow repo.</p>
	<pre class="shell"><code>docker run -it -e ARROW_DEPENDENCY_SOURCE=AUTO -v $(pwd):/arrow apache/arrow-dev:r-rhub-ubuntu-gcc-release-latest</code></pre>
	<p>Components:</p>
	<ul>
	<li>
	<code>docker run</code> - command to run the container</li>
	<li>
	<code>-it</code> - run with an interactive terminal so you can run commands on the containers</li>
	<li>
	<code>-e ARROW_DEPENDENCY_SOURCE=AUTO</code> - set the environment variable <code>ARROW_DEPENDENCY_SOURCE</code> to the value <code>AUTO</code>
	</li>
	<li>
	<code>-v $(pwd):/arrow</code> - mount the current directory at <code>/arrow</code> in the container</li>
	<li>
	<code>apache/arrow-dev</code> - the DockerHub repo to get this container from</li>
	<li>
	<code>r-rhub-ubuntu-gcc-release-latest</code> - the image tag</li>
	</ul>
	<p>Once you run this command, if you don’t have a copy of that particular image saved locally, it will first be downloaded before a container is spun up.</p>
	<p>In the example above, mounting the directory in which the Arrow repo was stored on the local machine, meant that that code could be built and tested on the container.</p>
	</div>
	<div class="section level2">
	<h2 id="how-do-i-exit-this-image">How do I exit this image?<a class="anchor" aria-label="anchor" href="#how-do-i-exit-this-image"></a>
	</h2>
	<p>On Linux, press Ctrl+D.</p>
	</div>
	<div class="section level2">
	<h2 id="how-do-i-show-all-images-saved">How do I show all images saved?<a class="anchor" aria-label="anchor" href="#how-do-i-show-all-images-saved"></a>
	</h2>
	<pre class="shell"><code>docker images</code></pre>
	</div>
	<div class="section level2">
	<h2 id="how-do-i-show-all-running-containers">How do I show all running containers?<a class="anchor" aria-label="anchor" href="#how-do-i-show-all-running-containers"></a>
	</h2>
	<pre class="shell"><code>docker ps</code></pre>
	</div>
	<div class="section level2">
	<h2 id="how-do-i-show-all-containers">How do I show all containers?<a class="anchor" aria-label="anchor" href="#how-do-i-show-all-containers"></a>
	</h2>
	<pre class="shell"><code>sudo docker ps -a</code></pre>
	</div>
	<div class="section level2">
	<h2 id="running-existing-workflows-from-docker-compose-yml">Running existing workflows from docker-compose.yml<a class="anchor" aria-label="anchor" href="#running-existing-workflows-from-docker-compose-yml"></a>
	</h2>
	<p>There are a number of workflows outlined in the file <code>docker-compose.yml</code> in the arrow repo root directory. For example, you can use the workflow called <code>r</code> to test building and installing the R package. This is advantageous as you can use existing utility scripts and install it onto a container which already has R on it.</p>
	<p>These workflows are also parameterized, which means you can specify different options (or just use the defaults, which can be found in <code>.env</code>)</p>
	<div class="section level3">
	<h3 id="example---the-manual-way">Example - The manual way<a class="anchor" aria-label="anchor" href="#example---the-manual-way"></a>
	</h3>
	<p>If you wanted to run <a href="https://hub.docker.com/r/rhub/ubuntu-gcc-release" class="external-link">RHub’s latest <code>ubuntu-gcc-release</code> image</a>, you could run:</p>
	<pre class="shell"><code>R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest docker-compose build r
	R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest docker-compose run r</code></pre>
	</div>
	<div class="section level3">
	<h3 id="example---using-archery">Example - Using Archery<a class="anchor" aria-label="anchor" href="#example---using-archery"></a>
	</h3>
	<p>Alternatively, you may prefer to use the <a href="https://arrow.apache.org/docs/developers/docker.html" class="external-link">Archery tool to run docker images</a>. This has the advantage of making it simpler to build some of the existing Arrow CI jobs which have hierarchical dependencies, and so for example, you could build the R package on a container which already has the C++ code pre-built.</p>
	<p>This is the same tool which our CI uses - via a tool called <a href="https://arrow.apache.org/docs/developers/crossbow.html" class="external-link">Crossbow</a>.</p>
	<p>If you want to run the <code>r</code> workflow discussed above, you could run:</p>
	<pre class="shell"><code>R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest archery docker run r</code></pre>
	</div>
	</div>
	</main><aside class="col-md-3"><nav id="toc"><h2>On this page</h2>
	</nav></aside>
	</div>



	<footer><div class="pkgdown-footer-left">
	<p></p>
	<p>Developed by Neal Richardson, Ian Cook, Nic Crane, Dewey Dunnington, Romain François, Jonathan Keane, Dragoș Moldovan-Grünfeld, Jeroen Ooms, Apache Arrow.</p>
	</div>

	<div class="pkgdown-footer-right">
	<p></p>
	<p>Site built with <a href="https://pkgdown.r-lib.org/" class="external-link">pkgdown</a> 2.0.7.</p>
	</div>

	</footer>
	</div>





	</body>
	</html>