| <html lang="en" dir="ltr" class="docs-wrapper docs-doc-page docs-version-current plugin-docs plugin-id-default docs-doc-id-tutorials/docker"> |
| <meta name="generator" content="Docusaurus v2.4.1"> |
| <title data-rh="true">Run with Docker | Apache® Druid</title><meta data-rh="true" name="viewport" content="width=device-width,initial-scale=1"><meta data-rh="true" name="twitter:card" content="summary_large_image"><meta data-rh="true" property="og:image" content="https://druid.apache.org/img/druid_nav.png"><meta data-rh="true" name="twitter:image" content="https://druid.apache.org/img/druid_nav.png"><meta data-rh="true" property="og:url" content="https://druid.apache.org/docs/latest/tutorials/docker"><meta data-rh="true" name="docusaurus_locale" content="en"><meta data-rh="true" name="docsearch:language" content="en"><meta data-rh="true" name="docusaurus_version" content="current"><meta data-rh="true" name="docusaurus_tag" content="docs-default-current"><meta data-rh="true" name="docsearch:version" content="current"><meta data-rh="true" name="docsearch:docusaurus_tag" content="docs-default-current"><meta data-rh="true" property="og:title" content="Run with Docker | Apache® Druid"><meta data-rh="true" name="description" content="<!--"><meta data-rh="true" property="og:description" content="<!--"><link data-rh="true" rel="icon" href="/img/favicon.png"><link data-rh="true" rel="canonical" href="https://druid.apache.org/docs/latest/tutorials/docker"><link data-rh="true" rel="alternate" href="https://druid.apache.org/docs/latest/tutorials/docker" hreflang="en"><link data-rh="true" rel="alternate" href="https://druid.apache.org/docs/latest/tutorials/docker" hreflang="x-default"><link rel="preconnect" href="https://www.google-analytics.com"> |
| <link rel="preconnect" href="https://www.googletagmanager.com"> |
| <script async src="https://www.googletagmanager.com/gtag/js?id=UA-131010415-1"></script> |
| <script>function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","UA-131010415-1",{})</script> |
| <link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.7.2/css/all.css"> |
| <script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.4/clipboard.min.js"></script><link rel="stylesheet" href="/assets/css/styles.f80751b3.css"> |
| <link rel="preload" href="/assets/js/runtime~main.dc5f839a.js" as="script"> |
| <link rel="preload" href="/assets/js/main.a03dfc13.js" as="script"> |
| <body class="navigation-with-keyboard"> |
| <script>!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}()</script><div id="__docusaurus"> |
| <div role="region" aria-label="Skip to main content"><a class="skipToContent_fXgn" href="#__docusaurus_skipToContent_fallback">Skip to main content</a></div><nav aria-label="Main" class="navbar navbar--fixed-top navbar--dark"><div class="navbar__inner"><div class="navbar__items"><button aria-label="Toggle navigation bar" aria-expanded="false" class="navbar__toggle clean-btn" type="button"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><a class="navbar__brand" href="/"><div class="navbar__logo"><img src="/img/druid_nav.png" alt="Apache® Druid" class="themedImage_ToTc themedImage--light_HNdA"><img src="/img/druid_nav.png" alt="Apache® Druid" class="themedImage_ToTc themedImage--dark_i4oU"></div></a></div><div class="navbar__items navbar__items--right"><a class="navbar__item navbar__link" href="/technology">Technology</a><a class="navbar__item navbar__link" href="/use-cases">Use Cases</a><a class="navbar__item navbar__link" href="/druid-powered">Powered By</a><a class="navbar__item navbar__link" href="/docs/latest/design/">Docs</a><a class="navbar__item navbar__link" href="/community/">Community</a><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link">Apache®</a><ul class="dropdown__menu"><li><a href="https://www.apache.org/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Foundation<svg width="12" height="12" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li><a href="https://apachecon.com/?ref=druid.apache.org" target="_blank" rel="noopener noreferrer" class="dropdown__link">Events<svg width="12" height="12" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li><a href="https://www.apache.org/licenses/" target="_blank" rel="noopener noreferrer" class="dropdown__link">License<svg width="12" height="12" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li><a href="https://www.apache.org/foundation/thanks.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Thanks<svg width="12" height="12" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li><a href="https://www.apache.org/security/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Security<svg width="12" height="12" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Sponsorship<svg width="12" height="12" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li></ul></div><a class="navbar__item navbar__link" href="/downloads/">Download</a><div class="searchBox_ZlJk"><div class="navbar__search"><span aria-label="expand searchbar" role="button" class="search-icon" tabindex="0"></span><input type="search" id="search_input_react" placeholder="Loading..." aria-label="Search" class="navbar__search-input search-bar" disabled=""></div></div></div></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div id="__docusaurus_skipToContent_fallback" class="main-wrapper mainWrapper_z2l0 docsWrapper_BCFX"><button aria-label="Scroll back to top" class="clean-btn theme-back-to-top-button backToTopButton_sjWU" type="button"></button><div class="docPage__5DB"><aside class="theme-doc-sidebar-container docSidebarContainer_b6E3"><div class="sidebarViewport_Xe31"><div class="sidebar_njMd"><nav aria-label="Docs sidebar" class="menu thin-scrollbar menu_SIkG"><ul class="theme-doc-sidebar-menu menu__list"><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/design/">Getting started</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret menu__link--active" aria-expanded="true" href="/docs/latest/tutorials/tutorial-msq-extern">Tutorials</a></div><ul style="display:block;overflow:visible;height:auto" class="menu__list"><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-msq-extern">Load files using SQL</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-kafka">Load from Apache Kafka</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-batch-hadoop">Load from Apache Hadoop</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-query">Query data</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-rollup">Aggregate data with rollup</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-sketches-theta">Theta sketches</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-retention">Configure data retention</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-update-data">Update existing data</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-compaction">Compact segments</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-delete-data">Deleting data</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-ingestion-spec">Write an ingestion spec</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-transform-spec">Transform input data</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-msq-convert-spec">Convert ingestion spec to SQL</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link menu__link--active" aria-current="page" tabindex="0" href="/docs/latest/tutorials/docker">Run with Docker</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-kerberos-hadoop">Kerberized HDFS deep storage</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-sql-query-view">Get to know Query view</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-unnest-arrays">Unnesting arrays</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-query-deep-storage">Query from deep storage</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-jupyter-index">Jupyter Notebook tutorials</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-jupyter-docker">Docker for tutorials</a></li><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class="menu__link" tabindex="0" href="/docs/latest/tutorials/tutorial-jdbc">JDBC connector</a></li></ul></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/design/architecture">Design</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/ingestion/">Ingestion</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/data-management/">Data management</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/querying/sql">Querying</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/api-reference/">API reference</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/configuration/">Configuration</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/operations/web-console">Operations</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/development/overview">Development</a></div></li><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item menu__list-item--collapsed"><div class="menu__list-item-collapsible"><a class="menu__link menu__link--sublist menu__link--sublist-caret" aria-expanded="false" href="/docs/latest/misc/papers-and-talks">Misc</a></div></li></ul></nav></div></div></aside><main class="docMainContainer_gTbr"><div class="container padding-top--md padding-bottom--lg"><div class="row"><div class="col docItemCol_VOVn"><div class="docItemContainer_Djhp"><article><nav class="theme-doc-breadcrumbs breadcrumbsContainer_Z_bl" aria-label="Breadcrumbs"><ul class="breadcrumbs" itemscope="" itemtype="https://schema.org/BreadcrumbList"><li class="breadcrumbs__item"><a aria-label="Home page" class="breadcrumbs__link" href="/"><svg viewBox="0 0 24 24" class="breadcrumbHomeIcon_YNFT"><path d="M10 19v-5h4v5c0 .55.45 1 1 1h3c.55 0 1-.45 1-1v-7h1.7c.46 0 .68-.57.33-.87L12.67 3.6c-.38-.34-.96-.34-1.34 0l-8.36 7.53c-.34.3-.13.87.33.87H5v7c0 .55.45 1 1 1h3c.55 0 1-.45 1-1z" fill="currentColor"></path></svg></a></li><li class="breadcrumbs__item"><span class="breadcrumbs__link">Tutorials</span><meta itemprop="position" content="1"></li><li itemscope="" itemprop="itemListElement" itemtype="https://schema.org/ListItem" class="breadcrumbs__item breadcrumbs__item--active"><span class="breadcrumbs__link" itemprop="name">Run with Docker</span><meta itemprop="position" content="2"></li></ul></nav><div class="tocCollapsible_ETCw theme-doc-toc-mobile tocMobile_ITEo"><button type="button" class="clean-btn tocCollapsibleButton_TO0P">On this page</button></div><div class="theme-doc-markdown markdown"><header><h1>Run with Docker</h1></header><p>This quickstart guides you through the steps to download the Apache Druid image from <a href="https://hub.docker.com/r/apache/druid" target="_blank" rel="noopener noreferrer">Docker Hub</a> and deploy it on a single machine using <a href="https://www.docker.com/get-started" target="_blank" rel="noopener noreferrer">Docker</a> and <a href="https://docs.docker.com/compose/" target="_blank" rel="noopener noreferrer">Docker Compose</a>. |
| After you finish the initial setup, the cluster will be ready to load data.</p><p>Before beginning the quickstart, it is helpful to read the <a href="/docs/latest/design/">general Druid overview</a> and the <a href="/docs/latest/ingestion/">ingestion overview</a>, because the tutorials refer to concepts discussed on those pages. It also helps to be familiar with <a href="https://www.docker.com/get-started" target="_blank" rel="noopener noreferrer">Docker</a>.</p><p>This tutorial assumes you will download the required files from GitHub. The files are also available in a Druid installation and in the Druid sources.</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="prerequisites">Prerequisites<a href="#prerequisites" class="hash-link" aria-label="Direct link to Prerequisites" title="Direct link to Prerequisites"></a></h2><ul><li><a href="https://www.docker.com/get-started" target="_blank" rel="noopener noreferrer">Docker</a></li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="docker-memory-requirements">Docker memory requirements<a href="#docker-memory-requirements" class="hash-link" aria-label="Direct link to Docker memory requirements" title="Direct link to Docker memory requirements"></a></h3><p>The default <code>docker-compose.yml</code> launches eight containers: Zookeeper, PostgreSQL, and six Druid containers based upon the <a href="/docs/latest/operations/single-server#single-server-reference-configurations-deprecated">micro quickstart configuration</a>. |
| Each Druid service is configured to use up to 7 GiB of memory (6 GiB direct memory and 1 GiB heap). However, the quickstart will not use all the available memory.</p><p>For this setup, Docker needs at least 6 GiB of memory available for the Druid cluster. For Docker Desktop on Mac OS, adjust the memory settings in the <a href="https://docs.docker.com/desktop/mac/" target="_blank" rel="noopener noreferrer">Docker Desktop preferences</a>. If you experience a crash with a 137 error code you likely don't have enough memory allocated to Docker.</p><p>You can modify the value of <code>DRUID_SINGLE_NODE_CONF</code> in the Docker <a href="#environment-file"><code>environment</code></a> to use different single-server mode. For example to use the nano quickstart: <code>DRUID_SINGLE_NODE_CONF=nano-quickstart</code>.</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="getting-started">Getting started<a href="#getting-started" class="hash-link" aria-label="Direct link to Getting started" title="Direct link to Getting started"></a></h2><p>Create a directory to hold the Druid Docker files.</p><p>The Druid source code contains <a href="https://github.com/apache/druid/blob/27.0.0/distribution/docker/docker-compose.yml" target="_blank" rel="noopener noreferrer">an example <code>docker-compose.yml</code></a> which pulls an image from Docker Hub and is suited to be used as an example environment and to experiment with Docker based Druid configuration and deployments. <a href="https://raw.githubusercontent.com/apache/druid/27.0.0/distribution/docker/docker-compose.yml" target="_blank" rel="noopener noreferrer">Download</a> this file to the directory created above.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="compose-file">Compose file<a href="#compose-file" class="hash-link" aria-label="Direct link to Compose file" title="Direct link to Compose file"></a></h3><p>The example <code>docker-compose.yml</code> will create a container for each Druid service, as well as ZooKeeper and a PostgreSQL container as the metadata store.</p><p>It will also create a named volume <code>druid_shared</code> as deep storage to keep and share segments and task logs among Druid services. The volume is mounted as <code>opt/shared</code> in the container.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="environment-file">Environment file<a href="#environment-file" class="hash-link" aria-label="Direct link to Environment file" title="Direct link to Environment file"></a></h3><p>The Druid <code>docker-compose.yml</code> example uses an <a href="https://docs.docker.com/compose/environment-variables/#the-env_file-configuration-option" target="_blank" rel="noopener noreferrer">environment file</a> to specify the complete Druid configuration, including the environment variables described in <a href="#configuration">Configuration</a>. This file is named <code>environment</code> by default, and must be in the same directory as the <code>docker-compose.yml</code> file. <a href="https://raw.githubusercontent.com/apache/druid/27.0.0/distribution/docker/environment" target="_blank" rel="noopener noreferrer">Download</a> the example <code>environment</code> file to the directory created above. The options in this file work well for trying Druid and for using the tutorial.</p><p>The single-file approach is inadequate for a production system. Instead we suggest using either <code>DRUID_COMMON_CONFIG</code> and <code>DRUID_CONFIG_${service}</code> or specially tailored, service-specific environment files.</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="configuration">Configuration<a href="#configuration" class="hash-link" aria-label="Direct link to Configuration" title="Direct link to Configuration"></a></h3><p>Configuration of the Druid Docker container is done via environment variables set within the container. Docker Compose passes the values from the <code>environment file</code> into the container. The variables may additionally specify paths to <a href="/docs/latest/configuration/">the standard Druid configuration files</a> which must be available within the container.</p><p>The default values are fine for the Quickstart. Production systems will want to modify the defaults.</p><p>Basic configuration:</p><ul><li><code>DRUID_MAXDIRECTMEMORYSIZE</code> -- set Java max direct memory size. Default is 6 GiB.</li><li><code>DRUID_XMX</code> -- set Java <code>Xmx</code>, the maximum heap size. Default is 1 GB.</li></ul><p>Production configuration:</p><ul><li><code>DRUID_CONFIG_COMMON</code> -- full path to a file for Druid common properties</li><li><code>DRUID_CONFIG_${service}</code> -- full path to a file for Druid service properties</li><li><code>JAVA_OPTS</code> -- set Java options</li></ul><p>Logging configuration:</p><ul><li><code>DRUID_LOG4J</code> -- set the entire <a href="https://logging.apache.org/log4j/2.x/manual/configuration.html#XML" target="_blank" rel="noopener noreferrer"><code>log4j.xml</code> configuration file</a> verbatim. (<a href="https://github.com/apache/druid/blob/27.0.0/distribution/docker/environment#L52" target="_blank" rel="noopener noreferrer">Example</a>)</li><li><code>DRUID_LOG_LEVEL</code> -- override the default <a href="https://en.wikipedia.org/wiki/Log4j#Log4j_log_levels" target="_blank" rel="noopener noreferrer">Log4j log level</a></li><li><code>DRUID_SERVICE_LOG4J</code> -- set the entire <a href="https://logging.apache.org/log4j/2.x/manual/configuration.html#XML" target="_blank" rel="noopener noreferrer"><code>log4j.xml</code> configuration file</a> verbatim specific to a service.</li><li><code>DRUID_SERVICE_LOG_LEVEL</code> -- override the default <a href="https://en.wikipedia.org/wiki/Log4j#Log4j_log_levels" target="_blank" rel="noopener noreferrer">Log4j log level</a> in the service specific log4j.</li></ul><p>Advanced memory configuration:</p><ul><li><code>DRUID_XMS</code> -- set Java <a href="https://docs.oracle.com/cd/E19900-01/819-4742/abeik/index.html" target="_blank" rel="noopener noreferrer"><code>Xms</code></a>, the initial heap size. Default is 1 GB.</li><li><code>DRUID_MAXNEWSIZE</code> -- set <a href="https://docs.oracle.com/cd/E19900-01/819-4742/abeik/index.html" target="_blank" rel="noopener noreferrer">Java max new size</a></li><li><code>DRUID_NEWSIZE</code> -- set <a href="https://docs.oracle.com/cd/E19900-01/819-4742/abeik/index.html" target="_blank" rel="noopener noreferrer">Java new size</a></li></ul><p>In addition to the special environment variables, the script which launches Druid in the container will use any environment variable starting with the <code>druid_</code> prefix as command-line configuration. For example, an environment variable</p><p><code>druid_metadata_storage_type=postgresql</code></p><p>is translated into the following option in the Java launch command for the Druid process in the container:</p><p><code>-Ddruid.metadata.storage.type=postgresql</code></p><p>Note that Druid uses port 8888 for the console. This port is also used by Jupyter and other tools. To avoid conflicts, you can change the port in the <a href="https://github.com/apache/druid/blob/0.21.1/distribution/docker/docker-compose.yml#L125" target="_blank" rel="noopener noreferrer"><code>ports</code></a> section of the <code>docker-compose.yml</code> file. For example, to expose the console on port 9999 of the host:</p><div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#bfc7d5;--prism-background-color:#292d3e"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#bfc7d5"><span class="token plain"> </span><span class="token key atrule">container_name</span><span class="token punctuation" style="color:rgb(199, 146, 234)">:</span><span class="token plain"> router</span><br></span><span class="token-line" style="color:#bfc7d5"><span class="token plain"> </span><span class="token punctuation" style="color:rgb(199, 146, 234)">...</span><span class="token plain"></span><br></span><span class="token-line" style="color:#bfc7d5"><span class="token plain"> </span><span class="token key atrule">ports</span><span class="token punctuation" style="color:rgb(199, 146, 234)">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#bfc7d5"><span class="token plain"> </span><span class="token punctuation" style="color:rgb(199, 146, 234)">-</span><span class="token plain"> </span><span class="token string" style="color:rgb(195, 232, 141)">"9999:8888"</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><h2 class="anchor anchorWithStickyNavbar_LWe7" id="launching-the-cluster">Launching the cluster<a href="#launching-the-cluster" class="hash-link" aria-label="Direct link to Launching the cluster" title="Direct link to Launching the cluster"></a></h2><p><code>cd</code> into the directory that contains the configuration files. This is the directory you created above, or the <code>distribution/docker/</code> in your Druid installation directory if you installed Druid locally.</p><p>Run <code>docker-compose up</code> to launch the cluster with a shell attached, or <code>docker-compose up -d</code> to run the cluster in the background.</p><p>Once the cluster has started, you can navigate to the <a href="/docs/latest/operations/web-console">web console</a> at <a href="http://localhost:8888" target="_blank" rel="noopener noreferrer">http://localhost:8888</a>. The <a href="/docs/latest/design/router">Druid router process</a> serves the UI.</p><p><img loading="lazy" alt="web console" src="/assets/images/tutorial-quickstart-01-55397fb67d2b63fa55d39770575a846f.png" title="web console" width="1250" height="740" class="img_ev3q"></p><p>It takes a few seconds for all the Druid processes to fully start up. If you open the console immediately after starting the services, you may see some errors that you can safely ignore.</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="using-the-cluster">Using the cluster<a href="#using-the-cluster" class="hash-link" aria-label="Direct link to Using the cluster" title="Direct link to Using the cluster"></a></h2><p>From here you can follow along with the <a href="/docs/latest/tutorials/#load-data">Quickstart</a>. For production use, refine your <code>docker-compose.yml</code> file to add any additional external service dependencies as necessary.</p><p>You can explore the Druid containers using Docker to start a shell:</p><div class="language-sh codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#bfc7d5;--prism-background-color:#292d3e"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-sh codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#bfc7d5"><span class="token plain">docker exec -ti <id> sh</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg viewBox="0 0 24 24" class="copyButtonIcon_y97N"><path fill="currentColor" d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg viewBox="0 0 24 24" class="copyButtonSuccessIcon_LjdS"><path fill="currentColor" d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>Where <code><id></code> is the container id found with <code>docker ps</code>. Druid is installed in <code>/opt/druid</code>. The <a href="https://github.com/apache/druid/blob/27.0.0/distribution/docker/druid.sh" target="_blank" rel="noopener noreferrer">script</a> which consumes the environment variables mentioned above, and which launches Druid, is located at <code>/druid.sh</code>.</p><p>Run <code>docker-compose down</code> to shut down the cluster. Your data is persisted as a set of <a href="https://docs.docker.com/storage/volumes/" target="_blank" rel="noopener noreferrer">Docker volumes</a> and will be available when you restart your Druid cluster.</p></div></article><nav class="pagination-nav docusaurus-mt-lg" aria-label="Docs pages"><a class="pagination-nav__link pagination-nav__link--prev" href="/docs/latest/tutorials/tutorial-msq-convert-spec"><div class="pagination-nav__sublabel">Previous</div><div class="pagination-nav__label">Convert ingestion spec to SQL</div></a><a class="pagination-nav__link pagination-nav__link--next" href="/docs/latest/tutorials/tutorial-kerberos-hadoop"><div class="pagination-nav__sublabel">Next</div><div class="pagination-nav__label">Kerberized HDFS deep storage</div></a></nav></div></div><div class="col col--3"><div class="tableOfContents_bqdL thin-scrollbar theme-doc-toc-desktop"><ul class="table-of-contents table-of-contents__left-border"><li><a href="#prerequisites" class="table-of-contents__link toc-highlight">Prerequisites</a><ul><li><a href="#docker-memory-requirements" class="table-of-contents__link toc-highlight">Docker memory requirements</a></li></ul></li><li><a href="#getting-started" class="table-of-contents__link toc-highlight">Getting started</a><ul><li><a href="#compose-file" class="table-of-contents__link toc-highlight">Compose file</a></li><li><a href="#environment-file" class="table-of-contents__link toc-highlight">Environment file</a></li><li><a href="#configuration" class="table-of-contents__link toc-highlight">Configuration</a></li></ul></li><li><a href="#launching-the-cluster" class="table-of-contents__link toc-highlight">Launching the cluster</a></li><li><a href="#using-the-cluster" class="table-of-contents__link toc-highlight">Using the cluster</a></li></ul></div></div></div></div></main></div></div><footer class="footer"><div class="container container-fluid"><div class="footer__bottom text--center"><div class="margin-bottom--sm"><img src="/img/favicon.png" class="themedImage_ToTc themedImage--light_HNdA footer__logo"><img src="/img/favicon.png" class="themedImage_ToTc themedImage--dark_i4oU footer__logo"></div><div class="footer__copyright">Copyright © 2023 Apache Software Foundation. Except where otherwise noted, licensed under CC BY-SA 4.0. Apache Druid, Druid, and the Druid logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</div></div></div></footer></div> |
| <script src="/assets/js/runtime~main.dc5f839a.js"></script> |
| <script src="/assets/js/main.a03dfc13.js"></script> |
