| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <link rel="stylesheet" type="text/css" href="/css/bootstrap/5.3.1/dist/css/bootstrap.min.css"> |
| <link rel="stylesheet" type="text/css" href="/css/fontawesome/fontawesome-free-6.4.2-web/css/all.min.css"> |
| <link rel="stylesheet" type="text/css" href="/css/datatables/bs5/dt-1.13.6/datatables.min.css"> |
| <link rel="stylesheet" type="text/css" href="/css/accumulo.css"> |
| |
| <title>Accumulo Documentation - On Disk Encryption</title> |
| |
| <script type="text/javascript" src="/js/jquery/3.7.0/jquery.min.js"></script> |
| <script type="text/javascript" src="/js/bootstrap/5.3.1/dist/js/bootstrap.bundle.min.js"></script> |
| <script type="text/javascript" src="/js/datatables/bs5/dt-1.13.6/datatables.min.js"></script> |
| <script type="text/javascript" src="https://www.apachecon.com/event-images/snippet.js"></script> |
| <script type="text/javascript" src="/js/accumulo.js"></script> |
| </head> |
| <body style="padding-top: 100px"> |
| |
| <nav class="navbar navbar-expand-lg navbar-light fixed-top bg-light"> |
| <div class="container"> |
| <a class="navbar-brand" href="/"> |
| <img alt="Apache Accumulo" id="nav-logo" src="/images/accumulo-logo.png" width="200"> |
| </a> |
| <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbar-items"> |
| <span class="navbar-toggler-icon"></span> |
| </button> |
| <div class="collapse navbar-collapse" id="navbar-items"> |
| <ul class="navbar-nav me-auto"> |
| <li class="nav-item"><a class="nav-link" href="/downloads">Download</a></li> |
| <li class="nav-item"><a class="nav-link" href="/tour">Tour</a></li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown">Releases</a> |
| <ul class="dropdown-menu"> |
| <li><a class="dropdown-item" href="/release/accumulo-3.0.0/">3.0.0 (Latest non-LTM)</a></li> |
| <li><a class="dropdown-item" href="/release/accumulo-2.1.2/">2.1.2 (Latest LTM)</a></li> |
| <li><a class="dropdown-item" href="/release/accumulo-1.10.4/">1.10.4 (Legacy LTM)</a></li> |
| <li><a class="dropdown-item" href="/release/">Archive</a></li> |
| </ul> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown">Documentation</a> |
| <ul class="dropdown-menu"> |
| <li><a class="dropdown-item" href="/docs/2.x">User Manual (2.x)</a></li> |
| <li><a class="dropdown-item" href="/docs/2.x/apidocs">Javadocs (2.x)</a></li> |
| <li><a class="dropdown-item" href="/api">Public API</a></li> |
| <li><a class="dropdown-item" href="/quickstart-1.x">Quickstart (1.x)</a></li> |
| <li><a class="dropdown-item" href="/accumulo2-maven-plugin">Accumulo Maven Plugin</a></li> |
| <li><a class="dropdown-item" href="/1.10/accumulo_user_manual.html">User Manual (1.10)</a></li> |
| <li><a class="dropdown-item" href="/1.10/apidocs">Javadocs (1.10)</a></li> |
| <li><a class="dropdown-item" href="/external-docs">External Docs</a></li> |
| <li><a class="dropdown-item" href="/docs-archive/">Archive</a></li> |
| </ul> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown">Community</a> |
| <ul class="dropdown-menu"> |
| <li><a class="dropdown-item" href="/contact-us">Contact Us</a></li> |
| <li><a class="dropdown-item" href="/how-to-contribute">How To Contribute</a></li> |
| <li><a class="dropdown-item" href="/people">People</a></li> |
| <li><a class="dropdown-item" href="/related-projects">Related Projects</a></li> |
| </ul> |
| </li> |
| <li class="nav-item"><a class="nav-link" href="/search">Search</a></li> |
| </ul> |
| <ul class="navbar-nav ms-auto"> |
| <li class="nav-item dropdown"> |
| <a class="nav-link dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown"> |
| <img alt="Apache Software Foundation" src="https://www.apache.org/foundation/press/kit/feather.svg" width="15"/> |
| </a> |
| <ul class="dropdown-menu"> |
| <li><a class="dropdown-item" href="https://www.apache.org">Apache Homepage <span class="fa-solid fa-up-right-from-square"></span></a></li> |
| <li><a class="dropdown-item" href="https://www.apache.org/licenses/">License <span class="fa-solid fa-up-right-from-square"></span></a></li> |
| <li><a class="dropdown-item" href="https://www.apache.org/foundation/sponsorship">Sponsorship <span class="fa-solid fa-up-right-from-square"></span></a></li> |
| <li><a class="dropdown-item" href="https://www.apache.org/security">Security <span class="fa-solid fa-up-right-from-square"></span></a></li> |
| <li><a class="dropdown-item" href="https://www.apache.org/foundation/thanks">Thanks <span class="fa-solid fa-up-right-from-square"></span></a></li> |
| <li><a class="dropdown-item" href="https://www.apache.org/foundation/policies/conduct">Code of Conduct <span class="fa-solid fa-up-right-from-square"></span></a></li> |
| <li><a class="dropdown-item" href="https://www.apache.org/foundation/policies/privacy.html">Privacy Policy<span class="fa-solid fa-up-right-from-square"></span></a></li> |
| <li><a class="dropdown-item" href="https://www.apache.org/events/current-event.html">Current Event <span class="fa-solid fa-up-right-from-square"></span></a></li> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="container"> |
| <div class="row"> |
| <div class="col-md-12"> |
| |
| <div id="non-canonical" style="display: none; background-color: #F0E68C; padding-left: 1em;"> |
| Visit the official site at: <a href="https://accumulo.apache.org">https://accumulo.apache.org</a> |
| </div> |
| <div id="content"> |
| |
| <div class="row"> |
| <div class="col-md-3"> |
| <div class="accordion sticky-top" id="myAccordion" style="top: 100px;"> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <div class="accordion-item"> |
| <div class="accordion-header fs-5 fw-bold" id="headinggetting-started"> |
| <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapsegetting-started" aria-expanded="false" aria-controls="collapsegetting-started"> |
| Getting started |
| </button> |
| </div> |
| <div id="collapsegetting-started" class="accordion-collapse collapse" aria-labelledby="headinggetting-started" data-bs-parent="#myAccordion"> |
| <div class="accordion-body"> |
| |
| |
| <div class="row"><a href="/docs/2.x/getting-started/quickstart">Setup</a></div> |
| |
| <div class="row"><a href="/docs/2.x/getting-started/design">Design</a></div> |
| |
| <div class="row"><a href="/docs/2.x/getting-started/features">Features</a></div> |
| |
| <div class="row"><a href="/docs/2.x/getting-started/clients">Accumulo Clients</a></div> |
| |
| <div class="row"><a href="/docs/2.x/getting-started/shell">Accumulo Shell</a></div> |
| |
| <div class="row"><a href="/docs/2.x/getting-started/table_design">Table Design</a></div> |
| |
| <div class="row"><a href="/docs/2.x/getting-started/table_configuration">Table Configuration</a></div> |
| |
| <div class="row"><a href="/docs/2.x/getting-started/glossary">Glossary</a></div> |
| |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <div class="accordion-item"> |
| <div class="accordion-header fs-5 fw-bold" id="headingdevelopment"> |
| <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapsedevelopment" aria-expanded="false" aria-controls="collapsedevelopment"> |
| Development |
| </button> |
| </div> |
| <div id="collapsedevelopment" class="accordion-collapse collapse" aria-labelledby="headingdevelopment" data-bs-parent="#myAccordion"> |
| <div class="accordion-body"> |
| |
| |
| <div class="row"><a href="/docs/2.x/development/iterators">Iterators</a></div> |
| |
| <div class="row"><a href="/docs/2.x/development/mapreduce">MapReduce</a></div> |
| |
| <div class="row"><a href="/docs/2.x/development/spark">Spark</a></div> |
| |
| <div class="row"><a href="/docs/2.x/development/development_tools">Development Tools</a></div> |
| |
| <div class="row"><a href="/docs/2.x/development/sampling">Sampling</a></div> |
| |
| <div class="row"><a href="/docs/2.x/development/summaries">Summary Statistics</a></div> |
| |
| <div class="row"><a href="/docs/2.x/development/proxy">Proxy</a></div> |
| |
| <div class="row"><a href="/docs/2.x/development/high_speed_ingest">High-Speed Ingest</a></div> |
| |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <div class="accordion-item"> |
| <div class="accordion-header fs-5 fw-bold" id="headingsecurity"> |
| <button class="accordion-button " type="button" data-bs-toggle="collapse" data-bs-target="#collapsesecurity" aria-expanded="true" aria-controls="collapsesecurity"> |
| Security |
| </button> |
| </div> |
| <div id="collapsesecurity" class="accordion-collapse collapse show" aria-labelledby="headingsecurity" data-bs-parent="#myAccordion"> |
| <div class="accordion-body"> |
| |
| |
| <div class="row"><a href="/docs/2.x/security/overview">Security Overview</a></div> |
| |
| <div class="row"><a href="/docs/2.x/security/authentication">Authentication</a></div> |
| |
| <div class="row"><a href="/docs/2.x/security/permissions">Permissions</a></div> |
| |
| <div class="row"><a href="/docs/2.x/security/authorizations">Authorizations</a></div> |
| |
| <div class="row selected"><a href="/docs/2.x/security/on-disk-encryption">On Disk Encryption</a></div> |
| |
| <div class="row"><a href="/docs/2.x/security/wire-encryption">Wire Encryption</a></div> |
| |
| <div class="row"><a href="/docs/2.x/security/kerberos">Kerberos</a></div> |
| |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <div class="accordion-item"> |
| <div class="accordion-header fs-5 fw-bold" id="headingconfiguration"> |
| <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapseconfiguration" aria-expanded="false" aria-controls="collapseconfiguration"> |
| Configuration |
| </button> |
| </div> |
| <div id="collapseconfiguration" class="accordion-collapse collapse" aria-labelledby="headingconfiguration" data-bs-parent="#myAccordion"> |
| <div class="accordion-body"> |
| |
| |
| <div class="row"><a href="/docs/2.x/configuration/overview">Configuration Overview</a></div> |
| |
| <div class="row"><a href="/docs/2.x/configuration/files">Configuration Files</a></div> |
| |
| <div class="row"><a href="/docs/2.x/configuration/client-properties">Client Properties (2.x)</a></div> |
| |
| <div class="row"><a href="/docs/2.x/configuration/client-properties3">Client Properties (3.x)</a></div> |
| |
| <div class="row"><a href="/docs/2.x/configuration/server-properties">Server Properties (2.x)</a></div> |
| |
| <div class="row"><a href="/docs/2.x/configuration/server-properties3">Server Properties (3.x)</a></div> |
| |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <div class="accordion-item"> |
| <div class="accordion-header fs-5 fw-bold" id="headingadministration"> |
| <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapseadministration" aria-expanded="false" aria-controls="collapseadministration"> |
| Administration |
| </button> |
| </div> |
| <div id="collapseadministration" class="accordion-collapse collapse" aria-labelledby="headingadministration" data-bs-parent="#myAccordion"> |
| <div class="accordion-body"> |
| |
| |
| <div class="row"><a href="/docs/2.x/administration/in-depth-install">In-depth Installation</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/monitoring-metrics">Monitoring & Metrics</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/fate">FATE</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/multivolume">Multi-Volume Installations</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/replication">Replication</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/caching">Caching</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/compaction">Compactions</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/upgrading">Upgrading Accumulo</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/scan-executors">Scan Executors</a></div> |
| |
| <div class="row"><a href="/docs/2.x/administration/erasure-coding">Erasure Coding</a></div> |
| |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <div class="accordion-item"> |
| <div class="accordion-header fs-5 fw-bold" id="headingtroubleshooting"> |
| <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapsetroubleshooting" aria-expanded="false" aria-controls="collapsetroubleshooting"> |
| Troubleshooting |
| </button> |
| </div> |
| <div id="collapsetroubleshooting" class="accordion-collapse collapse" aria-labelledby="headingtroubleshooting" data-bs-parent="#myAccordion"> |
| <div class="accordion-body"> |
| |
| |
| <div class="row"><a href="/docs/2.x/troubleshooting/basic">Basic Troubleshooting</a></div> |
| |
| <div class="row"><a href="/docs/2.x/troubleshooting/advanced">Advanced Troubleshooting</a></div> |
| |
| <div class="row"><a href="/docs/2.x/troubleshooting/tools">Troubleshooting Tools</a></div> |
| |
| <div class="row"><a href="/docs/2.x/troubleshooting/system-metadata-tables">System Metadata Tables</a></div> |
| |
| <div class="row"><a href="/docs/2.x/troubleshooting/performance">Performance</a></div> |
| |
| <div class="row"><a href="/docs/2.x/troubleshooting/tracing">Tracing</a></div> |
| |
| <div class="row"><a href="/docs/2.x/troubleshooting/zookeeper">ZooKeeper</a></div> |
| |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| </div> |
| </div> |
| <div class="col-md-9"> |
| |
| <p>Accumulo 2.x Documentation >> Security >> On Disk Encryption</p> |
| |
| |
| <div class="row mt-4"> |
| <div class="col-md-12 d-flex justify-content-between"> |
| <h1>On Disk Encryption</h1> |
| <a href="https://github.com/apache/accumulo-website/edit/main/_docs-2/security/on-disk-encryption.md" role="button"><span class="fa-solid fa-pen-to-square"></span> <small>Edit this page</small></a> |
| </div> |
| </div> |
| |
| <p>For an additional layer of security, Accumulo can encrypt files stored on-disk. On Disk encryption was reworked |
| for 2.0, making it easier to configure and more secure. Starting with 2.1, On Disk Encryption can now be configured |
| per table as well as for the entire instance (all tables). The files that can be encrypted include: <a href="/docs/2.x/getting-started/design#rfile">RFiles</a> and Write Ahead |
| Logs (WALs). NOTE: This feature is considered experimental and <a href="../administration/upgrading">upgrading</a> a previously encrypted instance |
| is not supported. For more information, see the <a href="#things-to-keep-in-mind">notes below</a>.</p> |
| |
| <h2 id="configuration">Configuration</h2> |
| |
| <p>To encrypt tables on-disk, encryption must be enabled before an Accumulo instance is initialized. This is |
| done by configuring a crypto service factory. If on-disk encryption is enabled on an existing cluster, only files |
| created after it is enabled will be encrypted and existing data won’t be encrypted until compaction.</p> |
| |
| <h3 id="encrypting-all-tables">Encrypting All Tables</h3> |
| |
| <p>To encrypt all tables, the generic crypto service factory can be used, <code class="language-plaintext highlighter-rouge">GenericCryptoServiceFactory</code>. This factory |
| is useful for general purpose on-disk encryption with no table context.</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>instance.crypto.opts.factory=org.apache.accumulo.core.spi.crypto.GenericCryptoServiceFactory |
| </code></pre></div></div> |
| |
| <p>The <code class="language-plaintext highlighter-rouge">GenericCryptoServiceFactory</code> requires configuring a crypto service to load and this can be done by setting the |
| <a href="/docs/2.x/configuration/server-properties#general_custom_crypto_service">general.custom.crypto.service</a> property. The value of this property is the |
| class name of the service which will perform crypto on RFiles and WALs.</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>general.custom.crypto.service=org.apache.accumulo.core.spi.crypto.AESCryptoService |
| </code></pre></div></div> |
| |
| <h3 id="per-table-encryption">Per Table Encryption</h3> |
| |
| <p>To encrypt per table, the per table crypto service factory can be used, <code class="language-plaintext highlighter-rouge">PerTableCryptoServiceFactory</code>. This factory |
| will load a crypto service configured by table.</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>instance.crypto.opts.factory=org.apache.accumulo.core.spi.crypto.PerTableCryptoServiceFactory |
| </code></pre></div></div> |
| |
| <p>The <code class="language-plaintext highlighter-rouge">PerTableCryptoServiceFactory</code> requires configuring a crypto service to load for the table RFiles and this can be done by adding the |
| <a href="/docs/2.x/configuration/server-properties#table_crypto_opts_service">table.crypto.opts.service</a> property to a table. Example in the accumulo shell:</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>createtable table1 -prop table.crypto.opts.service=org.apache.accumulo.core.spi.crypto.AESCryptoService |
| </code></pre></div></div> |
| <p>The <code class="language-plaintext highlighter-rouge">PerTableCryptoServiceFactory</code> also requires configuring a recovery and WAL crypto service by adding the following |
| properties to your <code class="language-plaintext highlighter-rouge">accumulo.properties</code> file.</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>general.custom.crypto.recovery.service=org.apache.accumulo.core.spi.crypto.AESCryptoService |
| general.custom.crypto.wal.service=org.apache.accumulo.core.spi.crypto.AESCryptoService |
| </code></pre></div></div> |
| |
| <p>Out of the box, Accumulo provides the <code class="language-plaintext highlighter-rouge">AESCryptoService</code> for basic encryption needs. This class provides AES encryption |
| with Galois/Counter Mode (GCM) for RFiles and Cipher Block Chaining (CBC) mode for WALs. The additional property |
| below is required by this crypto service to be set using the <a href="/docs/2.x/configuration/server-properties#general_custom_crypto_prefix">general.custom.crypto.*</a> prefix.</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>general.custom.crypto.key.uri=file:///secure/path/to/crypto-key-file |
| </code></pre></div></div> |
| <p>This property tells the crypto service where to find the file containing the key encryption key. The key file can be 16 or 32 bytes. |
| For example, openssl can be used to create a random 32 byte key:</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>openssl rand -out /path/to/keyfile 32 |
| </code></pre></div></div> |
| <p>Initializing Accumulo after these instance properties are set, will enable on-disk encryption across your entire cluster.</p> |
| |
| <h3 id="disabling-crypto">Disabling Crypto</h3> |
| |
| <p>When using the AESCryptoService, crypto can be disabled by setting the property <code class="language-plaintext highlighter-rouge">general.custom.crypto.enabled</code> to false. |
| However, this will disable all crypto as there is currently no way to disable only for specific tables. When disabled |
| existing encrypted files can still be read and scanned as long as the Accumulo instance and any table specific |
| properties are still configured but new files will not be encrypted.</p> |
| |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>general.custom.crypto.enabled=false |
| </code></pre></div></div> |
| |
| <h2 id="custom-crypto">Custom Crypto</h2> |
| |
| <p>The new crypto interface for 2.0 allows for easier custom implementation of encryption and decryption. Your |
| class only has to implement the <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.1.2/org/apache/accumulo/core/spi/crypto/CryptoService.html">CryptoService</a> interface to work with Accumulo. |
| The interface has 3 methods:</p> |
| <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kt">void</span> <span class="nf">init</span><span class="o">(</span><span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span><span class="nc">String</span><span class="o">></span> <span class="n">conf</span><span class="o">)</span> <span class="kd">throws</span> <span class="nc">CryptoException</span><span class="o">;</span> |
| <span class="nc">FileEncrypter</span> <span class="nf">getFileEncrypter</span><span class="o">(</span><span class="nc">CryptoEnvironment</span> <span class="n">environment</span><span class="o">);</span> |
| <span class="nc">FileDecrypter</span> <span class="nf">getFileDecrypter</span><span class="o">(</span><span class="nc">CryptoEnvironment</span> <span class="n">environment</span><span class="o">);</span> |
| </code></pre></div></div> |
| <p>The <code class="language-plaintext highlighter-rouge">init</code> method is where you will initialize any resources required for crypto and will get called once per Tablet Server. |
| The <code class="language-plaintext highlighter-rouge">getFileEncrypter</code> method requires implementation of a <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.1.2/org/apache/accumulo/core/spi/crypto/FileEncrypter.html">FileEncrypter</a> |
| for encryption and the <code class="language-plaintext highlighter-rouge">getFileDecrypter</code> method requires implementation of a <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.1.2/org/apache/accumulo/core/spi/crypto/FileDecrypter.html">FileDecrypter</a> |
| for decryption. The <code class="language-plaintext highlighter-rouge">CryptoEnvironment</code> passed into these methods will provide the scope of the crypto. |
| The FileEncrypter has two methods:</p> |
| <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nc">OutputStream</span> <span class="nf">encryptStream</span><span class="o">(</span><span class="nc">OutputStream</span> <span class="n">outputStream</span><span class="o">)</span> <span class="kd">throws</span> <span class="nc">CryptoService</span><span class="o">.</span><span class="na">CryptoException</span><span class="o">;</span> |
| <span class="kt">byte</span><span class="o">[]</span> <span class="nf">getDecryptionParameters</span><span class="o">();</span> |
| </code></pre></div></div> |
| <p>The <code class="language-plaintext highlighter-rouge">encryptStream</code> method performs the encryption on the provided OutputStream and returns an OutputStream, most likely |
| wrapped in at least one other OutputStream. The <code class="language-plaintext highlighter-rouge">getDecryptionParameters</code> returns a byte array of anything that will be |
| required to perform decryption. The FileDecrypter only has one method:</p> |
| <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nc">InputStream</span> <span class="nf">decryptStream</span><span class="o">(</span><span class="nc">InputStream</span> <span class="n">inputStream</span><span class="o">)</span> <span class="kd">throws</span> <span class="nc">CryptoService</span><span class="o">.</span><span class="na">CryptoException</span><span class="o">;</span> |
| </code></pre></div></div> |
| <p>For more help getting started see <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.1.2/org/apache/accumulo/core/security/crypto/impl/AESCryptoService.html">AESCryptoService</a>.</p> |
| |
| <h2 id="things-to-keep-in-mind">Things to keep in mind</h2> |
| |
| <h3 id="utilities-need-access-to-encryption-properties">Utilities need access to encryption properties</h3> |
| |
| <p>When utilities run that read encrypted files but do not connect to Zookeeper the utility needs to be provided |
| the encryption properties. For example, when using <a href="../troubleshooting/tools#rfileinfo">rfile-info</a> to examine |
| an encrypted rfile the accumulo.properties file can be copied, the necessary encryption parameters added, |
| and then the properties file can be passed to the utility with the <code class="language-plaintext highlighter-rouge">-p</code> argument.</p> |
| |
| <h3 id="some-data-will-be-unencrypted">Some data will be unencrypted</h3> |
| |
| <p>The on-disk encryption configured here is only for RFiles and Write Ahead Logs (WALs). The majority of data in Accumulo |
| is written to disk with these files, but there are a few scenarios that can take place where data will be unencrypted, |
| even with the crypto service enabled.</p> |
| |
| <h4 id="data-in-memory--logs">Data in Memory & Logs</h4> |
| |
| <p>For queries, data is decrypted when read from RFiles and cached in memory. This means that data is unencrypted in memory |
| while Accumulo is running. Depending on the situation, this also means that some data can be printed to logs. A stacktrace being logged |
| during an exception is one example. Accumulo developers have made sure not to expose data protected by authorizations during logging, but |
| it’s the additional data that gets encrypted on-disk that could be exposed in a log file.</p> |
| |
| <h4 id="bulk-import">Bulk Import</h4> |
| |
| <p>There are 2 ways to create RFiles for bulk ingest: with the <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.1.2/org/apache/accumulo/core/client/rfile/RFile.html">RFile API</a> and during Map Reduce using <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-hadoop-mapreduce/2.1.2/org/apache/accumulo/hadoop/mapreduce/AccumuloFileOutputFormat.html">AccumuloFileOutputFormat</a>. |
| The <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.1.2/org/apache/accumulo/core/client/rfile/RFile.html">RFile API</a> allows passing in the configuration properties for encryption mentioned above. The <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-hadoop-mapreduce/2.1.2/org/apache/accumulo/hadoop/mapreduce/AccumuloFileOutputFormat.html">AccumuloFileOutputFormat</a> does |
| not allow for encryption of RFiles so any data bulk imported through this process will be unencrypted.</p> |
| |
| <h4 id="zookeeper">Zookeeper</h4> |
| |
| <p>Accumulo stores a lot of metadata about the cluster in Zookeeper. Keep in mind that this metadata does not get encrypted with On Disk encryption enabled.</p> |
| |
| <h2 id="gcm-performance">GCM performance</h2> |
| |
| <p>The AESCryptoService uses GCM mode for RFiles. <a href="https://openjdk.java.net/jeps/246">Java 9 introduced GHASH hardware support used by GCM.</a></p> |
| |
| <p>A test was performed on a VM with 4 2.3GHz processors and 16GB of RAM. The test encrypted and decrypted arrays of size 131072 bytes 1000000 times. The results are as follows:</p> |
| |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Java 9 GCM times: |
| Time spent encrypting: 209.210s |
| Time spent decrypting: 276.800s |
| Java 8 GCM times: |
| Time spent encrypting: 2,818.440s |
| Time spent decrypting: 2,883.960s |
| </code></pre></div></div> |
| |
| <p>As you can see, there is a significant performance hit when running without the GHASH CPU instruction. It is advised Java 9 or later be used when enabling encryption.</p> |
| |
| |
| <div class="row mt-4"> |
| <div class="col-md-12 d-flex justify-content-between"> |
| <strong>Find documentation for all releases in the <a href="/docs-archive">archive</strong> |
| <a href="https://github.com/apache/accumulo-website/edit/main/_docs-2/security/on-disk-encryption.md" role="button"><span class="fa-solid fa-pen-to-square"></span> <small>Edit this page</small></a> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| </div> |
| |
| |
| <footer> |
| |
| <p><a href="https://www.apache.org/foundation/contributing"><img src="https://www.apache.org/images/SupportApache-small.png" alt="Support the ASF" id="asf-logo" height="100" /></a></p> |
| |
| <p>Copyright © 2011-2024 <a href="https://www.apache.org">The Apache Software Foundation</a>. |
| Licensed under the <a href="https://www.apache.org/licenses/">Apache License, Version 2.0</a>.</p> |
| |
| <p>Apache®, the names of Apache projects and their logos, and the multicolor feather |
| logo are registered trademarks or trademarks of The Apache Software Foundation |
| in the United States and/or other countries.</p> |
| |
| </footer> |
| |
| |
| </div> |
| </div> |
| </div> |
| </body> |
| </html> |
