blob: 6ddc11c928b09448ef1dad6b2cb7e5f80787f186 [file] [log] [blame]
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 1.5.8">
<meta name="author" content="Apache NiFi Team">
<title>NiFi System Administrator&#8217;s Guide</title>
<style>
/* Asciidoctor default stylesheet | MIT License | http://asciidoctor.org */
/* Copyright (C) 2012-2015 Dan Allen, Ryan Waldron and the Asciidoctor Project
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE. */
/* Remove the comments around the @import statement below when using this as a custom stylesheet */
@import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400";
article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section,summary{display:block}
audio,canvas,video{display:inline-block}
audio:not([controls]){display:none;height:0}
[hidden],template{display:none}
script{display:none!important}
html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%}
body{margin:0}
a{background:transparent}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
abbr[title]{border-bottom:1px dotted}
b,strong{font-weight:bold}
dfn{font-style:italic}
hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type="button"],input[type="reset"],input[type="submit"]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type="checkbox"],input[type="radio"]{box-sizing:border-box;padding:0}
input[type="search"]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box}
input[type="search"]::-webkit-search-cancel-button,input[type="search"]::-webkit-search-decoration{-webkit-appearance:none}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,*:before,*:after{-moz-box-sizing:border-box;-webkit-box-sizing:border-box;box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;font-weight:400;font-style:normal;line-height:1;position:relative;cursor:auto}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
#map_canvas img,#map_canvas embed,#map_canvas object,.map_canvas img,.map_canvas embed,.map_canvas object{max-width:none!important}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
.antialiased,body{-webkit-font-smoothing:antialiased}
img{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
p.lead,.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{font-size:1.21875em;line-height:1.6}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0;direction:ltr}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:none}
p{font-family:inherit;font-weight:400;font-size:1em;line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #ddddd8;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em;height:0}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9);padding-right: 1px;}
ul,ol,dl{font-size:1em;line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol,ul.no-bullet,ol.no-bullet{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0;font-size:1em}
ul.square li ul,ul.circle li ul,ul.disc li ul{list-style:inherit}
ul.square{list-style-type:square}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ul.no-bullet{list-style:none}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
abbr,acronym{text-transform:uppercase;font-size:90%;color:rgba(0,0,0,.8);border-bottom:1px dotted #ddd;cursor:help}
abbr{text-transform:none}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote cite{display:block;font-size:.9375em;color:rgba(0,0,0,.6)}
blockquote cite:before{content:"\2014 \0020"}
blockquote cite a,blockquote cite a:visited{color:rgba(0,0,0,.6)}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media only screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}table{background:#fff;margin-bottom:1.25em;border:solid 1px #dedede}
table thead,table tfoot{background:#f7f8f7;font-weight:bold}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt,table tr:nth-of-type(even){background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{display:table-cell;line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.clearfix:before,.clearfix:after,.float-group:before,.float-group:after{content:" ";display:table}
.clearfix:after,.float-group:after{clear:both}
*:not(pre)>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;word-spacing:-.15em;background-color:#f7f7f8;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
pre,pre>code{line-height:1.45;color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;text-rendering:optimizeSpeed}
.keyseq{color:rgba(51,51,51,.8)}
kbd{display:inline-block;color:rgba(0,0,0,.8);font-size:.75em;line-height:1.4;background-color:#f7f7f7;border:1px solid #ccc;-webkit-border-radius:3px;border-radius:3px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em white inset;box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em #fff inset;margin:-.15em .15em 0 .15em;padding:.2em .6em .2em .5em;vertical-align:middle;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menu{color:rgba(0,0,0,.8)}
b.button:before,b.button:after{position:relative;top:-1px;font-weight:400}
b.button:before{content:"[";padding:0 3px 0 2px}
b.button:after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin-left:auto;margin-right:auto;margin-top:0;margin-bottom:0;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header:before,#header:after,#content:before,#content:after,#footnotes:before,#footnotes:after,#footer:before,#footer:after{content:" ";display:table}
#header:after,#content:after,#footnotes:after,#footer:after{clear:both}
#content{margin-top:1.25em}
#content:before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #ddddd8}
#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #ddddd8;padding-bottom:8px}
#header .details{border-bottom:1px solid #ddddd8;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:-ms-flexbox;display:-webkit-flex;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span:before{content:"\00a0\2013\00a0"}
#header .details br+span.author:before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark:before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber:after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #ddddd8;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #efefed;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media only screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
#toc.toc2{margin-top:0!important;background-color:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #efefed;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #efefed;left:auto;right:0}}@media only screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}#content #toc{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:100%;background-color:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:rgba(255,255,255,.8);line-height:1.44}
.sect1{padding-bottom:.625em}
@media only screen and (min-width:768px){.sect1{padding-bottom:1.25em}}.sect1+.sect1{border-top:1px solid #efefed}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor:before,h2>a.anchor:before,h3>a.anchor:before,#toctitle>a.anchor:before,.sidebarblock>.content>.title>a.anchor:before,h4>a.anchor:before,h5>a.anchor:before,h6>a.anchor:before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock>caption.title{white-space:nowrap;overflow:visible;max-width:0}
.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{color:rgba(0,0,0,.85)}
table.tableblock #preamble>.sectionbody>.paragraph:first-of-type p{font-size:inherit}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:none}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #ddddd8;color:rgba(0,0,0,.6)}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border-style:solid;border-width:1px;border-color:#e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;-webkit-border-radius:4px;border-radius:4px}
.exampleblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child{margin-bottom:0}
.sidebarblock{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px}
.sidebarblock>:first-child{margin-top:0}
.sidebarblock>:last-child{margin-bottom:0}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock pre:not(.highlight),.listingblock pre[class="highlight"],.listingblock pre[class^="highlight "],.listingblock pre.CodeRay,.listingblock pre.prettyprint{background:#f7f7f8}
.sidebarblock .literalblock pre,.sidebarblock .listingblock pre:not(.highlight),.sidebarblock .listingblock pre[class="highlight"],.sidebarblock .listingblock pre[class^="highlight "],.sidebarblock .listingblock pre.CodeRay,.sidebarblock .listingblock pre.prettyprint{background:#f2f1f1}
.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{-webkit-border-radius:4px;border-radius:4px;word-wrap:break-word;padding:1em;font-size:.8125em}
.literalblock pre.nowrap,.literalblock pre[class].nowrap,.listingblock pre.nowrap,.listingblock pre[class].nowrap{overflow-x:auto;white-space:pre;word-wrap:normal}
@media only screen and (min-width:768px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:.90625em}}@media only screen and (min-width:1280px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:1em}}.literalblock.output pre{color:#f7f7f8;background-color:rgba(0,0,0,.9)}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;-webkit-border-radius:4px;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.listingblock>.content{position:relative}
.listingblock code[data-lang]:before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:#999}
.listingblock:hover code[data-lang]:before{display:block}
.listingblock.terminal pre .command:before{content:attr(data-prompt);padding-right:.5em;color:#999}
.listingblock.terminal pre .command:not([data-prompt]):before{content:"$"}
table.pyhltable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.pyhltable td{vertical-align:top;padding-top:0;padding-bottom:0}
table.pyhltable td.code{padding-left:.75em;padding-right:0}
pre.pygments .lineno,table.pyhltable td:not(.code){color:#999;padding-left:0;padding-right:.5em;border-right:1px solid #ddddd8}
pre.pygments .lineno{display:inline-block;margin-right:.25em}
table.pyhltable .linenodiv{background:none!important;padding-right:0!important}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock blockquote p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote:before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.5em;margin-right:.5ex;text-align:right}
.quoteblock .quoteblock{margin-left:0;margin-right:0;padding:.5em 0;border-left:3px solid rgba(0,0,0,.6)}
.quoteblock .quoteblock blockquote{padding:0 0 0 .75em}
.quoteblock .quoteblock blockquote:before{display:none}
.verseblock{margin:0 1em 1.25em 1em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.05em;color:rgba(0,0,0,.6)}
.quoteblock.abstract{margin:0 0 1.25em 0;display:block}
.quoteblock.abstract blockquote,.quoteblock.abstract blockquote p{text-align:left;word-spacing:0}
.quoteblock.abstract blockquote:before,.quoteblock.abstract blockquote p:first-of-type:before{display:none}
table.tableblock{max-width:100%;border-collapse:separate}
table.tableblock td>.paragraph:last-child p>p:last-child,table.tableblock th>p:last-child,table.tableblock td>p:last-child{margin-bottom:0}
table.spread{width:100%}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all th.tableblock,table.grid-all td.tableblock{border-width:0 1px 1px 0}
table.grid-all tfoot>tr>th.tableblock,table.grid-all tfoot>tr>td.tableblock{border-width:1px 1px 0 0}
table.grid-cols th.tableblock,table.grid-cols td.tableblock{border-width:0 1px 0 0}
table.grid-all *>tr>.tableblock:last-child,table.grid-cols *>tr>.tableblock:last-child{border-right-width:0}
table.grid-rows th.tableblock,table.grid-rows td.tableblock{border-width:0 0 1px 0}
table.grid-all tbody>tr:last-child>th.tableblock,table.grid-all tbody>tr:last-child>td.tableblock,table.grid-all thead:last-child>tr>th.tableblock,table.grid-rows tbody>tr:last-child>th.tableblock,table.grid-rows tbody>tr:last-child>td.tableblock,table.grid-rows thead:last-child>tr>th.tableblock{border-bottom-width:0}
table.grid-rows tfoot>tr>th.tableblock,table.grid-rows tfoot>tr>td.tableblock{border-width:1px 0 0 0}
table.frame-all{border-width:1px}
table.frame-sides{border-width:0 1px}
table.frame-topbot{border-width:1px 0}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{display:table-cell;line-height:1.6;background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
td>div.verse{white-space:pre}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
ol>li p,ul>li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.unstyled,ol.unnumbered,ul.checklist,ul.none{list-style-type:none}
ul.unstyled,ol.unnumbered,ul.checklist{margin-left:.625em}
ul.checklist li>p:first-child>.fa-square-o:first-child,ul.checklist li>p:first-child>.fa-check-square-o:first-child{width:1em;font-size:.85em}
ul.checklist li>p:first-child>input[type="checkbox"]:first-child{width:1em;position:relative;top:1px}
ul.inline{margin:0 auto .625em auto;margin-left:-1.375em;margin-right:0;padding:0;list-style:none;overflow:hidden}
ul.inline>li{list-style:none;float:left;margin-left:1.375em;display:block}
ul.inline>li>*{display:block}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1{padding-right:.75em;font-weight:bold}
td.hdlist1,td.hdlist2{vertical-align:top}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist>table tr>td:first-of-type{padding:0 .75em;line-height:1}
.colist>table tr>td:last-of-type{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:solid 4px #fff;-webkit-box-shadow:0 0 0 1px #ddd;box-shadow:0 0 0 1px #ddd}
.imageblock.left,.imageblock[style*="float: left"]{margin:.25em .625em 1.25em 0}
.imageblock.right,.imageblock[style*="float: right"]{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none}
span.footnote,span.footnoteref{vertical-align:super;font-size:.875em}
span.footnote a,span.footnoteref a{text-decoration:none}
span.footnote a:active,span.footnoteref a:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em 0;border-width:1px 0 0 0}
#footnotes .footnote{padding:0 .375em;line-height:1.3;font-size:.875em;margin-left:1.2em;text-indent:-1.2em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
.gist .file-data>table{border:0;background:#fff;width:100%;margin-bottom:0}
.gist .file-data>table td.line-data{width:99%}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background-color:#00fafa}
.black{color:#000}
.black-background{background-color:#000}
.blue{color:#0000bf}
.blue-background{background-color:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background-color:#fa00fa}
.gray{color:#606060}
.gray-background{background-color:#7d7d7d}
.green{color:#006000}
.green-background{background-color:#007d00}
.lime{color:#00bf00}
.lime-background{background-color:#00fa00}
.maroon{color:#600000}
.maroon-background{background-color:#7d0000}
.navy{color:#000060}
.navy-background{background-color:#00007d}
.olive{color:#606000}
.olive-background{background-color:#7d7d00}
.purple{color:#600060}
.purple-background{background-color:#7d007d}
.red{color:#bf0000}
.red-background{background-color:#fa0000}
.silver{color:#909090}
.silver-background{background-color:#bcbcbc}
.teal{color:#006060}
.teal-background{background-color:#007d7d}
.white{color:#bfbfbf}
.white-background{background-color:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background-color:#fafa00}
span.icon>.fa{cursor:default}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note:before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip:before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning:before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution:before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important:before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background-color:rgba(0,0,0,.8);-webkit-border-radius:100px;border-radius:100px;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]:after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
h1,h2{letter-spacing:-.01em}
dt,th.tableblock,td.content{text-rendering:optimizeLegibility}
p,td.content{letter-spacing:-.01em}
p strong,td.content strong{letter-spacing:-.005em}
p,blockquote,dt,td.content{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background-color:#fffef7;border-color:#e0e0dc;-webkit-box-shadow:0 1px 4px #e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@media print{@page{margin:1.25cm .75cm}
*{-webkit-box-shadow:none!important;box-shadow:none!important;text-shadow:none!important}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare):after,a[href^="https:"]:not(.bare):after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]:after{content:" (" attr(title) ")"}
pre,blockquote,tr,img{page-break-inside:avoid}
thead{display:table-header-group}
img{max-width:100%!important}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #ddddd8!important;padding-bottom:0!important}
.sect1{padding-bottom:0!important}
.sect1+.sect1{border:0!important}
#header>h1:first-child{margin-top:1.25rem}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em 0}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span:before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]:before{display:block}
#footer{background:none!important;padding:0 .9375em}
#footer-text{color:rgba(0,0,0,.6)!important;font-size:.9em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
</style>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
</head>
<body class="article">
<div id="header">
<h1>NiFi System Administrator&#8217;s Guide</h1>
<div class="details">
<span id="author" class="author">Apache NiFi Team</span><br>
<span id="email" class="email"><a href="mailto:dev@nifi.apache.org">dev@nifi.apache.org</a></span><br>
</div>
<div id="toc" class="toc">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="administration-guide.html#system_requirements">System Requirements</a></li>
<li><a href="administration-guide.html#how-to-install-and-start-nifi">How to install and start NiFi</a></li>
<li><a href="administration-guide.html#build-a-custom-distribution">Build a Custom Distribution</a></li>
<li><a href="administration-guide.html#port-configuration">Port Configuration</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#nifi">NiFi</a></li>
<li><a href="administration-guide.html#embedded-zookeeper">Embedded ZooKeeper</a></li>
</ul>
</li>
<li><a href="administration-guide.html#configuration-best-practices">Configuration Best Practices</a></li>
<li><a href="administration-guide.html#recommended-antivirus-exclusions">Recommended Antivirus Exclusions</a></li>
<li><a href="administration-guide.html#logging_configuration">Logging Configuration</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#standard-log-files">Standard Log Files</a></li>
<li><a href="administration-guide.html#deprecation-logging">Deprecation Logging</a></li>
</ul>
</li>
<li><a href="administration-guide.html#security_configuration">Security Configuration</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#tls_generation_toolkit">TLS Generation Toolkit</a></li>
<li><a href="administration-guide.html#tls_cipher_suites">TLS Cipher Suites</a></li>
</ul>
</li>
<li><a href="administration-guide.html#user_authentication">User Authentication</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#single_user_identity_provider">Single User</a></li>
<li><a href="administration-guide.html#ldap_login_identity_provider">Lightweight Directory Access Protocol (LDAP)</a></li>
<li><a href="administration-guide.html#kerberos_login_identity_provider">Kerberos</a></li>
<li><a href="administration-guide.html#openid_connect">OpenId Connect</a></li>
<li><a href="administration-guide.html#saml">SAML</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#saml-rest-resources">SAML REST Resources</a></li>
</ul>
</li>
<li><a href="administration-guide.html#apache_knox">Apache Knox</a></li>
<li><a href="administration-guide.html#json_web_token">JSON Web Tokens</a></li>
</ul>
</li>
<li><a href="administration-guide.html#multi-tenant-authorization">Multi-Tenant Authorization</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#authorizer-configuration">Authorizer Configuration</a></li>
<li><a href="administration-guide.html#authorizers-setup">Authorizers.xml Setup</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#fileusergroupprovider">FileUserGroupProvider</a></li>
<li><a href="administration-guide.html#ldapusergroupprovider">LdapUserGroupProvider</a></li>
<li><a href="administration-guide.html#shellusergroupprovider">ShellUserGroupProvider</a></li>
<li><a href="administration-guide.html#azuregraphusergroupprovider">AzureGraphUserGroupProvider</a></li>
<li><a href="administration-guide.html#composite-implementations">Composite Implementations</a></li>
<li><a href="administration-guide.html#fileaccesspolicyprovider">FileAccessPolicyProvider</a></li>
<li><a href="administration-guide.html#standardmanagedauthorizer">StandardManagedAuthorizer</a></li>
<li><a href="administration-guide.html#fileauthorizer">FileAuthorizer</a></li>
<li><a href="administration-guide.html#initial-admin-identity">Initial Admin Identity (New NiFi Instance)</a></li>
<li><a href="administration-guide.html#legacy-authorized-users">Legacy Authorized Users (NiFi Instance Upgrade)</a></li>
<li><a href="administration-guide.html#cluster-node-identities">Cluster Node Identities</a></li>
</ul>
</li>
<li><a href="administration-guide.html#config-users-access-policies">Configuring Users &amp; Access Policies</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#creating-users-groups">Creating Users and Groups</a></li>
<li><a href="administration-guide.html#access-policies">Access Policies</a></li>
<li><a href="administration-guide.html#viewing-policies-users">Viewing Policies on Users</a></li>
<li><a href="administration-guide.html#access-policy-config-examples">Access Policy Configuration Examples</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="administration-guide.html#repository-encryption">Repository Encryption</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#configuration-change-considerations">Configuration Change Considerations</a></li>
<li><a href="administration-guide.html#repository-encryption-protocol-version-1">Repository Encryption Protocol Version 1</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#encryption-metadata-serialization">Encryption Metadata Serialization</a></li>
</ul>
</li>
<li><a href="administration-guide.html#repository-encryption-configuration">Repository Encryption Configuration</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#protocol-version-configuration">Protocol Version Configuration</a></li>
<li><a href="administration-guide.html#key-provider-configuration">Key Provider Configuration</a></li>
<li><a href="administration-guide.html#secret-key-generation-and-storage-using-keytool">Secret Key Generation and Storage using Keytool</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="administration-guide.html#encryption">Encryption Configuration</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#key-derivation-functions">Key Derivation Functions</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#nifi-legacy-kdf">NiFi Legacy KDF</a></li>
<li><a href="administration-guide.html#openssl-pkcs5-evp-bytestokey">OpenSSL PKCS#5 v1.5 EVP_BytesToKey</a></li>
<li><a href="administration-guide.html#bcrypt-kdf">Bcrypt</a></li>
<li><a href="administration-guide.html#scrypt-kdf">Scrypt</a></li>
<li><a href="administration-guide.html#pbkdf2-kdf">PBKDF2</a></li>
<li><a href="administration-guide.html#none-kdf">None</a></li>
<li><a href="administration-guide.html#argon2-kdf">Argon2</a></li>
</ul>
</li>
<li><a href="administration-guide.html#salt-and-iv-encoding">Salt and IV Encoding</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#nifi-legacy">NiFi Legacy</a></li>
<li><a href="administration-guide.html#openssl-pkcs5-v1-5-evp_bytestokey">OpenSSL PKCS#5 v1.5 EVP_BytesToKey</a></li>
<li><a href="administration-guide.html#bcrypt-scrypt-pbkdf2-argon2">Bcrypt, Scrypt, PBKDF2, Argon2</a></li>
</ul>
</li>
<li><a href="administration-guide.html#java-cryptography-extension-jce-limited-strength-jurisdiction-policies">Java Cryptography Extension (JCE) Limited Strength Jurisdiction Policies</a></li>
<li><a href="administration-guide.html#allow-insecure-cryptographic-modes">Allow Insecure Cryptographic Modes</a></li>
</ul>
</li>
<li><a href="administration-guide.html#nifi_sensitive_props_key">Encrypted Passwords in Flows</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#property-encryption-algorithms">Property Encryption Algorithms</a></li>
</ul>
</li>
<li><a href="administration-guide.html#encrypt-config_tool">Encrypted Passwords in Configuration Files</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#hashicorp-vault-providers">HashiCorp Vault providers</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#required-properties">Required properties</a></li>
<li><a href="administration-guide.html#optional-properties">Optional properties</a></li>
</ul>
</li>
<li><a href="administration-guide.html#aws-kms-provider">AWS KMS provider</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#required-properties-2">Required properties</a></li>
<li><a href="administration-guide.html#optional-properties-2">Optional properties</a></li>
</ul>
</li>
<li><a href="administration-guide.html#aws-secrets-manager-provider">AWS Secrets Manager provider</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#optional-properties-3">Optional properties</a></li>
</ul>
</li>
<li><a href="administration-guide.html#azure-key-vault-key-provider">Azure Key Vault Key Provider</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#required-properties-3">Required properties</a></li>
</ul>
</li>
<li><a href="administration-guide.html#azure-key-vault-secret-provider">Azure Key Vault Secret Provider</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#required-properties-4">Required properties</a></li>
</ul>
</li>
<li><a href="administration-guide.html#google-cloud-kms-provider">Google Cloud KMS provider</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#required-properties-5">Required properties</a></li>
</ul>
</li>
<li><a href="administration-guide.html#property-context-mapping">Property Context Mapping</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#example">Example</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="administration-guide.html#admin-toolkit">NiFi Toolkit Administrative Tools</a></li>
<li><a href="administration-guide.html#clustering">Clustering Configuration</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#zero-leader-clustering">Zero-Leader Clustering</a></li>
<li><a href="administration-guide.html#why-cluster">Why Cluster?</a></li>
<li><a href="administration-guide.html#terminology">Terminology</a></li>
<li><a href="administration-guide.html#communication-within-the-cluster">Communication within the Cluster</a></li>
<li><a href="administration-guide.html#managing_nodes">Managing Nodes</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#disconnect-nodes">Disconnect Nodes</a></li>
<li><a href="administration-guide.html#offload-nodes">Offload Nodes</a></li>
<li><a href="administration-guide.html#delete-nodes">Delete Nodes</a></li>
<li><a href="administration-guide.html#decommission-nodes">Decommission Nodes</a></li>
<li><a href="administration-guide.html#nifi-cli-node-commands">NiFi CLI Node Commands</a></li>
</ul>
</li>
<li><a href="administration-guide.html#flow-election">Flow Election</a></li>
<li><a href="administration-guide.html#basic-cluster-setup">Basic Cluster Setup</a></li>
<li><a href="administration-guide.html#cluster_firewall_configuration">Cluster Firewall Configuration</a></li>
<li><a href="administration-guide.html#troubleshooting">Troubleshooting</a></li>
</ul>
</li>
<li><a href="administration-guide.html#state_management">State Management</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#state_providers">Configuring State Providers</a></li>
<li><a href="administration-guide.html#embedded_zookeeper">Embedded ZooKeeper Server</a></li>
<li><a href="administration-guide.html#zk_access_control">ZooKeeper Access Control</a></li>
<li><a href="administration-guide.html#securing_zookeeper">Securing ZooKeeper with Kerberos</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#zk_kerberos_server">Kerberizing Embedded ZooKeeper Server</a></li>
<li><a href="administration-guide.html#zk_kerberos_client">Kerberizing NiFi&#8217;s ZooKeeper Client</a></li>
<li><a href="administration-guide.html#troubleshooting_kerberos">Troubleshooting Kerberos Configuration</a></li>
</ul>
</li>
<li><a href="administration-guide.html#zk_tls_client">Securing ZooKeeper with TLS</a></li>
<li><a href="administration-guide.html#embedded-zookeeper-with-tls">Embedded ZooKeeper with TLS</a></li>
<li><a href="administration-guide.html#zookeeper_migrator">ZooKeeper Migrator</a></li>
</ul>
</li>
<li><a href="administration-guide.html#bootstrap_properties">Bootstrap Properties</a></li>
<li><a href="administration-guide.html#notification_services">Notification Services</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#email-notification-service">Email Notification Service<br></a></li>
<li><a href="administration-guide.html#http-notification-service">HTTP Notification Service<br></a></li>
</ul>
</li>
<li><a href="administration-guide.html#proxy_configuration">Proxy Configuration</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#session_affinity">Session Affinity</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#session-affinity-configuration">Session Affinity Configuration</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="administration-guide.html#kerberos_service">Kerberos Service</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#kerberos_service_notes">Notes</a></li>
</ul>
</li>
<li><a href="administration-guide.html#analytics_framework">Analytics Framework</a></li>
<li><a href="administration-guide.html#system_properties">System Properties</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#upgrade-recommendations">Upgrade Recommendations</a></li>
<li><a href="administration-guide.html#core-properties">Core Properties<br></a></li>
<li><a href="administration-guide.html#state-management">State Management<br></a></li>
<li><a href="administration-guide.html#h2-settings">H2 Settings</a></li>
<li><a href="administration-guide.html#repository-encryption-properties">Repository Encryption</a></li>
<li><a href="administration-guide.html#flowfile-repository">FlowFile Repository</a></li>
<li><a href="administration-guide.html#write-ahead-flowfile-repository">Write Ahead FlowFile Repository</a></li>
<li><a href="administration-guide.html#encrypted-write-ahead-flowfile-repository-properties">Encrypted Write Ahead FlowFile Repository Properties</a></li>
<li><a href="administration-guide.html#volatile-flowfile-repository">Volatile FlowFile Repository</a></li>
<li><a href="administration-guide.html#rocksdb-flowfile-repository">RocksDB FlowFile Repository</a></li>
<li><a href="administration-guide.html#swap-management">Swap Management</a></li>
<li><a href="administration-guide.html#content-repository">Content Repository</a></li>
<li><a href="administration-guide.html#file-system-content-repository-properties">File System Content Repository Properties</a></li>
<li><a href="administration-guide.html#encrypted-file-system-content-repository-properties">Encrypted File System Content Repository Properties</a></li>
<li><a href="administration-guide.html#provenance-repository">Provenance Repository</a></li>
<li><a href="administration-guide.html#write-ahead-provenance-repository-properties">Write Ahead Provenance Repository Properties</a></li>
<li><a href="administration-guide.html#encrypted-write-ahead-provenance-repository-properties">Encrypted Write Ahead Provenance Repository Properties</a></li>
<li><a href="administration-guide.html#persistent-provenance-repository-properties">Persistent Provenance Repository Properties</a></li>
<li><a href="administration-guide.html#volatile-provenance-repository-properties">Volatile Provenance Repository Properties</a></li>
<li><a href="administration-guide.html#status-history-repository">Status History Repository</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#in-memory-repository">In memory repository</a></li>
<li><a href="administration-guide.html#persistent-repository">Persistent repository</a></li>
</ul>
</li>
<li><a href="administration-guide.html#site_to_site_properties">Site to Site Properties</a></li>
<li><a href="administration-guide.html#site_to_site_reverse_proxy_properties">Site to Site Routing Properties for Reverse Proxies</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#site-to-site-protocol-sequence">Site to Site protocol sequence</a></li>
<li><a href="administration-guide.html#reverse-proxy-configurations">Reverse Proxy Configurations</a></li>
<li><a href="administration-guide.html#site-to-site-and-reverse-proxy-examples">Site to Site and Reverse Proxy Examples</a></li>
</ul>
</li>
<li><a href="administration-guide.html#web-properties">Web Properties</a></li>
<li><a href="administration-guide.html#security_properties">Security Properties</a></li>
<li><a href="administration-guide.html#bootstrap-properties">Bootstrap Properties</a></li>
<li><a href="administration-guide.html#identity-mapping-properties">Identity Mapping Properties</a></li>
<li><a href="administration-guide.html#cluster_common_properties">Cluster Common Properties</a></li>
<li><a href="administration-guide.html#cluster_node_properties">Cluster Node Properties</a></li>
<li><a href="administration-guide.html#zookeeper-properties">ZooKeeper Properties</a></li>
<li><a href="administration-guide.html#kerberos_properties">Kerberos Properties</a></li>
<li><a href="administration-guide.html#analytics_properties">Analytics Properties</a></li>
<li><a href="administration-guide.html#runtime_monitoring_properties">Runtime Monitoring Properties</a></li>
<li><a href="administration-guide.html#performance_tracking_properties">Performance Tracking Properties</a></li>
<li><a href="administration-guide.html#custom_properties">Custom Properties</a></li>
<li><a href="administration-guide.html#nar_provider_properties">NAR Provider Properties</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#hdfs-nar-provider">HDFS NAR Provider</a></li>
<li><a href="administration-guide.html#nifi-registry-nar-provider">NiFi Registry NAR Provider</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="administration-guide.html#upgrading_nifi">Upgrading NiFi</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#preserve-custom-processors">Preserve Custom Processors</a></li>
<li><a href="administration-guide.html#preserve-modified-nars">Preserve Modified NARs</a></li>
<li><a href="administration-guide.html#clear-activity-and-shutdown-existing-nifi">Clear Activity and Shutdown Existing NiFi</a></li>
<li><a href="administration-guide.html#install-the-new-nifi-version">Install the new NiFi Version</a></li>
<li><a href="administration-guide.html#update-the-configuration-files-for-your-new-nifi-installation">Update the Configuration Files for Your New NiFi Installation</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#sensitive_flow_migration">Migrating a Flow with Sensitive Properties</a></li>
<li><a href="administration-guide.html#updating-the-sensitive-properties-algorithm">Updating the Sensitive Properties Algorithm</a></li>
<li><a href="administration-guide.html#updating-the-sensitive-properties-key">Updating the Sensitive Properties Key</a></li>
</ul>
</li>
<li><a href="administration-guide.html#start-new-nifi">Start New NiFi</a></li>
</ul>
</li>
<li><a href="administration-guide.html#processor-locations">Processor Locations</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#processor-location-options">Available Configuration Options</a></li>
<li><a href="administration-guide.html#installing-custom-processors">Installing Custom Processors</a></li>
<li><a href="administration-guide.html#autoloading-processors">Autoloading Custom Processors</a></li>
<li><a href="administration-guide.html#nar-providers">NAR Providers</a>
<ul class="sectlevel3">
<li><a href="administration-guide.html#hdfs-external-resource-provider">HDFS External Resource Provider</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="administration-guide.html#nifi_diagnostics">NiFi diagnostics</a>
<ul class="sectlevel2">
<li><a href="administration-guide.html#automatic-diagnostics-on-restart-and-shutdown">Automatic diagnostics on restart and shutdown</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="system_requirements"><a class="anchor" href="administration-guide.html#system_requirements"></a>System Requirements</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Apache NiFi can run on something as simple as a laptop, but it can also be clustered across many enterprise-class servers. Therefore, the amount of hardware and memory needed will depend on the size and nature of the dataflow involved. The data is stored on disk while NiFi is processing it. So NiFi needs to have sufficient disk space allocated for its various repositories, particularly the content repository, flowfile repository, and provenance repository (see the <a href="administration-guide.html#system_properties">System Properties</a> section for more information about these repositories). NiFi has the following minimum system requirements:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Requires Java 8 or Java 11</p>
</li>
<li>
<p>Supported Operating Systems:</p>
<div class="ulist">
<ul>
<li>
<p>Linux</p>
</li>
<li>
<p>Unix</p>
</li>
<li>
<p>Windows</p>
</li>
<li>
<p>macOS</p>
</li>
</ul>
</div>
</li>
<li>
<p>Supported Web Browsers:</p>
<div class="ulist">
<ul>
<li>
<p>Microsoft Edge: Current &amp; (Current - 1)</p>
</li>
<li>
<p>Mozilla FireFox: Current &amp; (Current - 1)</p>
</li>
<li>
<p>Google Chrome: Current &amp; (Current - 1)</p>
</li>
<li>
<p>Safari: Current &amp; (Current - 1)</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Under sustained and extremely high throughput the CodeCache settings may need to be tuned to avoid sudden performance loss. See the <a href="administration-guide.html#bootstrap_properties">Bootstrap Properties</a> section for more information.
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect1">
<h2 id="how-to-install-and-start-nifi"><a class="anchor" href="administration-guide.html#how-to-install-and-start-nifi"></a>How to install and start NiFi</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Linux/Unix/macOS</p>
<div class="ulist">
<ul>
<li>
<p>Decompress and untar into desired installation directory</p>
</li>
<li>
<p>Make any desired edits in files found under <code>&lt;installdir&gt;/conf</code></p>
<div class="ulist">
<ul>
<li>
<p>At a minimum, we recommend editing the <em>nifi.properties</em> file and entering a password for the <code>nifi.sensitive.props.key</code> (see <a href="administration-guide.html#system_properties">System Properties</a> below)</p>
</li>
</ul>
</div>
</li>
<li>
<p>From the <code>&lt;installdir&gt;/bin</code> directory, execute the following commands by typing <code>./nifi.sh &lt;command&gt;</code>:</p>
<div class="ulist">
<ul>
<li>
<p><code>start</code>: starts NiFi in the background</p>
</li>
<li>
<p><code>stop</code>: stops NiFi that is running in the background</p>
</li>
<li>
<p><code>status</code>: provides the current status of NiFi</p>
</li>
<li>
<p><code>run</code>: runs NiFi in the foreground and waits for a Ctrl-C to initiate shutdown of NiFi</p>
</li>
<li>
<p><code>install</code>: installs NiFi as a service that can then be controlled via</p>
<div class="ulist">
<ul>
<li>
<p><code>service nifi start</code></p>
</li>
<li>
<p><code>service nifi stop</code></p>
</li>
<li>
<p><code>service nifi status</code></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</li>
</ul>
</div>
</li>
<li>
<p>Windows</p>
<div class="ulist">
<ul>
<li>
<p>Decompress into the desired installation directory</p>
</li>
<li>
<p>Make any desired edits in the files found under <code>&lt;installdir&gt;/conf</code></p>
<div class="ulist">
<ul>
<li>
<p>At a minimum, we recommend editing the <em>nifi.properties</em> file and entering a password for the <code>nifi.sensitive.props.key</code> (see <a href="administration-guide.html#system_properties">System Properties</a> below)</p>
</li>
</ul>
</div>
</li>
<li>
<p>Navigate to the <code>&lt;installdir&gt;/bin</code> directory</p>
</li>
<li>
<p>Double-click <code>run-nifi.bat</code>. This runs NiFi in the foreground and waits for a Ctrl-C to initiate shutdown of NiFi</p>
</li>
<li>
<p>To see the current status of NiFi, double-click <code>status-nifi.bat</code></p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>When NiFi first starts up, the following files and directories are created:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>content_repository</code></p>
</li>
<li>
<p><code>database_repository</code></p>
</li>
<li>
<p><code>flowfile_repository</code></p>
</li>
<li>
<p><code>provenance_repository</code></p>
</li>
<li>
<p><code>work</code> directory</p>
</li>
<li>
<p><code>logs</code> directory</p>
</li>
<li>
<p>Within the <code>conf</code> directory, the <em>flow.json.gz</em> file is created</p>
</li>
</ul>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
For security purposes, when no security configuration is provided NiFi will now bind to 127.0.0.1 by default and the UI will only be accessible through this loopback interface. HTTPS properties should be configured to access NiFi from other interfaces. See the <a href="administration-guide.html#security_configuration">Security Configuration</a> for guidance on how to do this.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>See the <a href="administration-guide.html#system_properties">System Properties</a> section of this guide for more information about configuring NiFi repositories and configuration files.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="build-a-custom-distribution"><a class="anchor" href="administration-guide.html#build-a-custom-distribution"></a>Build a Custom Distribution</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The binary build of Apache NiFi that is provided by the Apache mirrors does not contain every NAR file that is part of the official release. This is due to size constraints imposed by the mirrors to reduce the expenses associated with hosting such a large project. The Developer Guide <a href="developer-guide.html#build">has a list</a> of optional Maven profiles that can be activated to build a binary distribution of NiFi with these extra capabilities.</p>
</div>
<div class="paragraph">
<p>To execute build, download either Java 8 or Java 11 from <a href="https://www.adoptium.net">Adoptium</a> or whichever distribution of the JDK your team uses (Adoptium is the rebranding of AdoptOpenJDK which is one of the most popular). Java 8 and 11 are the only officially supported JVM releases. Then install <a href="https://maven.apache.org">Apache Maven</a>.</p>
</div>
<div class="paragraph">
<p>The next step is to download a copy of the Apache NiFi source code from the <a href="https://nifi.apache.org/download.html">NiFi Downloads page</a>. The reason you need the source build is that it includes a module called <code>nifi-assembly</code> which is the Maven module that builds a binary distribution. Expand the archive and run a Maven clean build. The following example shows how to build a distribution that activates the <code>graph</code> and <code>media</code> bundle profiles to add in support for graph databases and Apache Tika content and metadata extraction.</p>
</div>
<div class="paragraph">
<p><code>cd &lt;nifi_source_folder&gt;/nifi-assembly</code></p>
</div>
<div class="paragraph">
<p><code>mvn clean install -Pinclude-grpc,include-graph,include-media</code></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="port-configuration"><a class="anchor" href="administration-guide.html#port-configuration"></a>Port Configuration</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="nifi"><a class="anchor" href="administration-guide.html#nifi"></a>NiFi</h3>
<div class="paragraph">
<p>The following table lists the default ports used by NiFi and the corresponding property in the <em>nifi.properties</em> file.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Function</th>
<th class="tableblock halign-left valign-top">Property</th>
<th class="tableblock halign-left valign-top">Default Value</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">HTTPS Port</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>8443</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Remote Input Socket Port*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.input.socket.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>10443</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Cluster Node Protocol Port*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.protocol.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>11443</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Cluster Node Load Balancing Port</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.load.balance.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>6342</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Web HTTP Forwarding Port</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.http.port.forwarding</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Listener Bootstrap Port</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.listener.bootstrap.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>random ephemeral</em></p></td>
</tr>
</tfoot>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
The ports marked with an asterisk (*) have property values that are blank by default in <em>nifi.properties</em>. The values shown in the table are the default values for these ports when <a href="toolkit-guide.html#tls_toolkit">TLS Toolkit</a> is used to generate <em>nifi.properties</em> for a secured NiFi instance. The default Certificate Authority Port used by <a href="toolkit-guide.html#tls_toolkit">TLS Toolkit</a> is <code>9443</code>.
</td>
</tr>
</table>
</div>
</div>
<div class="sect2">
<h3 id="embedded-zookeeper"><a class="anchor" href="administration-guide.html#embedded-zookeeper"></a>Embedded ZooKeeper</h3>
<div class="paragraph">
<p>The following table lists the default ports used by an <a href="administration-guide.html#embedded_zookeeper">Embedded ZooKeeper Server</a> and the corresponding property in the <em>zookeeper.properties</em> file.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Function</th>
<th class="tableblock halign-left valign-top">Property</th>
<th class="tableblock halign-left valign-top">Default Value</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ZooKeeper Client Port (Deprecated: client port is no longer specified on a separate line as of NiFi 1.10.x)</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>clientPort</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>2181</code></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ZooKeeper Server Quorum and Leader Election Ports</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>server.1</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Commented examples for the ZooKeeper server ports are included in the <em>zookeeper.properties</em> file in the form <code>server.N=nifi-nodeN-hostname:2888:3888;2181</code>.
</td>
</tr>
</table>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="configuration-best-practices"><a class="anchor" href="administration-guide.html#configuration-best-practices"></a>Configuration Best Practices</h2>
<div class="sectionbody">
<div class="paragraph">
<p>If you are running on Linux, consider these best practices. Typical Linux defaults are not necessarily well-tuned for the needs of an IO intensive application like NiFi. For all of these areas, your distribution&#8217;s requirements may vary. Use these sections as advice, but
consult your distribution-specific documentation for how best to achieve these recommendations.</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Maximum File Handles</dt>
<dd>
<p>NiFi will at any one time potentially have a very large number of file handles open. Increase the limits by
editing <em>/etc/security/limits.conf</em> to add
something like</p>
</dd>
</dl>
</div>
<div class="listingblock">
<div class="content">
<pre>* hard nofile 50000
* soft nofile 50000</pre>
</div>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Maximum Forked Processes</dt>
<dd>
<p>NiFi may be configured to generate a significant number of threads. To increase the allowable number, edit <em>/etc/security/limits.conf</em></p>
</dd>
</dl>
</div>
<div class="listingblock">
<div class="content">
<pre>* hard nproc 10000
* soft nproc 10000</pre>
</div>
</div>
<div class="paragraph">
<p>And your distribution may require an edit to <em>/etc/security/limits.d/90-nproc.conf</em> by adding</p>
</div>
<div class="listingblock">
<div class="content">
<pre>* soft nproc 10000</pre>
</div>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Increase the number of TCP socket ports available</dt>
<dd>
<p>This is particularly important if your flow will be setting up and tearing
down a large number of sockets in a small period of time.</p>
</dd>
</dl>
</div>
<div class="listingblock">
<div class="content">
<pre>sudo sysctl -w net.ipv4.ip_local_port_range="10000 65000"</pre>
</div>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Set how long sockets stay in a TIMED_WAIT state when closed</dt>
<dd>
<p>You don&#8217;t want your sockets to sit and linger too long given that you want to be
able to quickly setup and teardown new sockets. It is a good idea to read more about
it and adjust to something like</p>
</dd>
</dl>
</div>
<div class="paragraph">
<p>for kernel 2.6</p>
</div>
<div class="listingblock">
<div class="content">
<pre>sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1"</pre>
</div>
</div>
<div class="paragraph">
<p>for kernel 3.0</p>
</div>
<div class="listingblock">
<div class="content">
<pre>sudo sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait="1"</pre>
</div>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Tell Linux you never want NiFi to swap</dt>
<dd>
<p>Swapping is fantastic for some applications. It isn&#8217;t good for something like
NiFi that always wants to be running. To tell Linux you&#8217;d like swapping off, you
can edit <em>/etc/sysctl.conf</em> to add the following line</p>
</dd>
</dl>
</div>
<div class="listingblock">
<div class="content">
<pre>vm.swappiness = 0</pre>
</div>
</div>
<div class="paragraph">
<p>For the partitions handling the various NiFi repos, turn off things like <code>atime</code>.
Doing so can cause a surprising bump in throughput. Edit the <code>/etc/fstab</code> file
and for the partition(s) of interest, add the <code>noatime</code> option.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="recommended-antivirus-exclusions"><a class="anchor" href="administration-guide.html#recommended-antivirus-exclusions"></a>Recommended Antivirus Exclusions</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Antivirus software can take a long time to scan large directories and the numerous files within them. Additionally, if the antivirus software locks files or directories during a scan, those resources are unavailable to NiFi processes, causing latency or unavailability of these resources in a NiFi instance/cluster. To prevent these performance and reliability issues from occurring, it is highly recommended to configure your antivirus software to skip scans on the following NiFi directories:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>content_repository</code></p>
</li>
<li>
<p><code>flowfile_repository</code></p>
</li>
<li>
<p><code>logs</code></p>
</li>
<li>
<p><code>provenance_repository</code></p>
</li>
<li>
<p><code>state</code></p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="logging_configuration"><a class="anchor" href="administration-guide.html#logging_configuration"></a>Logging Configuration</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi uses <a href="https://logback.qos.ch/" target="_blank" rel="noopener">logback</a> as the runtime logging implementation. The <code>conf</code> directory contains a
standard <code>logback.xml</code> configuration with default appender and level settings. The
<a href="https://logback.qos.ch/manual/index.html">logback manual</a> provides a complete reference of available options.</p>
</div>
<div class="sect2">
<h3 id="standard-log-files"><a class="anchor" href="administration-guide.html#standard-log-files"></a>Standard Log Files</h3>
<div class="paragraph">
<p>The standard logback configuration includes the following appender definitions and associated log files:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">File</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi-app.log</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Application log containing framework and component messages</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi-bootstrap.log</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Bootstrap log containing startup and shutdown messages</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi-deprecation.log</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Deprecation log containing warnings for deprecated components and features</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi-request.log</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">HTTP request log containing user interface and REST API access messages</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi-user.log</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">User log containing authentication and authorization messages</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="deprecation-logging"><a class="anchor" href="administration-guide.html#deprecation-logging"></a>Deprecation Logging</h3>
<div class="paragraph">
<p>The <code>nifi-deprecation.log</code> contains warning messages describing components and features that will be removed in
subsequent versions. Deprecation warnings should be evaluated and addressed to avoid breaking changes when upgrading to
a new major version. Resolving deprecation warnings involves upgrading to new components, changing component property
settings, or refactoring custom component classes.</p>
</div>
<div class="paragraph">
<p>Deprecation logging provides a method for checking compatibility before upgrading from one major release version to
another. Upgrading to the latest minor release version will provide the most accurate set of deprecation warnings.</p>
</div>
<div class="paragraph">
<p>It is important to note that deprecation logging applies to both components and features. Logging for deprecated
features requires a runtime reference to the property or method impacted. Disabled components with deprecated properties
or methods will not generate deprecation logs. For this reason, it is important to exercise all configured components
long enough to exercise standard flow behavior.</p>
</div>
<div class="paragraph">
<p>Deprecation logging can generate repeated messages depending on component configuration and usage patterns. Disabling
deprecation logging for a specific component class can be configured by adding a <code>logger</code> element to <code>logback.xml</code>.
The <code>name</code> attribute must start with <code>deprecation</code>, followed by the component class. Setting the <code>level</code> attribute to
<code>OFF</code> disables deprecation logging for the component specified.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-xml" data-lang="xml">&lt;logger name="deprecation.org.apache.nifi.processors.ListenLegacyProtocol" level="OFF" /&gt;</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="security_configuration"><a class="anchor" href="administration-guide.html#security_configuration"></a>Security Configuration</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi provides several different configuration options for security purposes. The most important properties are those under the
"security properties" heading in the <em>nifi.properties</em> file. In order to run securely, the following properties must be set:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keystore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filename of the Keystore that contains the server&#8217;s private key.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keystoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The type of Keystore. Must be <code>PKCS12</code> or <code>JKS</code> or <code>BCFKS</code>. JKS is the preferred type, BCFKS and PKCS12 files will be loaded with BouncyCastle provider.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keystorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Keystore.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keyPasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the certificate in the Keystore. If not set, the value of <code>nifi.security.keystorePasswd</code> will be used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.truststore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filename of the Truststore that will be used to authorize those connecting to NiFi. A secured instance with no Truststore will refuse all incoming connections.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.truststoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The type of the Truststore. Must be <code>PKCS12</code> or <code>JKS</code> or <code>BCFKS</code>. JKS is the preferred type, BCFKS and PKCS12 files will be loaded with BouncyCastle provider.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.truststorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Truststore.</p></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>Once the above properties have been configured, we can enable the User Interface to be accessed over HTTPS instead of HTTP. This is accomplished
by setting the <code>nifi.web.https.host</code> and <code>nifi.web.https.port</code> properties. The <code>nifi.web.https.host</code> property indicates which hostname the server
should run on. If it is desired that the HTTPS interface be accessible from all network interfaces, a value of <code>0.0.0.0</code> should be used. To allow
admins to configure the application to run only on specific network interfaces, <code>nifi.web.http.network.interface*</code> or <code>nifi.web.https.network.interface*</code>
properties can be specified.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
It is important when enabling HTTPS that the <code>nifi.web.http.port</code> property be unset. NiFi only supports running on HTTP <strong>or</strong> HTTPS, not both simultaneously.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>NiFi&#8217;s web server will REQUIRE certificate based client authentication for users accessing the User Interface when not configured with an alternative
authentication mechanism which would require one way SSL (for instance LDAP, OpenId Connect, etc). Enabling an alternative authentication mechanism will
configure the web server to WANT certificate base client authentication. This will allow it to support users with certificates and those without that
may be logging in with credentials. See <a href="administration-guide.html#user_authentication">User Authentication</a> for more details.</p>
</div>
<div class="paragraph">
<p>Now that the User Interface has been secured, we can easily secure Site-to-Site connections and inner-cluster communications, as well. This is
accomplished by setting the <code>nifi.remote.input.secure</code> and <code>nifi.cluster.protocol.is.secure</code> properties, respectively, to <code>true</code>. These communications
will always REQUIRE two way SSL as the nodes will use their configured keystore/truststore for authentication.</p>
</div>
<div class="paragraph">
<p>Automatic refreshing of NiFi&#8217;s web SSL context factory can be enabled using the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.autoreload.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether the SSL context factory should be automatically reloaded if updates to the keystore and truststore are detected. By default, it is set to <code>false</code>.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.autoreload.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the interval at which the keystore and truststore are checked for updates. Only applies if <code>nifi.security.autoreload.enabled</code> is set to <code>true</code>. The default value is <code>10 secs</code>.</p></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>Once the <code>nifi.security.autoreload.enabled</code> property is set to <code>true</code>, any valid changes to the configured keystore and truststore will cause NiFi&#8217;s SSL context factory to be reloaded, allowing clients to pick up the changes. This is intended to allow expired certificates to be updated in the keystore and new trusted certificates to be added in the truststore, all without having to restart the NiFi server.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Changes to any of the <code>nifi.security.keystore*</code> or <code>nifi.security.truststore*</code> properties will not be picked up by the auto-refreshing logic, which assumes the passwords and store paths will remain the same.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="tls_generation_toolkit"><a class="anchor" href="administration-guide.html#tls_generation_toolkit"></a>TLS Generation Toolkit</h3>
<div class="paragraph">
<p>In order to facilitate the secure setup of NiFi, you can use the <code>tls-toolkit</code> command line utility to automatically generate the required keystores, truststore, and relevant configuration files. This is especially useful for securing multiple NiFi nodes, which can be a tedious and error-prone process. For more information, see the <a href="toolkit-guide.html#tls_toolkit">TLS Toolkit</a> section in the <a href="toolkit-guide.html">NiFi Toolkit Guide</a>. Related topics include:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="toolkit-guide.html#wildcard_certificates">Wildcard Certificates</a></p>
</li>
<li>
<p><a href="toolkit-guide.html#tls_operation_modes">Operation Modes: Standalone and Client/Server</a></p>
</li>
<li>
<p><a href="toolkit-guide.html#tls_intermediate_ca">Using An Existing Intermediate Certificate Authority</a></p>
</li>
<li>
<p><a href="toolkit-guide.html#additional_certificate_commands">Additional Certificate Commands</a></p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="tls_cipher_suites"><a class="anchor" href="administration-guide.html#tls_cipher_suites"></a>TLS Cipher Suites</h3>
<div class="paragraph">
<p>The Java Runtime Environment provides the ability to specify custom TLS cipher suites to be used by servers when accepting client connections. See
<a href="https://java.com/en/configure_crypto.html" target="_blank" rel="noopener">here</a> for more information. To use this feature for the NiFi web service, the following NiFi properties
may be set:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.ciphersuites.include</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Set of ciphers that are available to be used by incoming client connections. Replaces system defaults if set.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.ciphersuites.exclude</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Set of ciphers that must not be used by incoming client connections. Filters available ciphers if set.</p></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>Each property should take the form of a comma-separated list of common cipher names as specified
<a href="https://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#ciphersuites" target="_blank" rel="noopener">here</a>. Regular expressions
(for example <code>^.*GCM_SHA256$</code>) may also be specified.</p>
</div>
<div class="paragraph">
<p>The semantics match the use of the following Jetty APIs:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/util/ssl/SslContextFactory.html#setIncludeCipherSuites(java.lang.String...)">SslContextFactory.setIncludeCipherSuites()</a></p>
</li>
<li>
<p><a href="https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/util/ssl/SslContextFactory.html#setExcludeCipherSuites(java.lang.String...)">SslContextFactory.setExcludeCipherSuites()</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="user_authentication"><a class="anchor" href="administration-guide.html#user_authentication"></a>User Authentication</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi supports user authentication via client certificates, via username/password, via Apache Knox, or via <a href="http://openid.net/connect" target="_blank" rel="noopener">OpenId Connect</a>.</p>
</div>
<div class="paragraph">
<p>Username/password authentication is performed by a 'Login Identity Provider'. The Login Identity Provider is a pluggable mechanism for
authenticating users via their username/password. Which Login Identity Provider to use is configured in the <em>nifi.properties</em> file.
Currently NiFi offers username/password with Login Identity Providers options for <a href="administration-guide.html#single_user_identity_provider">Single User</a>, <a href="administration-guide.html#ldap_login_identity_provider">Lightweight Directory Access Protocol (LDAP)</a> and <a href="administration-guide.html#kerberos_login_identity_provider">Kerberos</a>.</p>
</div>
<div class="paragraph">
<p>The <code>nifi.login.identity.provider.configuration.file</code> property specifies the configuration file for Login Identity Providers. By default, this property is set to <code>./conf/login-identity-providers.xml</code>.</p>
</div>
<div class="paragraph">
<p>The <code>nifi.security.user.login.identity.provider</code> property indicates which of the configured Login Identity Provider should be
used. The default value of this property is <code>single-user-provider</code> supporting authentication with a generated username and password.</p>
</div>
<div class="paragraph">
<p>During OpenId Connect authentication, NiFi will redirect users to login with the Provider before returning to NiFi. NiFi will then
call the Provider to obtain the user identity.</p>
</div>
<div class="paragraph">
<p>During Apache Knox authentication, NiFi will redirect users to login with Apache Knox before returning to NiFi. NiFi will verify the Apache Knox
token during authentication.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
NiFi can only be configured for username/password, OpenId Connect, or Apache Knox at a given time. It does not support running each of
these concurrently. NiFi will require client certificates for authenticating users over HTTPS if none of these are configured.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>A user cannot anonymously authenticate with a secured instance of NiFi unless <code>nifi.security.allow.anonymous.authentication</code> is set to <code>true</code>.
If this is the case, NiFi must also be configured with an Authorizer that supports authorizing an anonymous user. Currently, NiFi does not ship
with any Authorizers that support this. There is a feature request here to help support it (<a href="https://issues.apache.org/jira/browse/NIFI-2730" target="_blank" rel="noopener">NIFI-2730</a>).</p>
</div>
<div class="paragraph">
<p>There are three scenarios to consider when setting <code>nifi.security.allow.anonymous.authentication</code>. When the user is directly calling an endpoint
with no attempted authentication then <code>nifi.security.allow.anonymous.authentication</code> will control whether the request is authenticated or rejected.
The other two scenarios are when the request is proxied. This could either be proxied by a NiFi node (e.g. a node in the NiFi cluster) or by a separate
proxy that is proxying a request for an anonymous user. In these proxy scenarios <code>nifi.security.allow.anonymous.authentication</code> will control whether the
request is authenticated or rejected. In all three of these scenarios if the request is authenticated it will subsequently be subjected to normal
authorization based on the requested resource.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
NiFi does not perform user authentication over HTTP. Using HTTP, all users will be granted all roles.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="single_user_identity_provider"><a class="anchor" href="administration-guide.html#single_user_identity_provider"></a>Single User</h3>
<div class="paragraph">
<p>The default Single User Login Identity Provider supports automated generation of username and password credentials.</p>
</div>
<div class="paragraph">
<p>The generated username will be a random UUID consisting of 36 characters. The generated password will be a random string
consisting of 32 characters and stored using bcrypt hashing.</p>
</div>
<div class="paragraph">
<p>The default configuration in <em>nifi.properties</em> enables Single User authentication:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>nifi.security.user.login.identity.provider=single-user-provider</pre>
</div>
</div>
<div class="paragraph">
<p>The default <em>login-identity-providers.xml</em> includes a blank provider definition:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>&lt;provider&gt;
&lt;identifier&gt;single-user-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authentication.single.user.SingleUserLoginIdentityProvider&lt;/class&gt;
&lt;property name="Username"/&gt;
&lt;property name="Password"/&gt;
&lt;/provider&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>The following command can be used to change the Username and Password:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>$ ./bin/nifi.sh set-single-user-credentials &lt;username&gt; &lt;password&gt;</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="ldap_login_identity_provider"><a class="anchor" href="administration-guide.html#ldap_login_identity_provider"></a>Lightweight Directory Access Protocol (LDAP)</h3>
<div class="paragraph">
<p>Below is an example and description of configuring a Login Identity Provider that integrates with a Directory Server to authenticate users.</p>
</div>
<div class="paragraph">
<p>Set the following in <em>nifi.properties</em> to enable LDAP username/password authentication:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>nifi.security.user.login.identity.provider=ldap-provider</pre>
</div>
</div>
<div class="paragraph">
<p>Modify <em>login-identity-providers.xml</em> to enable the <code>ldap-provider</code>. Here is the sample provided in the file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>&lt;provider&gt;
&lt;identifier&gt;ldap-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.ldap.LdapProvider&lt;/class&gt;
&lt;property name="Authentication Strategy"&gt;START_TLS&lt;/property&gt;
&lt;property name="Manager DN"&gt;&lt;/property&gt;
&lt;property name="Manager Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Client Auth"&gt;&lt;/property&gt;
&lt;property name="TLS - Protocol"&gt;&lt;/property&gt;
&lt;property name="TLS - Shutdown Gracefully"&gt;&lt;/property&gt;
&lt;property name="Referral Strategy"&gt;FOLLOW&lt;/property&gt;
&lt;property name="Connect Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Read Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Url"&gt;&lt;/property&gt;
&lt;property name="User Search Base"&gt;&lt;/property&gt;
&lt;property name="User Search Filter"&gt;&lt;/property&gt;
&lt;property name="Identity Strategy"&gt;USE_DN&lt;/property&gt;
&lt;property name="Authentication Expiration"&gt;12 hours&lt;/property&gt;
&lt;/provider&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>ldap-provider</code> has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Authentication Strategy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">How the connection to the LDAP server is authenticated. Possible values are <code>ANONYMOUS</code>, <code>SIMPLE</code>, <code>LDAPS</code>, or <code>START_TLS</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Manager DN</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The DN of the manager that is used to bind to the LDAP server to search for users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Manager Password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password of the manager that is used to bind to the LDAP server to search for users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Keystore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Path to the Keystore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Keystore Password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Password for the Keystore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Keystore Type</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type of the Keystore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. <code>JKS</code> or <code>PKCS12</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Truststore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Path to the Truststore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Truststore Password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Password for the Truststore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Truststore Type</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type of the Truststore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. <code>JKS</code> or <code>PKCS12</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Client Auth</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Client authentication policy when connecting to LDAP using LDAPS or START_TLS. Possible values are <code>REQUIRED</code>, <code>WANT</code>, <code>NONE</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Protocol</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Protocol to use when connecting to LDAP using LDAPS or START_TLS. (i.e. <code>TLS</code>, <code>TLSv1.1</code>, <code>TLSv1.2</code>, etc).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Shutdown Gracefully</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether the TLS should be shut down gracefully before the target context is closed. Defaults to false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Referral Strategy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Strategy for handling referrals. Possible values are <code>FOLLOW</code>, <code>IGNORE</code>, <code>THROW</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Connect Timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of connect timeout. (i.e. <code>10 secs</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Read Timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of read timeout. (i.e. <code>10 secs</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Space-separated list of URLs of the LDAP servers (i.e. <code>ldap://&lt;hostname&gt;:&lt;port&gt;</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Search Base</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Base DN for searching for users (i.e. <code>CN=Users,DC=example,DC=com</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Search Filter</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filter for searching for users against the <code>User Search Base</code>. (i.e. <code>sAMAccountName={0}</code>). The user specified name is inserted into '{0}'.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Identity Strategy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Strategy to identify users. Possible values are <code>USE_DN</code> and <code>USE_USERNAME</code>. The default functionality if this property is missing is USE_DN in order to retain backward
compatibility. <code>USE_DN</code> will use the full DN of the user entry if possible. <code>USE_USERNAME</code> will use the username the user logged in with.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Authentication Expiration</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The duration of how long the user authentication is valid for. If the user never logs out, they will be required to log back in following this duration.</p></td>
</tr>
</tfoot>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
For changes to <em>nifi.properties</em> and <em>login-identity-providers.xml</em> to take effect, NiFi needs to be restarted. If NiFi is clustered, configuration files must be the same on all nodes.
</td>
</tr>
</table>
</div>
</div>
<div class="sect2">
<h3 id="kerberos_login_identity_provider"><a class="anchor" href="administration-guide.html#kerberos_login_identity_provider"></a>Kerberos</h3>
<div class="paragraph">
<p>Below is an example and description of configuring a Login Identity Provider that integrates with a Kerberos Key Distribution Center (KDC) to authenticate users.</p>
</div>
<div class="paragraph">
<p>Set the following in <em>nifi.properties</em> to enable Kerberos username/password authentication:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>nifi.security.user.login.identity.provider=kerberos-provider</pre>
</div>
</div>
<div class="paragraph">
<p>Modify <em>login-identity-providers.xml</em> to enable the <code>kerberos-provider</code>. Here is the sample provided in the file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>&lt;provider&gt;
&lt;identifier&gt;kerberos-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.kerberos.KerberosProvider&lt;/class&gt;
&lt;property name="Default Realm"&gt;NIFI.APACHE.ORG&lt;/property&gt;
&lt;property name="Authentication Expiration"&gt;12 hours&lt;/property&gt;
&lt;/provider&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>kerberos-provider</code> has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Default Realm</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Default realm to provide when user enters incomplete user principal (i.e. <code>NIFI.APACHE.ORG</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Authentication Expiration</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The duration of how long the user authentication is valid for. If the user never logs out, they will be required to log back in following this duration.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>See also <a href="administration-guide.html#kerberos_service">Kerberos Service</a> to allow single sign-on access via client Kerberos tickets.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
For changes to <em>nifi.properties</em> and <em>login-identity-providers.xml</em> to take effect, NiFi needs to be restarted. If NiFi is clustered, configuration files must be the same on all nodes.
</td>
</tr>
</table>
</div>
</div>
<div class="sect2">
<h3 id="openid_connect"><a class="anchor" href="administration-guide.html#openid_connect"></a>OpenId Connect</h3>
<div class="paragraph">
<p>To enable authentication via OpenId Connect the following properties must be configured in <em>nifi.properties</em>.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.discovery.url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The discovery URL for the desired OpenId Connect Provider (<a href="http://openid.net/specs/openid-connect-discovery-1_0.html" target="_blank" rel="noopener">http://openid.net/specs/openid-connect-discovery-1_0.html</a>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.connect.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Connect timeout when communicating with the OpenId Connect Provider. The default value is <code>5 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.read.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Read timeout when communicating with the OpenId Connect Provider. The default value is <code>5 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.client.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The client id for NiFi after registration with the OpenId Connect Provider.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.client.secret</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The client secret for NiFi after registration with the OpenId Connect Provider.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.preferred.jwsalgorithm</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The preferred algorithm for validating identity tokens. If this value is blank, it will default to <code>RS256</code> which is required to be supported
by the OpenId Connect Provider according to the specification. If this value is <code>HS256</code>, <code>HS384</code>, or <code>HS512</code>, NiFi will attempt to validate HMAC protected tokens using the specified client secret.
If this value is <code>none</code>, NiFi will attempt to validate unsecured/plain tokens. Other values for this algorithm will attempt to parse as an RSA or EC algorithm to be used in conjunction with the
JSON Web Key (JWK) provided through the jwks_uri in the metadata found at the discovery URL.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.additional.scopes</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma separated scopes that are sent to OpenId Connect Provider in addition to <code>openid</code> and <code>email</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.claim.identifying.user</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Claim that identifies the user to be logged in; default is <code>email</code>. May need to be requested via the <code>nifi.security.user.oidc.additional.scopes</code> before usage.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.fallback.claims.identifying.user</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma separated possible fallback claims used to identify the user in case <code>nifi.security.user.oidc.claim.identifying.user</code> claim is not present for the login user.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.truststore.strategy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If value is <code>NIFI</code>, use the NiFi truststore when connecting to the OIDC service, otherwise if value is <code>JDK</code> use Java&#8217;s default <code>cacerts</code> truststore. The default value is <code>JDK</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="saml"><a class="anchor" href="administration-guide.html#saml"></a>SAML</h3>
<div class="paragraph">
<p>To enable authentication via SAML the following properties must be configured in <em>nifi.properties</em>.</p>
</div>
<div class="paragraph">
<p>Configuring a Metadata URL and an Entity Identifier enables Apache NiFi to act as a SAML 2.0 Relying Party, allowing users
to authenticate using an account managed through a SAML 2.0 Asserting Party.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.idp.metadata.url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL for obtaining the identity provider&#8217;s metadata. The metadata can be retrieved from the identity provider via <code>http://</code> or <code>https://</code>, or a local file can be referenced using <code>file://</code> .</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.sp.entity.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The entity id of the service provider (i.e. NiFi). This value will be used as the <code>Issuer</code> for SAML authentication requests and should be a valid URI. In some cases the service provider entity id must be registered ahead of time with the identity provider.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.identity.attribute.name</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of a SAML assertion attribute containing the user&#8217;sidentity. This property is optional and if not specified, or if the attribute is not found, then the NameID of the Subject will be used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.group.attribute.name</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of a SAML assertion attribute containing group names the user belongs to. This property is optional, but if populated the groups will be passed along to the authorization process.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.request.signing.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Controls the value of <code>AuthnRequestsSigned</code> in the generated service provider metadata from <code>nifi-api/access/saml/metadata</code>. This indicates that the service provider (i.e. NiFi) should not sign authentication requests sent to the identity provider, but the requests may still need to be signed if the identity provider indicates <code>WantAuthnRequestSigned=true</code>. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.want.assertions.signed</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Controls the value of <code>WantAssertionsSigned</code> in the generated service provider metadata from <code>nifi-api/access/saml/metadata</code>. This indicates that the identity provider should sign assertions, but some identity providers may provide their own configuration for controlling whether assertions are signed. The default value is <code>true</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.signature.algorithm</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The algorithm to use when signing SAML messages. Reference the <a href="https://git.shibboleth.net/view/?p=java-xmltooling.git;a=blob;f=src/main/java/org/opensaml/xml/signature/SignatureConstants.java">Open SAML Signature Constants</a> for a list of valid values. If not specified, a default of SHA-256 will be used. The default value is <code><a href="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" class="bare">http://www.w3.org/2001/04/xmldsig-more#rsa-sha256</a></code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.authentication.expiration</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The expiration of the NiFi JWT that will be produced from a successful SAML authentication response. The default value is <code>12 hours</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.single.logout.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Enables SAML SingleLogout which causes a logout from NiFi to logout of the identity provider. By default, a logout of NiFi will only remove the NiFi JWT. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.http.client.truststore.strategy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The truststore strategy when the IDP metadata URL begins with https. A value of <code>JDK</code> indicates to use the JDK&#8217;s default truststore. A value of <code>NIFI</code> indicates to use the truststore specified by <code>nifi.security.truststore</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.http.client.connect.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The connection timeout when communicating with the SAML IDP. The default value is <code>30 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.saml.http.client.read.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The read timeout when communicating with the SAML IDP. The default value is <code>30 secs</code>.</p></td>
</tr>
</tbody>
</table>
<div class="sect3">
<h4 id="saml-rest-resources"><a class="anchor" href="administration-guide.html#saml-rest-resources"></a>SAML REST Resources</h4>
<div class="paragraph">
<p>SAML authentication enables the following REST API resources for integration with a SAML 2.0 Asserting Party:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Resource Path</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">/nifi-api/access/saml/local-logout/request</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Complete SAML 2.0 Logout processing without communicating with the Asserting Party</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">/nifi-api/access/saml/login/consumer</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Process SAML 2.0 Login Requests assertions using HTTP-POST or HTTP-REDIRECT binding</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">/nifi-api/access/saml/metadata</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Retrieve SAML 2.0 entity descriptor metadata as XML</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">/nifi-api/access/saml/single-logout/consumer</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Process SAML 2.0 Single Logout Request assertions using HTTP-POST or HTTP-REDIRECT binding. Requires Single Logout to be enabled.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">/nifi-api/access/saml/single-logout/request</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Complete SAML 2.0 Single Logout processing initiating a request to the Asserting Party. Requires Single Logout to be enabled.</p></td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="sect2">
<h3 id="apache_knox"><a class="anchor" href="administration-guide.html#apache_knox"></a>Apache Knox</h3>
<div class="paragraph">
<p>To enable authentication via Apache Knox the following properties must be configured in <em>nifi.properties</em>.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.knox.url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL for the Apache Knox login page.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.knox.publicKey</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The path to the Apache Knox public key that will be used to verify the signatures of the authentication tokens in the HTTP Cookie.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.knox.cookieName</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the HTTP Cookie that Apache Knox will generate after successful login. The default value is <code>hadoop-jwt</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.knox.audiences</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. A comma separate listed of allowed audiences. If set, the audience in the token must be present in
this listing. The audience that is populated in the token can be configured in Knox.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="json_web_token"><a class="anchor" href="administration-guide.html#json_web_token"></a>JSON Web Tokens</h3>
<div class="paragraph">
<p>NiFi uses JSON Web Tokens to provide authenticated access after the initial login process. Generated JSON Web Tokens include the authenticated user identity
as well as the issuer and expiration from the configured Login Identity Provider.</p>
</div>
<div class="paragraph">
<p>NiFi uses generated RSA Key Pairs with a key size of 4096 bits to support the <code>PS512</code> algorithm for JSON Web Signatures. The system stores RSA
Public Keys using the configured local State Provider and retains the RSA Private Key in memory. This approach supports signature verification
for the expiration configured in the Login Identity Provider without persisting the private key.</p>
</div>
<div class="paragraph">
<p>JSON Web Token support includes revocation on logout using JSON Web Token Identifiers. The system denies access for expired tokens based on the
Login Identity Provider configuration, but revocation invalidates the token prior to expiration. The system stores revoked identifiers using the
configured local State Provider and runs a scheduled command to delete revoked identifiers after the associated expiration.</p>
</div>
<div class="paragraph">
<p>The following settings can be configured in <em>nifi.properties</em> to control JSON Web Token signing.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.jws.key.rotation.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">JSON Web Signature Key Rotation Period defines how often the system generates a new RSA Key Pair, expressed as an ISO 8601 duration. The default is one hour: <code>PT1H</code></p></td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div class="sect1">
<h2 id="multi-tenant-authorization"><a class="anchor" href="administration-guide.html#multi-tenant-authorization"></a>Multi-Tenant Authorization</h2>
<div class="sectionbody">
<div class="paragraph">
<p>After you have configured NiFi to run securely and with an authentication mechanism, you must configure who has access to the system, and the level of their access.
You can do this using 'multi-tenant authorization'. Multi-tenant authorization enables multiple groups of users (tenants) to command, control, and observe different
parts of the dataflow, with varying levels of authorization. When an authenticated user attempts to view or modify a NiFi resource, the system checks whether the
user has privileges to perform that action. These privileges are defined by policies that you can apply system-wide or to individual components.</p>
</div>
<div class="sect2">
<h3 id="authorizer-configuration"><a class="anchor" href="administration-guide.html#authorizer-configuration"></a>Authorizer Configuration</h3>
<div class="paragraph">
<p>An 'authorizer' grants users the privileges to manage users and policies by creating preliminary authorizations at startup.</p>
</div>
<div class="paragraph">
<p>Authorizers are configured using two properties in the <em>nifi.properties</em> file:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>The <code>nifi.authorizer.configuration.file</code> property specifies the configuration file where authorizers are defined. By default, the <em>authorizers.xml</em> file located in the root installation conf directory is selected.</p>
</li>
<li>
<p>The <code>nifi.security.user.authorizer</code> property indicates which of the configured authorizers in the <em>authorizers.xml</em> file to use.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="authorizers-setup"><a class="anchor" href="administration-guide.html#authorizers-setup"></a>Authorizers.xml Setup</h3>
<div class="paragraph">
<p>The <em>authorizers.xml</em> file is used to define and configure available authorizers. The default authorizer is the StandardManagedAuthorizer. The managed authorizer is comprised of a UserGroupProvider
and a AccessPolicyProvider. The users, group, and access policies will be loaded and optionally configured through these providers. The managed authorizer will make all access decisions based on
these provided users, groups, and access policies.</p>
</div>
<div class="paragraph">
<p>During startup there is a check to ensure that there are no two users/groups with the same identity/name. This check is executed regardless of the configured implementation. This is necessary because this is how users/groups are identified and authorized during access decisions.</p>
</div>
<div class="sect3">
<h4 id="fileusergroupprovider"><a class="anchor" href="administration-guide.html#fileusergroupprovider"></a>FileUserGroupProvider</h4>
<div class="paragraph">
<p>The default UserGroupProvider is the FileUserGroupProvider, however, you can develop additional UserGroupProviders as extensions. The FileUserGroupProvider has the following properties:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Users File - The file where the FileUserGroupProvider stores users and groups. By default, the <em>users.xml</em> in the <code>conf</code> directory is chosen.</p>
</li>
<li>
<p>Legacy Authorized Users File - The full path to an existing <em>authorized-users.xml</em> that will be automatically be used to load the users and groups into the Users File.</p>
</li>
<li>
<p>Initial User Identity - The identity of a users and systems to seed the Users File. The name of each property must be unique, for example: "Initial User Identity A", "Initial User Identity B", "Initial User Identity C" or "Initial User Identity 1", "Initial User Identity 2", "Initial User Identity 3"</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="ldapusergroupprovider"><a class="anchor" href="administration-guide.html#ldapusergroupprovider"></a>LdapUserGroupProvider</h4>
<div class="paragraph">
<p>Another option for the UserGroupProvider is the LdapUserGroupProvider. By default, this option is commented out but can be configured in lieu of the FileUserGroupProvider. This will sync users and groups from a directory server and will present them in the NiFi UI in read only form.</p>
</div>
<div class="paragraph">
<p>The LdapUserGroupProvider has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Authentication Strategy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">How the connection to the LDAP server is authenticated. Possible values are <code>ANONYMOUS</code>, <code>SIMPLE</code>, <code>LDAPS</code>, or <code>START_TLS</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Manager DN</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The DN of the manager that is used to bind to the LDAP server to search for users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Manager Password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password of the manager that is used to bind to the LDAP server to search for users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Keystore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Path to the Keystore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Keystore Password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Password for the Keystore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Keystore Type</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type of the Keystore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. <code>JKS</code> or <code>PKCS12</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Truststore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Path to the Truststore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Truststore Password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Password for the Truststore that is used when connecting to LDAP using LDAPS or START_TLS.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Truststore Type</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type of the Truststore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. <code>JKS</code> or <code>PKCS12</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Client Auth</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Client authentication policy when connecting to LDAP using LDAPS or START_TLS. Possible values are <code>REQUIRED</code>, <code>WANT</code>, <code>NONE</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Protocol</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Protocol to use when connecting to LDAP using LDAPS or START_TLS. (i.e. <code>TLS</code>, <code>TLSv1.1</code>, <code>TLSv1.2</code>, etc).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>TLS - Shutdown Gracefully</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether the TLS should be shut down gracefully before the target context is closed. Defaults to false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Referral Strategy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Strategy for handling referrals. Possible values are <code>FOLLOW</code>, <code>IGNORE</code>, <code>THROW</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Connect Timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of connect timeout. (i.e. <code>10 secs</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Read Timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of read timeout. (i.e. <code>10 secs</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Space-separated list of URLs of the LDAP servers (i.e. <code>ldap://&lt;hostname&gt;:&lt;port&gt;</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Page Size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Sets the page size when retrieving users and groups. If not specified, no paging is performed.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Membership - Enforce Case Sensitivity</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Sets whether group membership decisions are case sensitive. When a user or group is inferred (by not specifying or user or group search base or user identity attribute or group name attribute) case sensitivity is enforced since the value to use for the user identity or group name would be ambiguous. Defaults to false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Sync Interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of time between syncing users and groups. (i.e. <code>30 mins</code>). Minimum allowable value is <code>10 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Search Base</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Base DN for searching for users (i.e. <code>ou=users,o=nifi</code>). Required to search users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Object Class</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Object class for identifying users (i.e. <code>person</code>). Required if searching users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Search Scope</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Search scope for searching users (<code>ONE_LEVEL</code>, <code>OBJECT</code>, or <code>SUBTREE</code>). Required if searching users.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Search Filter</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filter for searching for users against the <code>User Search Base</code> (i.e. <code>(memberof=cn=team1,ou=groups,o=nifi)</code>). Optional.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Identity Attribute</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Attribute to use to extract user identity (i.e. <code>cn</code>). Optional. If not set, the entire DN is used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Group Name Attribute</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Attribute to use to define group membership (i.e. <code>memberof</code>). Optional. If not set group membership will not be calculated through the users. Will rely on group membership being defined through <code>Group Member Attribute</code> if set. The value of this property is the name of the attribute in the user ldap entry that associates them with a group. The value of that user attribute could be a dn or group name for instance. What value is expected is configured in the <code>User Group Name Attribute - Referenced Group Attribute</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Group Name Attribute - Referenced Group Attribute</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If blank, the value of the attribute defined in <code>User Group Name Attribute</code> is expected to be the full dn of the group. If not blank, this property will define the attribute of the group ldap entry that the value of the attribute defined in <code>User Group Name Attribute</code> is referencing (i.e. <code>name</code>). Use of this property requires that <code>Group Search Base</code> is also configured.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Search Base</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Base DN for searching for groups (i.e. <code>ou=groups,o=nifi</code>). Required to search groups.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Object Class</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Object class for identifying groups (i.e. <code>groupOfNames</code>). Required if searching groups.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Search Scope</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Search scope for searching groups (<code>ONE_LEVEL</code>, <code>OBJECT</code>, or <code>SUBTREE</code>). Required if searching groups.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Search Filter</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filter for searching for groups against the <code>Group Search Base</code>. Optional.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Name Attribute</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Attribute to use to extract group name (i.e. <code>cn</code>). Optional. If not set, the entire DN is used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Member Attribute</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Attribute to use to define group membership (i.e. <code>member</code>). Optional. If not set group membership will not be calculated through the groups. Will rely on group membership being defined through <code>User Group Name Attribute</code> if set. The value of this property is the name of the attribute in the group ldap entry that associates them with a user. The value of that group attribute could be a dn or memberUid for instance. What value is expected is configured in the <code>Group Member Attribute - Referenced User Attribute</code>. (i.e. <code>member: cn=User 1,ou=users,o=nifi</code> vs. <code>memberUid: user1</code>)</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Member Attribute - Referenced User Attribute</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If blank, the value of the attribute defined in <code>Group Member Attribute</code> is expected to be the full dn of the user. If not blank, this property will define the attribute of the user ldap entry that the value of the attribute defined in <code>Group Member Attribute</code> is referencing (i.e. <code>uid</code>). Use of this property requires that <code>User Search Base</code> is also configured. (i.e. <code>member: cn=User 1,ou=users,o=nifi</code> vs. <code>memberUid: user1</code>)</p></td>
</tr>
</tfoot>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Any identity mapping rules specified in <em>nifi.properties</em> will also be applied to the user identities. Group names are not mapped.
</td>
</tr>
</table>
</div>
</div>
<div class="sect3">
<h4 id="shellusergroupprovider"><a class="anchor" href="administration-guide.html#shellusergroupprovider"></a>ShellUserGroupProvider</h4>
<div class="paragraph">
<p>The ShellUserGroupProvider fetches user and group details from Unix-like systems using shell commands.</p>
</div>
<div class="paragraph">
<p>This provider executes various shell pipelines with commands such as <code>getent</code> on Linux and <code>dscl</code> on macOS.</p>
</div>
<div class="paragraph">
<p>Supported systems may be configured to retrieve users and groups from an external source, such as LDAP or NIS. In these cases the shell commands
will return those external users and groups. This provides administrators another mechanism to integrate user and group directory services.</p>
</div>
<div class="paragraph">
<p>The ShellUserGroupProvider has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Initial Refresh Delay</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of initial delay before first user and group refresh. (i.e. <code>10 secs</code>). Default is <code>5 mins</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Refresh Delay</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of delay between each user and group refresh. (i.e. <code>10 secs</code>). Default is <code>5 mins</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Exclude Groups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Regular expression used to exclude groups. Default is '', which means no groups are excluded.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Exclude Users</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Regular expression used to exclude users. Default is '', which means no users are excluded.</p></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>Like LdapUserGroupProvider, the ShellUserGroupProvider is commented out in the <em>authorizers.xml</em> file. Refer to that comment for usage examples.</p>
</div>
</div>
<div class="sect3">
<h4 id="azuregraphusergroupprovider"><a class="anchor" href="administration-guide.html#azuregraphusergroupprovider"></a>AzureGraphUserGroupProvider</h4>
<div class="paragraph">
<p>The AzureGraphUserGroupProvider fetches users and groups from Azure Active Directory (AAD) using the Microsoft Graph API.</p>
</div>
<div class="paragraph">
<p>A subset of groups are fetched based on filter conditions (<code>Group Filter Prefix</code>, <code>Group Filter Suffix</code>, <code>Group Filter Substring</code>, and <code>Group Filter List Inclusion</code>) evaluated against the <em>displayName</em> property of the Azure AD group. Member users are then loaded from these groups. At least one filter condition should be specified.</p>
</div>
<div class="paragraph">
<p>This provider requires an Azure app registration with:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Microsoft Graph Group.Read.All and User.Read.All API permissions with admin consent</p>
</li>
<li>
<p>A client secret or application password</p>
</li>
<li>
<p>ID token claims for upn and/or email</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>See <a href="https://docs.microsoft.com/en-us/graph/auth-v2-service" target="_blank" rel="noopener">here</a> and <a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/scenario-daemon-app-registration" target="_blank" rel="noopener">here</a> for more information on how to create a valid app registration.</p>
</div>
<div class="paragraph">
<p>The AzureGraphUserGroupProvider has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Refresh Delay</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Duration of delay between each user and group refresh. Default is <code>5 mins</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Authority Endpoint</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The endpoint of the Azure AD login. This can be found in the Azure portal under Azure Active Directory &#8594; App registrations &#8594; [application name] &#8594; Endpoints. For example, the global authority endpoint is <a href="https://login.microsoftonline.com" class="bare">https://login.microsoftonline.com</a>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Directory ID</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Tenant ID or Directory ID of the Azure AD tenant. This can be found in the Azure portal under Azure Active Directory &#8594; App registrations &#8594; [application name] &#8594; Directory (tenant) ID.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Application ID</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Client ID or Application ID of the Azure app registration. This can be found in the Azure portal under Azure Active Directory &#8594; App registrations &#8594; [application name] &#8594; Overview &#8594; Application (client) ID.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Client Secret</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A client secret from the Azure app registration. Secrets can be created in the Azure portal under Azure Active Directory &#8594; App registrations &#8594; [application name] &#8594; Certificates &amp; secrets &#8594; Client secrets &#8594; [+] New client secret.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Filter Prefix</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Prefix filter for Azure AD groups. Matches against the group displayName to retrieve only groups with names starting with the provided prefix.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Filter Suffix</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Suffix filter for Azure AD groups. Matches against the group displayName to retrieve only groups with names ending with the provided suffix.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Filter Substring</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Substring filter for Azure AD groups. Matches against the group displayName to retrieve only groups with names containing the provided substring.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Group Filter List Inclusion</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma-separated list of Azure AD groups. If no string-based matching filter (i.e., prefix, suffix, and substring) is specified, set this property to avoid fetching all groups and users in the Azure AD tenant.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Page Size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Page size to use with the Microsoft Graph API. Set to 0 to disable paging API calls. Default: 50, Max: 999.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Claim for Username</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The property of the user directory object mapped to the NiFi user name field. Default is 'upn'. 'email' is another option when <code>nifi.security.user.oidc.fallback.claims.identifying.user</code> is set to 'upn'.</p></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>Like LdapUserGroupProvider and ShellUserGroupProvider, the AzureGraphUserGroupProvider configuration is commented out in the <em>authorizers.xml</em> file. Refer to the comment for a starter configuration.</p>
</div>
</div>
<div class="sect3">
<h4 id="composite-implementations"><a class="anchor" href="administration-guide.html#composite-implementations"></a>Composite Implementations</h4>
<div class="paragraph">
<p>Another option for the UserGroupProvider are composite implementations. This means that multiple sources/implementations can be configured and composed. For instance, an admin can configure users/groups to be loaded from a file and a directory server. There are two composite implementations, one that supports multiple UserGroupProviders and one that supports multiple UserGroupProviders and a single configurable UserGroupProvider.</p>
</div>
<div class="paragraph">
<p>The CompositeUserGroupProvider will provide support for retrieving users and groups from multiple sources. The CompositeUserGroupProvider has the following property:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Group Provider [unique key]</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identifier of user group providers to load from. The name of each property must be unique, for example: "User Group Provider A", "User Group Provider B", "User Group Provider C" or "User Group Provider 1", "User Group Provider 2", "User Group Provider 3"</p></td>
</tr>
</tfoot>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Any identity mapping rules specified in <em>nifi.properties</em> are not applied in this implementation. This behavior would need to be applied by the base implementation.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The CompositeConfigurableUserGroupProvider will provide support for retrieving users and groups from multiple sources. Additionally, a single configurable user group provider is required. Users from the configurable user group provider are configurable, however users loaded from one of the User Group Provider [unique key] will not be. The CompositeConfigurableUserGroupProvider has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Configurable User Group Provider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A configurable user group provider.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Group Provider [unique key]</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identifier of user group providers to load from. The name of each property must be unique, for example: "User Group Provider A", "User Group Provider B", "User Group Provider C" or "User Group Provider 1", "User Group Provider 2", "User Group Provider 3"</p></td>
</tr>
</tfoot>
</table>
</div>
<div class="sect3">
<h4 id="fileaccesspolicyprovider"><a class="anchor" href="administration-guide.html#fileaccesspolicyprovider"></a>FileAccessPolicyProvider</h4>
<div class="paragraph">
<p>The default AccessPolicyProvider is the FileAccessPolicyProvider, however, you can develop additional AccessPolicyProvider as extensions. The FileAccessPolicyProvider has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>User Group Provider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identifier for an User Group Provider defined above that will be used to access users and groups for use in the managed access policies.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Authorizations File</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The file where the FileAccessPolicyProvider will store policies.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Initial Admin Identity</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identity of an initial admin user that will be granted access to the UI and given the ability to create additional users, groups, and policies. The value of this property could be a DN when using certificates or LDAP, or a Kerberos principal. This property will only be used when there are no other policies defined. If this property is specified then a Legacy Authorized Users File can not be specified.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Legacy Authorized Users File</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The full path to an existing <em>authorized-users.xml</em> that will be automatically converted to the new authorizations model. If this property is specified then an Initial Admin Identity can not be specified, and this property will only be used when there are no other users, groups, and policies defined.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Node Identity</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identity of a NiFi cluster node. When clustered, a property for each node should be defined, so that every node knows about every other node. If not clustered these properties can be ignored. The name of each property must be unique, for example for a three node cluster: "Node Identity A", "Node Identity B", "Node Identity C" or "Node Identity 1", "Node Identity 2", "Node Identity 3"</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Node Group</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of a group containing NiFi cluster nodes. The typical use for this is when nodes are dynamically added/removed from the cluster.</p></td>
</tr>
</tfoot>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
The identities configured in the Initial Admin Identity, the Node Identity properties, or discovered in a Legacy Authorized Users File must be available in the configured User Group Provider.
</td>
</tr>
</table>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Any users in the legacy users file must be found in the configured User Group Provider.
</td>
</tr>
</table>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Any identity mapping rules specified in <em>nifi.properties</em> will also be applied to the node identities,
so the values should be the unmapped identities (i.e. full DN from a certificate). This identity must be found
in the configured User Group Provider.
</td>
</tr>
</table>
</div>
</div>
<div class="sect3">
<h4 id="standardmanagedauthorizer"><a class="anchor" href="administration-guide.html#standardmanagedauthorizer"></a>StandardManagedAuthorizer</h4>
<div class="paragraph">
<p>The default authorizer is the StandardManagedAuthorizer, however, you can develop additional authorizers as extensions. The StandardManagedAuthorizer has the following property:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Access Policy Provider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identifier for an Access Policy Provider defined above.</p></td>
</tr>
</tfoot>
</table>
</div>
<div class="sect3">
<h4 id="fileauthorizer"><a class="anchor" href="administration-guide.html#fileauthorizer"></a>FileAuthorizer</h4>
<div class="paragraph">
<p>The FileAuthorizer has been replaced with the more granular StandardManagedAuthorizer approach described above. However, it is still available for backwards compatibility reasons. The FileAuthorizer has the following properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Authorizations File</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The file where the FileAuthorizer stores policies. By default, the <em>authorizations.xml</em> in the <code>conf</code> directory is chosen.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Users File</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The file where the FileAuthorizer stores users and groups. By default, the <em>users.xml</em> in the <code>conf</code> directory is chosen.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Initial Admin Identity</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identity of an initial admin user that is granted access to the UI and given the ability to create additional users, groups, and policies. This property is only used when there are no other users, groups, and policies defined.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Legacy Authorized Users File</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The full path to an existing <em>authorized-users.xml</em> that is automatically converted to the multi-tenant authorization model. This property is only used when there are no other users, groups, and policies defined.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Node Identity</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identity of a NiFi cluster node. When clustered, a property for each node should be defined, so that every node knows about every other node. If not clustered, these properties can be ignored.</p></td>
</tr>
</tfoot>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Any identity mapping rules specified in <em>nifi.properties</em> will also be applied to the initial admin identity, so the value should be the unmapped identity.
</td>
</tr>
</table>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Any identity mapping rules specified in <em>nifi.properties</em> will also be applied to the node identities, so the values should be the unmapped identities (i.e. full DN from a certificate).
</td>
</tr>
</table>
</div>
</div>
<div class="sect3">
<h4 id="initial-admin-identity"><a class="anchor" href="administration-guide.html#initial-admin-identity"></a>Initial Admin Identity (New NiFi Instance)</h4>
<div class="paragraph">
<p>If you are setting up a secured NiFi instance for the first time, you must manually designate an “Initial Admin Identity” in the <em>authorizers.xml</em> file. This initial admin user is granted access to the UI and given the ability to create additional users, groups, and policies. The value of this property could be a DN (when using certificates or LDAP) or a Kerberos principal. If you are the NiFi administrator, add yourself as the “Initial Admin Identity”.</p>
</div>
<div class="paragraph">
<p>After you have edited and saved the <em>authorizers.xml</em> file, restart NiFi. The “Initial Admin Identity” user and administrative policies are added to the <em>users.xml</em> and <em>authorizations.xml</em> files during restart. Once NiFi starts, the “Initial Admin Identity” user is able to access the UI and begin managing users, groups, and policies.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
For a brand new secure flow, providing the "Initial Admin Identity" gives that user access to get into the UI and to manage users, groups and policies. But if that user wants to start
modifying the flow, they need to grant themselves policies for the root process group. The system is unable to do this automatically because in a new flow the UUID of the root process group is not
permanent until the <em>flow.json.gz</em> is generated. If the NiFi instance is an upgrade from an existing <em>flow.json.gz</em> or a 1.x instance going from unsecure to secure, then the "Initial Admin Identity"
user
is automatically given the privileges to modify the flow.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Some common use cases are described below.</p>
</div>
<div class="sect4">
<h5 id="file-based-ldap-authentication"><a class="anchor" href="administration-guide.html#file-based-ldap-authentication"></a>File-based (LDAP Authentication)</h5>
<div class="paragraph">
<p>Here is an example LDAP entry using the name John Smith:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>&lt;authorizers&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;file-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileUserGroupProvider&lt;/class&gt;
&lt;property name="Users File"&gt;./conf/users.xml&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Initial User Identity 1"&gt;cn=John Smith,ou=people,dc=example,dc=com&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;accessPolicyProvider&gt;
&lt;identifier&gt;file-access-policy-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileAccessPolicyProvider&lt;/class&gt;
&lt;property name="User Group Provider"&gt;file-user-group-provider&lt;/property&gt;
&lt;property name="Authorizations File"&gt;./conf/authorizations.xml&lt;/property&gt;
&lt;property name="Initial Admin Identity"&gt;cn=John Smith,ou=people,dc=example,dc=com&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Node Identity 1"&gt;&lt;/property&gt;
&lt;/accessPolicyProvider&gt;
&lt;authorizer&gt;
&lt;identifier&gt;managed-authorizer&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.StandardManagedAuthorizer&lt;/class&gt;
&lt;property name="Access Policy Provider"&gt;file-access-policy-provider&lt;/property&gt;
&lt;/authorizer&gt;
&lt;/authorizers&gt;</pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="file-based-kerberos-authentication"><a class="anchor" href="administration-guide.html#file-based-kerberos-authentication"></a>File-based (Kerberos Authentication)</h5>
<div class="paragraph">
<p>Here is an example Kerberos entry using the name John Smith and realm <code>NIFI.APACHE.ORG</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>&lt;authorizers&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;file-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileUserGroupProvider&lt;/class&gt;
&lt;property name="Users File"&gt;./conf/users.xml&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Initial User Identity 1"&gt;johnsmith@NIFI.APACHE.ORG&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;accessPolicyProvider&gt;
&lt;identifier&gt;file-access-policy-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileAccessPolicyProvider&lt;/class&gt;
&lt;property name="User Group Provider"&gt;file-user-group-provider&lt;/property&gt;
&lt;property name="Authorizations File"&gt;./conf/authorizations.xml&lt;/property&gt;
&lt;property name="Initial Admin Identity"&gt;johnsmith@NIFI.APACHE.ORG&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Node Identity 1"&gt;&lt;/property&gt;
&lt;/accessPolicyProvider&gt;
&lt;authorizer&gt;
&lt;identifier&gt;managed-authorizer&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.StandardManagedAuthorizer&lt;/class&gt;
&lt;property name="Access Policy Provider"&gt;file-access-policy-provider&lt;/property&gt;
&lt;/authorizer&gt;
&lt;/authorizers&gt;</pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="ldap-based-usersgroups-referencing-user-dn"><a class="anchor" href="administration-guide.html#ldap-based-usersgroups-referencing-user-dn"></a>LDAP-based Users/Groups Referencing User DN</h5>
<div class="paragraph">
<p>Here is an example loading users and groups from LDAP. Group membership will be driven through the member attribute of each group. Authorization will still use file-based access policies:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>dn: cn=User 1,ou=users,o=nifi
objectClass: organizationalPerson
objectClass: person
objectClass: inetOrgPerson
objectClass: top
cn: User 1
sn: User1
uid: user1
dn: cn=User 2,ou=users,o=nifi
objectClass: organizationalPerson
objectClass: person
objectClass: inetOrgPerson
objectClass: top
cn: User 2
sn: User2
uid: user2
dn: cn=admins,ou=groups,o=nifi
objectClass: groupOfNames
objectClass: top
cn: admins
member: cn=User 1,ou=users,o=nifi
member: cn=User 2,ou=users,o=nifi
&lt;authorizers&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;ldap-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.ldap.tenants.LdapUserGroupProvider&lt;/class&gt;
&lt;property name="Authentication Strategy"&gt;ANONYMOUS&lt;/property&gt;
&lt;property name="Manager DN"&gt;&lt;/property&gt;
&lt;property name="Manager Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Client Auth"&gt;&lt;/property&gt;
&lt;property name="TLS - Protocol"&gt;&lt;/property&gt;
&lt;property name="TLS - Shutdown Gracefully"&gt;&lt;/property&gt;
&lt;property name="Referral Strategy"&gt;FOLLOW&lt;/property&gt;
&lt;property name="Connect Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Read Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Url"&gt;ldap://localhost:10389&lt;/property&gt;
&lt;property name="Page Size"&gt;&lt;/property&gt;
&lt;property name="Sync Interval"&gt;30 mins&lt;/property&gt;
&lt;property name="Group Membership - Enforce Case Sensitivity"&gt;false&lt;/property&gt;
&lt;property name="User Search Base"&gt;ou=users,o=nifi&lt;/property&gt;
&lt;property name="User Object Class"&gt;person&lt;/property&gt;
&lt;property name="User Search Scope"&gt;ONE_LEVEL&lt;/property&gt;
&lt;property name="User Search Filter"&gt;&lt;/property&gt;
&lt;property name="User Identity Attribute"&gt;cn&lt;/property&gt;
&lt;property name="User Group Name Attribute"&gt;&lt;/property&gt;
&lt;property name="User Group Name Attribute - Referenced Group Attribute"&gt;&lt;/property&gt;
&lt;property name="Group Search Base"&gt;ou=groups,o=nifi&lt;/property&gt;
&lt;property name="Group Object Class"&gt;groupOfNames&lt;/property&gt;
&lt;property name="Group Search Scope"&gt;ONE_LEVEL&lt;/property&gt;
&lt;property name="Group Search Filter"&gt;&lt;/property&gt;
&lt;property name="Group Name Attribute"&gt;cn&lt;/property&gt;
&lt;property name="Group Member Attribute"&gt;member&lt;/property&gt;
&lt;property name="Group Member Attribute - Referenced User Attribute"&gt;&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;accessPolicyProvider&gt;
&lt;identifier&gt;file-access-policy-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileAccessPolicyProvider&lt;/class&gt;
&lt;property name="User Group Provider"&gt;ldap-user-group-provider&lt;/property&gt;
&lt;property name="Authorizations File"&gt;./conf/authorizations.xml&lt;/property&gt;
&lt;property name="Initial Admin Identity"&gt;John Smith&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Node Identity 1"&gt;&lt;/property&gt;
&lt;/accessPolicyProvider&gt;
&lt;authorizer&gt;
&lt;identifier&gt;managed-authorizer&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.StandardManagedAuthorizer&lt;/class&gt;
&lt;property name="Access Policy Provider"&gt;file-access-policy-provider&lt;/property&gt;
&lt;/authorizer&gt;
&lt;/authorizers&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>Initial Admin Identity</code> value would have loaded from the cn from John Smith&#8217;s entry based on the <code>User Identity Attribute</code> value.</p>
</div>
</div>
<div class="sect4">
<h5 id="ldap-based-usersgroups-referencing-user-attribute"><a class="anchor" href="administration-guide.html#ldap-based-usersgroups-referencing-user-attribute"></a>LDAP-based Users/Groups Referencing User Attribute</h5>
<div class="paragraph">
<p>Here is an example loading users and groups from LDAP. Group membership will be driven through the member uid attribute of each group. Authorization will still use file-based access policies:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>dn: uid=User 1,ou=Users,dc=local
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
uid: user1
cn: User 1
dn: uid=User 2,ou=Users,dc=local
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
uid: user2
cn: User 2
dn: cn=Managers,ou=Groups,dc=local
objectClass: posixGroup
cn: Managers
memberUid: user1
memberUid: user2
&lt;authorizers&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;ldap-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.ldap.tenants.LdapUserGroupProvider&lt;/class&gt;
&lt;property name="Authentication Strategy"&gt;ANONYMOUS&lt;/property&gt;
&lt;property name="Manager DN"&gt;&lt;/property&gt;
&lt;property name="Manager Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Client Auth"&gt;&lt;/property&gt;
&lt;property name="TLS - Protocol"&gt;&lt;/property&gt;
&lt;property name="TLS - Shutdown Gracefully"&gt;&lt;/property&gt;
&lt;property name="Referral Strategy"&gt;FOLLOW&lt;/property&gt;
&lt;property name="Connect Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Read Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Url"&gt;ldap://localhost:10389&lt;/property&gt;
&lt;property name="Page Size"&gt;&lt;/property&gt;
&lt;property name="Sync Interval"&gt;30 mins&lt;/property&gt;
&lt;property name="Group Membership - Enforce Case Sensitivity"&gt;false&lt;/property&gt;
&lt;property name="User Search Base"&gt;ou=Users,dc=local&lt;/property&gt;
&lt;property name="User Object Class"&gt;posixAccount&lt;/property&gt;
&lt;property name="User Search Scope"&gt;ONE_LEVEL&lt;/property&gt;
&lt;property name="User Search Filter"&gt;&lt;/property&gt;
&lt;property name="User Identity Attribute"&gt;cn&lt;/property&gt;
&lt;property name="User Group Name Attribute"&gt;&lt;/property&gt;
&lt;property name="User Group Name Attribute - Referenced Group Attribute"&gt;&lt;/property&gt;
&lt;property name="Group Search Base"&gt;ou=Groups,dc=local&lt;/property&gt;
&lt;property name="Group Object Class"&gt;posixGroup&lt;/property&gt;
&lt;property name="Group Search Scope"&gt;ONE_LEVEL&lt;/property&gt;
&lt;property name="Group Search Filter"&gt;&lt;/property&gt;
&lt;property name="Group Name Attribute"&gt;cn&lt;/property&gt;
&lt;property name="Group Member Attribute"&gt;memberUid&lt;/property&gt;
&lt;property name="Group Member Attribute - Referenced User Attribute"&gt;uid&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;accessPolicyProvider&gt;
&lt;identifier&gt;file-access-policy-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileAccessPolicyProvider&lt;/class&gt;
&lt;property name="User Group Provider"&gt;ldap-user-group-provider&lt;/property&gt;
&lt;property name="Authorizations File"&gt;./conf/authorizations.xml&lt;/property&gt;
&lt;property name="Initial Admin Identity"&gt;John Smith&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Node Identity 1"&gt;&lt;/property&gt;
&lt;/accessPolicyProvider&gt;
&lt;authorizer&gt;
&lt;identifier&gt;managed-authorizer&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.StandardManagedAuthorizer&lt;/class&gt;
&lt;property name="Access Policy Provider"&gt;file-access-policy-provider&lt;/property&gt;
&lt;/authorizer&gt;
&lt;/authorizers&gt;</pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="composite-file-and-ldap-based-usersgroups"><a class="anchor" href="administration-guide.html#composite-file-and-ldap-based-usersgroups"></a>Composite - File and LDAP-based Users/Groups</h5>
<div class="paragraph">
<p>Here is an example composite implementation loading users and groups from LDAP and a local file. Group membership will be driven through the member attribute of each group. The users from LDAP will be read only while the users loaded from the file will be configurable in UI.</p>
</div>
<div class="listingblock">
<div class="content">
<pre>dn: cn=User 1,ou=users,o=nifi
objectClass: organizationalPerson
objectClass: person
objectClass: inetOrgPerson
objectClass: top
cn: User 1
sn: User1
uid: user1
dn: cn=User 2,ou=users,o=nifi
objectClass: organizationalPerson
objectClass: person
objectClass: inetOrgPerson
objectClass: top
cn: User 2
sn: User2
uid: user2
dn: cn=admins,ou=groups,o=nifi
objectClass: groupOfNames
objectClass: top
cn: admins
member: cn=User 1,ou=users,o=nifi
member: cn=User 2,ou=users,o=nifi
&lt;authorizers&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;file-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileUserGroupProvider&lt;/class&gt;
&lt;property name="Users File"&gt;./conf/users.xml&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Initial User Identity 1"&gt;cn=nifi-node1,ou=servers,dc=example,dc=com&lt;/property&gt;
&lt;property name="Initial User Identity 2"&gt;cn=nifi-node2,ou=servers,dc=example,dc=com&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;ldap-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.ldap.tenants.LdapUserGroupProvider&lt;/class&gt;
&lt;property name="Authentication Strategy"&gt;ANONYMOUS&lt;/property&gt;
&lt;property name="Manager DN"&gt;&lt;/property&gt;
&lt;property name="Manager Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Keystore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Password"&gt;&lt;/property&gt;
&lt;property name="TLS - Truststore Type"&gt;&lt;/property&gt;
&lt;property name="TLS - Client Auth"&gt;&lt;/property&gt;
&lt;property name="TLS - Protocol"&gt;&lt;/property&gt;
&lt;property name="TLS - Shutdown Gracefully"&gt;&lt;/property&gt;
&lt;property name="Referral Strategy"&gt;FOLLOW&lt;/property&gt;
&lt;property name="Connect Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Read Timeout"&gt;10 secs&lt;/property&gt;
&lt;property name="Url"&gt;ldap://localhost:10389&lt;/property&gt;
&lt;property name="Page Size"&gt;&lt;/property&gt;
&lt;property name="Sync Interval"&gt;30 mins&lt;/property&gt;
&lt;property name="Group Membership - Enforce Case Sensitivity"&gt;false&lt;/property&gt;
&lt;property name="User Search Base"&gt;ou=users,o=nifi&lt;/property&gt;
&lt;property name="User Object Class"&gt;person&lt;/property&gt;
&lt;property name="User Search Scope"&gt;ONE_LEVEL&lt;/property&gt;
&lt;property name="User Search Filter"&gt;&lt;/property&gt;
&lt;property name="User Identity Attribute"&gt;cn&lt;/property&gt;
&lt;property name="User Group Name Attribute"&gt;&lt;/property&gt;
&lt;property name="User Group Name Attribute - Referenced Group Attribute"&gt;&lt;/property&gt;
&lt;property name="Group Search Base"&gt;ou=groups,o=nifi&lt;/property&gt;
&lt;property name="Group Object Class"&gt;groupOfNames&lt;/property&gt;
&lt;property name="Group Search Scope"&gt;ONE_LEVEL&lt;/property&gt;
&lt;property name="Group Search Filter"&gt;&lt;/property&gt;
&lt;property name="Group Name Attribute"&gt;cn&lt;/property&gt;
&lt;property name="Group Member Attribute"&gt;member&lt;/property&gt;
&lt;property name="Group Member Attribute - Referenced User Attribute"&gt;&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;composite-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider&lt;/class&gt;
&lt;property name="Configurable User Group Provider"&gt;file-user-group-provider&lt;/property&gt;
&lt;property name="User Group Provider 1"&gt;ldap-user-group-provider&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;accessPolicyProvider&gt;
&lt;identifier&gt;file-access-policy-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileAccessPolicyProvider&lt;/class&gt;
&lt;property name="User Group Provider"&gt;composite-user-group-provider&lt;/property&gt;
&lt;property name="Authorizations File"&gt;./conf/authorizations.xml&lt;/property&gt;
&lt;property name="Initial Admin Identity"&gt;John Smith&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Node Identity 1"&gt;cn=nifi-node1,ou=servers,dc=example,dc=com&lt;/property&gt;
&lt;property name="Node Identity 2"&gt;cn=nifi-node2,ou=servers,dc=example,dc=com&lt;/property&gt;
&lt;/accessPolicyProvider&gt;
&lt;authorizer&gt;
&lt;identifier&gt;managed-authorizer&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.StandardManagedAuthorizer&lt;/class&gt;
&lt;property name="Access Policy Provider"&gt;file-access-policy-provider&lt;/property&gt;
&lt;/authorizer&gt;
&lt;/authorizers&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>In this example, the users and groups are loaded from LDAP but the servers are managed in a local file. The <code>Initial Admin Identity</code> value came from an attribute in a LDAP entry based on the <code>User Identity Attribute</code>. The <code>Node Identity</code> values are established in the local file using the <code>Initial User Identity</code> properties.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="legacy-authorized-users"><a class="anchor" href="administration-guide.html#legacy-authorized-users"></a>Legacy Authorized Users (NiFi Instance Upgrade)</h4>
<div class="paragraph">
<p>If you are upgrading from a 0.x NiFi instance, you can convert your previously configured users and roles to the multi-tenant authorization model. In the <em>authorizers.xml</em> file, specify the location of your existing <em>authorized-users.xml</em> file in the <code>Legacy Authorized Users File</code> property.</p>
</div>
<div class="paragraph">
<p>Here is an example entry:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>&lt;authorizers&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;file-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileUserGroupProvider&lt;/class&gt;
&lt;property name="Users File"&gt;./conf/users.xml&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;/Users/johnsmith/config_files/authorized-users.xml&lt;/property&gt;
&lt;property name="Initial User Identity 1"&gt;&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;accessPolicyProvider&gt;
&lt;identifier&gt;file-access-policy-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileAccessPolicyProvider&lt;/class&gt;
&lt;property name="User Group Provider"&gt;file-user-group-provider&lt;/property&gt;
&lt;property name="Authorizations File"&gt;./conf/authorizations.xml&lt;/property&gt;
&lt;property name="Initial Admin Identity"&gt;&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;/Users/johnsmith/config_files/authorized-users.xml&lt;/property&gt;
&lt;property name="Node Identity 1"&gt;&lt;/property&gt;
&lt;/accessPolicyProvider&gt;
&lt;authorizer&gt;
&lt;identifier&gt;managed-authorizer&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.StandardManagedAuthorizer&lt;/class&gt;
&lt;property name="Access Policy Provider"&gt;file-access-policy-provider&lt;/property&gt;
&lt;/authorizer&gt;
&lt;/authorizers&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>After you have edited and saved the <em>authorizers.xml</em> file, restart NiFi. Users and roles from the <em>authorized-users.xml</em> file are converted and added as identities and policies in the <em>users.xml</em> and <em>authorizations.xml</em> files. Once the application starts, users who previously had a legacy Administrator role can access the UI and begin managing users, groups, and policies.</p>
</div>
<div class="paragraph">
<p>The following tables summarize the global and component policies assigned to each legacy role if the NiFi instance has an existing <em>flow.json.gz</em>:</p>
</div>
<div class="sect4">
<h5 id="global-access-policies"><a class="anchor" href="administration-guide.html#global-access-policies"></a>Global Access Policies</h5>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2858%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-right valign-top"></th>
<th class="tableblock halign-center valign-top">Admin</th>
<th class="tableblock halign-center valign-top">DFM</th>
<th class="tableblock halign-center valign-top">Monitor</th>
<th class="tableblock halign-center valign-top">Provenance</th>
<th class="tableblock halign-center valign-top">NiFi</th>
<th class="tableblock halign-center valign-top">Proxy</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the UI</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access the controller - view</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access the controller - modify</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access parameter contexts - view</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access parameter contexts - modify</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>query provenance</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access restricted components</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access all policies - view</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access all policies - modify</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access users/user groups - view</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access users/user groups - modify</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>retrieve site-to-site details</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view system diagnostics</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>proxy user requests</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>access counters</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
</tbody>
</table>
</div>
<div class="sect4">
<h5 id="component-access-policies-on-the-root-process-group"><a class="anchor" href="administration-guide.html#component-access-policies-on-the-root-process-group"></a>Component Access Policies on the Root Process Group</h5>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2857%;">
<col style="width: 14.2858%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-right valign-top"></th>
<th class="tableblock halign-center valign-top">Admin</th>
<th class="tableblock halign-center valign-top">DFM</th>
<th class="tableblock halign-center valign-top">Monitor</th>
<th class="tableblock halign-center valign-top">Provenance</th>
<th class="tableblock halign-center valign-top">NiFi</th>
<th class="tableblock halign-center valign-top">Proxy</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the component</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>modify the component</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view the data</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>modify the data</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view provenance</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td>
<td class="tableblock halign-center valign-top"></td>
<td class="tableblock halign-center valign-top"></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>For details on the individual policies in the table, see <a href="administration-guide.html#access-policies">Access Policies</a>.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
NiFi fails to restart if values exist for both the <code>Initial Admin Identity</code> and <code>Legacy Authorized Users File</code> properties. You can specify only one of these values to initialize authorizations.
</td>
</tr>
</table>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Do not manually edit the <em>authorizations.xml</em> file. Create authorizations only during initial setup and afterwards using the NiFi UI.
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect3">
<h4 id="cluster-node-identities"><a class="anchor" href="administration-guide.html#cluster-node-identities"></a>Cluster Node Identities</h4>
<div class="paragraph">
<p>If you are running NiFi in a clustered environment, you must specify the identities for each node. The authorization policies required for the nodes to communicate are created during startup.</p>
</div>
<div class="paragraph">
<p>For example, if you are setting up a 2 node cluster with the following DNs for each node:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>cn=nifi-1,ou=people,dc=example,dc=com
cn=nifi-2,ou=people,dc=example,dc=com</pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre>&lt;authorizers&gt;
&lt;userGroupProvider&gt;
&lt;identifier&gt;file-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileUserGroupProvider&lt;/class&gt;
&lt;property name="Users File"&gt;./conf/users.xml&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Initial User Identity 1"&gt;johnsmith@NIFI.APACHE.ORG&lt;/property&gt;
&lt;property name="Initial User Identity 2"&gt;cn=nifi-1,ou=people,dc=example,dc=com&lt;/property&gt;
&lt;property name="Initial User Identity 3"&gt;cn=nifi-2,ou=people,dc=example,dc=com&lt;/property&gt;
&lt;/userGroupProvider&gt;
&lt;accessPolicyProvider&gt;
&lt;identifier&gt;file-access-policy-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.FileAccessPolicyProvider&lt;/class&gt;
&lt;property name="User Group Provider"&gt;file-user-group-provider&lt;/property&gt;
&lt;property name="Authorizations File"&gt;./conf/authorizations.xml&lt;/property&gt;
&lt;property name="Initial Admin Identity"&gt;johnsmith@NIFI.APACHE.ORG&lt;/property&gt;
&lt;property name="Legacy Authorized Users File"&gt;&lt;/property&gt;
&lt;property name="Node Identity 1"&gt;cn=nifi-1,ou=people,dc=example,dc=com&lt;/property&gt;
&lt;property name="Node Identity 2"&gt;cn=nifi-2,ou=people,dc=example,dc=com&lt;/property&gt;
&lt;/accessPolicyProvider&gt;
&lt;authorizer&gt;
&lt;identifier&gt;managed-authorizer&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.authorization.StandardManagedAuthorizer&lt;/class&gt;
&lt;property name="Access Policy Provider"&gt;file-access-policy-provider&lt;/property&gt;
&lt;/authorizer&gt;
&lt;/authorizers&gt;</pre>
</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
In a cluster, all nodes must have the same <em>authorizations.xml</em> and <em>users.xml</em>. The only exception is if a node has empty <em>authorizations.xml</em> and <em>user.xml</em> files prior to joining the cluster. In this scenario, the node inherits them from the cluster during startup.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Now that initial authorizations have been created, additional users, groups and authorizations can be created and managed in the NiFi UI.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="config-users-access-policies"><a class="anchor" href="administration-guide.html#config-users-access-policies"></a>Configuring Users &amp; Access Policies</h3>
<div class="paragraph">
<p>Depending on the capabilities of the configured UserGroupProvider and AccessPolicyProvider the users, groups, and policies will be configurable in the UI. If the extensions are not configurable the
users, groups, and policies will read-only in the UI. If the configured authorizer does not use UserGroupProvider and AccessPolicyProvider the users and policies may or may not be visible and
configurable in the UI based on the underlying implementation.</p>
</div>
<div class="paragraph">
<p>This section assumes the users, groups, and policies are configurable in the UI and describes:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>How to create users and groups</p>
</li>
<li>
<p>How access policies are used to define authorizations</p>
</li>
<li>
<p>How to view policies that are set on a user</p>
</li>
<li>
<p>How to configure access policies by walking through specific examples</p>
</li>
</ul>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Instructions requiring interaction with the UI assume the application is being accessed by User1, a user with administrator privileges, such as the “Initial Admin Identity” user or a converted legacy admin user (see <a href="administration-guide.html#authorizers-setup">Authorizers.xml Setup</a>).
</td>
</tr>
</table>
</div>
<div class="sect3">
<h4 id="creating-users-groups"><a class="anchor" href="administration-guide.html#creating-users-groups"></a>Creating Users and Groups</h4>
<div class="paragraph">
<p>From the UI, select “Users” from the Global Menu. This opens a dialog to create and manage users and groups.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/nifi-users-dialog.png" alt="NiFi Users Dialog"></span></p>
</div>
<div class="paragraph">
<p>Click the Add icon (<span class="image"><img src="images/iconAddUser.png" alt="Add User Icon"></span>). To create a user, enter the 'Identity' information relevant to the authentication method chosen to secure your NiFi instance. Click OK.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user-creation-dialog.png" alt="User Creation Dialog"></span></p>
</div>
<div class="paragraph">
<p>To create a group, select the “Group” radio button, enter the name of the group and select the users to be included in the group. Click OK.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/group-creation-dialog.png" alt="Group Creation Dialog"></span></p>
</div>
</div>
<div class="sect3">
<h4 id="access-policies"><a class="anchor" href="administration-guide.html#access-policies"></a>Access Policies</h4>
<div class="paragraph">
<p>You can manage the ability for users and groups to view or modify NiFi resources using 'access policies'. There are two types of access policies that can be applied to a resource:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>View&#8201;&#8212;&#8201; If a view policy is created for a resource, only the users or groups that are added to that policy are able to see the details of that resource.</p>
</li>
<li>
<p>Modify&#8201;&#8212;&#8201;If a resource has a modify policy, only the users or groups that are added to that policy can change the configuration of that resource.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>You can create and apply access policies on both global and component levels.</p>
</div>
<div class="sect4">
<h5 id="global-access-policies"><a class="anchor" href="administration-guide.html#global-access-policies"></a>Global Access Policies</h5>
<div class="paragraph">
<p>Global access policies govern the following system level authorizations:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 25%;">
<col style="width: 25%;">
<col style="width: 25%;">
<col style="width: 25%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Policy</th>
<th class="tableblock halign-left valign-top">Privilege</th>
<th class="tableblock halign-left valign-top">Global Menu Selection</th>
<th class="tableblock halign-left valign-top">Resource Descriptor</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">view the UI</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view the UI</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/flow</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">access the controller</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the controller including Management Controller Services, Reporting Tasks, Registry Clients, Parameter Providers and nodes in the cluster</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Controller Settings</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/controller</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">access parameter contexts</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify Parameter Contexts. Access to Parameter Contexts are inherited from the "access the controller" policies unless overridden.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Parameter Contexts</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/parameter-contexts</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">query provenance</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to submit a Provenance Search and request Event Lineage</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Data Provenance</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/provenance</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">access restricted components</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to create/modify restricted components assuming other permissions are sufficient. The restricted
components may indicate which specific permissions are required. Permissions can be granted for specific
restrictions or be granted regardless of restrictions. If permission is granted regardless of restrictions,
the user can create/modify all restricted components.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/restricted-components</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">access all policies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the policies for all components</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Policies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/policies</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">access users/user groups</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the users and user groups</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Users</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/tenants</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">retrieve site-to-site details</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows other NiFi instances to retrieve Site-To-Site details</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/site-to-site</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">view system diagnostics</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view System Diagnostics</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Summary</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/system</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">proxy user requests</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows proxy machines to send requests on the behalf of others</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/proxy</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">access counters</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify Counters</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Counters</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/counters</code></p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect4">
<h5 id="component-level-access-policies"><a class="anchor" href="administration-guide.html#component-level-access-policies"></a>Component Level Access Policies</h5>
<div class="paragraph">
<p>Component level access policies govern the following component level authorizations:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Policy</th>
<th class="tableblock halign-left valign-top">Privilege</th>
<th class="tableblock halign-left valign-top">Resource Descriptor &amp; Action</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">view the component</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view component configuration details</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/&lt;component-type&gt;/&lt;component-UUID&gt;" action="R"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">modify the component</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to modify component configuration details</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/&lt;component-type&gt;/&lt;component-UUID&gt;" action="W"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">operate the component</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to operate components by changing component run status (start/stop/enable/disable), remote port transmission status, or terminating processor threads</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/operation/&lt;component-type&gt;/&lt;component-UUID&gt;" action="W"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">view provenance</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view provenance events generated by this component</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/provenance-data/&lt;component-type&gt;/&lt;component-UUID&gt;" action="R"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">view the data</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view metadata and content for this component in flowfile queues in outbound connections and through provenance events</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data/&lt;component-type&gt;/&lt;component-UUID&gt;" action="R"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">modify the data</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to empty flowfile queues in outbound connections and submit replays through provenance events</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data/&lt;component-type&gt;/&lt;component-UUID&gt;" action="W"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">view the policies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view the list of users who can view/modify a component</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/policies/&lt;component-type&gt;/&lt;component-UUID&gt;" action="R"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">modify the policies</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to modify the list of users who can view/modify a component</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/policies/&lt;component-type&gt;/&lt;component-UUID&gt;" action="W"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">receive data via site-to-site</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows a port to receive data from NiFi instances</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data-transfer/input-ports/&lt;port-UUID&gt;" action="W"</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">send data via site-to-site</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows a port to send data from NiFi instances</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data-transfer/output-ports/&lt;port-UUID&gt;" action="W"</code></p></td>
</tr>
</tbody>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
You can apply access policies to all component types except connections. Connection authorizations are inferred by the individual access policies on the source and destination components of the connection, as well as the access policy of the process group containing the components. This is discussed in more detail in the <a href="administration-guide.html#creating-a-connection">Creating a Connection</a> and <a href="administration-guide.html#editing-a-connection">Editing a Connection</a> examples below.
</td>
</tr>
</table>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
In order to access List Queue or Delete Queue for a connection, a user requires permission to the "view the data" and "modify the data" policies on the component. In a clustered environment, all nodes must be be added to these policies as well, as a user request could be replicated through any node in the cluster.
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="access-policy-inheritance"><a class="anchor" href="administration-guide.html#access-policy-inheritance"></a>Access Policy Inheritance</h5>
<div class="paragraph">
<p>An administrator does not need to manually create policies for every component in the dataflow. To reduce the amount of time admins spend on authorization management, policies are inherited from parent resource to child resource. For example, if a user is given access to view and modify a process group, that user can also view and modify the components in the process group. Policy inheritance enables an administrator to assign policies at one time and have the policies apply throughout the entire dataflow.</p>
</div>
<div class="paragraph">
<p>You can override an inherited policy (as described in the <a href="administration-guide.html#moving-a-processor">Moving a Processor</a> example below). Overriding a policy removes the inherited policy, breaking the chain of inheritance from parent to child, and creates a replacement policy to add users as desired. Inherited policies and their users can be restored by deleting the replacement policy.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
“View the policies” and “modify the policies” component-level access policies are an exception to this inherited behavior. When a user is added to either policy, they are added to the current list of administrators. They do not override higher level administrators. For this reason, only component specific administrators are displayed for the “view the policies” and “modify the policies" access policies.
</td>
</tr>
</table>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
You cannot modify the users/groups on an inherited policy. Users and groups can only be added or removed from a parent policy or an override policy.
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect3">
<h4 id="viewing-policies-users"><a class="anchor" href="administration-guide.html#viewing-policies-users"></a>Viewing Policies on Users</h4>
<div class="paragraph">
<p>From the UI, select “Users” from the Global Menu. This opens the NiFi Users dialog.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user-policies.png" alt="User Policies Window"></span></p>
</div>
<div class="paragraph">
<p>Select the View User Policies icon (<span class="image"><img src="images/iconUserPolicies.png" alt="User Policies Icon"></span>).</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user-policies-detail.png" alt="User Policies Detail"></span></p>
</div>
<div class="paragraph">
<p>The User Policies window displays the global and component level policies that have been set for the chosen user. Select the Go To icon (<span class="image"><img src="images/iconGoTo.png" alt="Go To Icon"></span>) to navigate to that component in the canvas.</p>
</div>
</div>
<div class="sect3">
<h4 id="access-policy-config-examples"><a class="anchor" href="administration-guide.html#access-policy-config-examples"></a>Access Policy Configuration Examples</h4>
<div class="paragraph">
<p>The most effective way to understand how to create and apply access policies is to walk through some common examples. The following scenarios assume User1 is an administrator and User2 is a newly added user that has only been given access to the UI.</p>
</div>
<div class="paragraph">
<p>Let’s begin with two processors on the canvas as our starting point: GenerateFlowFile and LogAttribute.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/access-policy-config-start.png" alt="Access Policy Config Start"></span></p>
</div>
<div class="paragraph">
<p>User1 can add components to the dataflow and is able to move, edit and connect all processors. The details and properties of the root process group and processors are visible to User1.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user1-full-access.png" alt="User1 Full Access"></span></p>
</div>
<div class="paragraph">
<p>User1 wants to maintain their current privileges to the dataflow and its components.</p>
</div>
<div class="paragraph">
<p>User2 is unable to add components to the dataflow or move, edit, or connect components. The details and properties of the root process group and processors are hidden from User2.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user2-restricted-access.png" alt="User2 Restricted Access"></span></p>
</div>
<div class="sect4">
<h5 id="moving-a-processor"><a class="anchor" href="administration-guide.html#moving-a-processor"></a>Moving a Processor</h5>
<div class="paragraph">
<p>To allow User2 to move the GenerateFlowFile processor in the dataflow and only that processor, User1 performs the following steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Select the GenerateFlowFile processor so that it is highlighted.</p>
</li>
<li>
<p>Select the Access Policies icon (<span class="image"><img src="images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette and the Access Policies dialog opens.</p>
</li>
<li>
<p>Select “modify the component” from the policy drop-down. The “modify the component” policy that currently exists on the processor (child) is the “modify the component” policy inherited from the root process group (parent) on which User1 has privileges.</p>
<div class="imageblock">
<div class="content">
<img src="images/processor-modify-policy.png" alt="Processor Modify Policy">
</div>
</div>
</li>
<li>
<p>Select the Override link in the policy inheritance message. When creating the replacement policy, you are given a choice to override with a copy of the inherited policy or an empty policy. Select the Override button to create a copy.</p>
<div class="imageblock">
<div class="content">
<img src="images/override_policy_copy_empty.png" alt="Create Override Policy">
</div>
</div>
</li>
<li>
<p>On the replacement policy that is created, select the Add User icon (<span class="image"><img src="images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User2 in the User Identity field and select OK. With these changes, User1 maintains the ability to move both processors on the canvas. User2 can now move the GenerateFlowFile processor but cannot move the LogAttribute processor.</p>
<div class="imageblock">
<div class="content">
<img src="images/processor-replacement-modify-policy.png" alt="Processor Replacement Modify Policy">
</div>
</div>
<div class="imageblock">
<div class="content">
<img src="images/user2-moved-processor.png" alt="User2 Moved Processor">
</div>
</div>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="editing-a-processor"><a class="anchor" href="administration-guide.html#editing-a-processor"></a>Editing a Processor</h5>
<div class="paragraph">
<p>In the “Moving a Processor” example above, User2 was added to the “modify the component” policy for GenerateFlowFile. Without the ability to view the processor properties, User2 is unable to modify the processor’s configuration. In order to edit a component, a user must be on both the “view the component” and “modify the component” policies. To implement this, User1 performs the following steps:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Select the GenerateFlowFile processor.</p>
</li>
<li>
<p>Select the Access Policies icon (<span class="image"><img src="images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette and the Access Policies dialog opens.</p>
</li>
<li>
<p>Select "view the component” from the policy drop-down. The view the component” policy that currently exists on the processor (child) is the "view the component” policy inherited from the root process group (parent) on which User1 has privileges.</p>
<div class="imageblock">
<div class="content">
<img src="images/processor-view-policy.png" alt="Processor View Policy">
</div>
</div>
</li>
<li>
<p>Select the Override link in the policy inheritance message, keep the default of Copy policy and select the Override button.</p>
</li>
<li>
<p>On the override policy that is created, select the Add User icon (<span class="image"><img src="images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User2 in the User Identity field and select OK. With these changes, User1 maintains the ability to view and edit the processors on the canvas. User2 can now view and edit the GenerateFlowFile processor.</p>
<div class="imageblock">
<div class="content">
<img src="images/processor-replacement-view-policy.png" alt="Processor Replacement View Policy">
</div>
</div>
<div class="imageblock">
<div class="content">
<img src="images/user2-edit-processor.png" alt="User2 Edit Processor">
</div>
</div>
</li>
</ol>
</div>
</div>
<div class="sect4">
<h5 id="creating-a-connection"><a class="anchor" href="administration-guide.html#creating-a-connection"></a>Creating a Connection</h5>
<div class="paragraph">
<p>With the access policies configured as discussed in the previous two examples, User1 is able to connect GenerateFlowFile to LogAttribute:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user1-create-connection.png" alt="User1 Create Connection"></span></p>
</div>
<div class="paragraph">
<p>User2 cannot make the connection:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user2-no-connection.png" alt="User2 No Connection"></span></p>
</div>
<div class="paragraph">
<p>This is because:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>User2 does not have modify access on the process group.</p>
</li>
<li>
<p>Even though User2 has view and modify access to the source component (GenerateFlowFile), User2 does not have an access policy on the destination component (LogAttribute).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>To allow User2 to connect GenerateFlowFile to LogAttribute, as User1:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Select the root process group. The Operate palette is updated with details for the root process group.</p>
</li>
<li>
<p>Select the Access Policies icon (<span class="image"><img src="images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette and the Access Policies dialog opens.</p>
</li>
<li>
<p>Select "modify the component” from the policy drop-down.
<span class="image"><img src="images/process-group-modify-policy.png" alt="Process Group Modify Policy"></span></p>
</li>
<li>
<p>Select the Add User icon (<span class="image"><img src="images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User2 and select OK.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/process-group-modify-policy-add-user2.png" alt="Process Group Modify Policy Add User2"></span></p>
</div>
<div class="paragraph">
<p>By adding User2 to the “modify the component” policy on the process group, User2 is added to the “modify the component” policy on the LogAttribute processor by policy inheritance. To confirm this, highlight the LogAttribute processor and select the Access Policies icon (<span class="image"><img src="images/iconAccessPolicies.png" alt="Access Policies Icon"></span>) from the Operate palette:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/processor-inherited-modify-policy.png" alt="User2 Inherited Edit Processor"></span></p>
</div>
<div class="paragraph">
<p>With these changes, User2 can now connect the GenerateFlowFile processor to the LogAttribute processor.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user2-can-connect.png" alt="User2 Can Connect"></span></p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user2-connected-processors.png" alt="User2 Connected Processors"></span></p>
</div>
</div>
<div class="sect4">
<h5 id="editing-a-connection"><a class="anchor" href="administration-guide.html#editing-a-connection"></a>Editing a Connection</h5>
<div class="paragraph">
<p>Assume User1 or User2 adds a ReplaceText processor to the root process group:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/replacetext-processor-added.png" alt="ReplaceText Processor Added"></span></p>
</div>
<div class="paragraph">
<p>User1 can select and change the existing connection (between GenerateFlowFile to LogAttribute) to now connect GenerateFlowFile to ReplaceText:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user1-edit-connection.png" alt="User1 Edit Connection"></span></p>
</div>
<div class="paragraph">
<p>User 2 is unable to perform this action.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user2-no-edit-connection.png" alt="User2 No Edit Connection"></span></p>
</div>
<div class="paragraph">
<p>To allow User2 to connect GenerateFlowFile to ReplaceText, as User1:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Select the root process group. The Operate palette is updated with details for the root process group.</p>
</li>
<li>
<p>Select the Access Policies icon (<span class="image"><img src="images/iconAccessPolicies.png" alt="Access Policies Icon"></span>).</p>
</li>
<li>
<p>Select "view the component” from the policy drop-down.
<span class="image"><img src="images/process-group-view-policy.png" alt="Process Group View Policy"></span></p>
</li>
<li>
<p>Select the Add User icon (<span class="image"><img src="images/iconAddUser.png" alt="Add User Icon"></span>). Find or enter User2 and select OK.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/process-group-view-policy-add-user2.png" alt="Process Group View Policy Add User2"></span></p>
</div>
<div class="paragraph">
<p>Being added to both the view and modify policies for the process group, User2 can now connect the GenerateFlowFile processor to the ReplaceText processor.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/user2-edit-connection.png" alt="User2 Edit Connection"></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="repository-encryption"><a class="anchor" href="administration-guide.html#repository-encryption"></a>Repository Encryption</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi supports encryption of local repositories using a configurable Key Provider to enable protection of information
on the filesystem. Repository encryption configuration uses a version number to indicate the cipher algorithms, metadata
format, and repository implementation classes. This approach provides a generalized method for configuration without the
need to customize each repository implementation class.</p>
</div>
<div class="paragraph">
<p>Repository encryption incurs a performance cost due to the overhead of cipher operations. Filesystem encryption at the
operating system level provides an alternative solution, with different performance characteristics. For deployments
where filesystem encryption is not configured, repository encryption provides an enhanced level of data protection.
Due to increased performance requirements, more computing resources may be necessary to achieve sufficient throughput
when enabling repository encryption.</p>
</div>
<div class="paragraph">
<p>The security of repository encryption depends on a combination of the cipher algorithms and the protection of encryption
keys. Key protection and key rotation are important parts of securing an encrypted repository configuration.
Key protection involves limiting access to the Key Provider and key rotation requires manual updates to generate and
specify a new encryption key.</p>
</div>
<div class="sect2">
<h3 id="configuration-change-considerations"><a class="anchor" href="administration-guide.html#configuration-change-considerations"></a>Configuration Change Considerations</h3>
<div class="paragraph">
<p>Disabling repository encryption on existing installations requires removing existing repository contents, and
restarting the system after making configuration changes. For this reason, flow administrators should confirm that the
system has processed all available FlowFiles to avoid losing information when disabling repository encryption.</p>
</div>
</div>
<div class="sect2">
<h3 id="repository-encryption-protocol-version-1"><a class="anchor" href="administration-guide.html#repository-encryption-protocol-version-1"></a>Repository Encryption Protocol Version 1</h3>
<div class="paragraph">
<p>The first version of support for repository encryption includes the following cipher algorithms:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Repository Type</th>
<th class="tableblock halign-left valign-top">Cipher Algorithm</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Content Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">AES/CTR/NoPadding</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">FlowFile Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">AES/GCM/NoPadding</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">FlowFile Swap Manager</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">AES/GCM/NoPadding</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Provenance Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">AES/GCM/NoPadding</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>The following classes provide the direct repository encryption implementation, extending standard classes:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Repository Type</th>
<th class="tableblock halign-left valign-top">Class</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Content Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">org.apache.nifi.content.EncryptedFileSystemRepository</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">FlowFile Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">org.apache.nifi.wali.EncryptedSequentialAccessWriteAheadLog</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">FlowFile Swap Manager</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">org.apache.nifi.controller.EncryptedFileSystemSwapManager</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Provenance Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository</p></td>
</tr>
</tbody>
</table>
<div class="sect3">
<h4 id="encryption-metadata-serialization"><a class="anchor" href="administration-guide.html#encryption-metadata-serialization"></a>Encryption Metadata Serialization</h4>
<div class="paragraph">
<p>Each repository implementation class leverages standard cipher operations to perform encryption and decryption. In order
to support AES, the encryption process writes metadata associated with each encryption operation. Encryption protocol
version 1 uses Java Object serialization to write objects containing the encryption Key Identifier, the cipher
Initialization Vector, and other required properties. Serialized objects include the following required properties:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Property Type</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">keyId</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">String</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Encryption key identifier</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">ivBytes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">byte[]</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Cipher initialization vector</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">algorithm</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">String</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Cipher algorithm</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">version</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">String</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Encryption protocol version</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">cipherByteLength</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">int</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Length of enciphered record</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Metadata serialization uses the standard <code>java.io.ObjectOutputStream.writeObject()</code> method to write objects to a stream
that can be converted to a byte array. The deserialization process uses a custom extension of the
<code>java.io.ObjectInputStream</code> to read objects regardless of the original class name associated with the record. This
approach requires the presence of the standard metadata properties, but provides a compatibility layer that avoids
linking the implementation to a specific Java class.</p>
</div>
<div class="paragraph">
<p>The initial implementation of encrypted repositories used different byte array markers when writing metadata. Each
repository implementation uses the following byte array markers before writing a serialized metadata record:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Repository Type</th>
<th class="tableblock halign-left valign-top">Byte Array</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Content Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">byte[]{0x00, 0x00}</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">FlowFile Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">byte[]{}</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Provenance Repository</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">byte[]{0x01}</p></td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="sect2">
<h3 id="repository-encryption-configuration"><a class="anchor" href="administration-guide.html#repository-encryption-configuration"></a>Repository Encryption Configuration</h3>
<div class="paragraph">
<p>Configuring repository encryption requires specifying the encryption protocol version and the associated Key Provider
properties. Repository encryption can be configured on new or existing installations using standard properties. Records
in existing repositories should be readable using standard capabilities, and the encrypted repository will write new
records using the specified configuration.</p>
</div>
<div class="sect3">
<h4 id="protocol-version-configuration"><a class="anchor" href="administration-guide.html#protocol-version-configuration"></a>Protocol Version Configuration</h4>
<div class="paragraph">
<p>Setting the following protocol version property enables encryption for all repositories:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.repository.encryption.protocol.version=1</pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="key-provider-configuration"><a class="anchor" href="administration-guide.html#key-provider-configuration"></a>Key Provider Configuration</h4>
<div class="paragraph">
<p>All encrypted repositories require a Key Provider to perform encryption and decryption operations. NiFi supports
configuring the Key Provider implementation as well as the Key Identifier that will be used for new encryption
operations. Key Provider implementations can hold multiple keys to support using a new key while maintaining access to
information encrypted using the previous key.</p>
</div>
<div class="paragraph">
<p>Repository encryption supports access to secret keys using standard <code>java.security.KeyStore</code> files.
See <a href="administration-guide.html#secret-key-generation-and-storage-using-keytool">Secret Key Generation and Storage using Keytool</a> for details on supported KeyStore types, as well as examples of
generating secret keys.</p>
</div>
<div class="paragraph">
<p>The following configuration properties provide an example using a PKCS12 KeyStore file named <code>repository.p12</code> containing
a secret key labeled with an alias of <code>primary-key</code>:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.repository.encryption.key.id=primary-key
nifi.repository.encryption.key.provider=KEYSTORE
nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
nifi.repository.encryption.key.provider.keystore.password=2fRKmwDyMYmT7P5L</pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="secret-key-generation-and-storage-using-keytool"><a class="anchor" href="administration-guide.html#secret-key-generation-and-storage-using-keytool"></a>Secret Key Generation and Storage using Keytool</h4>
<div class="paragraph">
<p>The <code>KeyStoreKeyProvider</code> supports reading from a <code>java.security.KeyStore</code> using a configured password to load AES Secret Key entries.
The <code>KeyStoreKeyProvider</code> can be configured with any of the encrypted repository implementations.</p>
</div>
<div class="paragraph">
<p>The provider supports the following KeyStore Types:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>BCFKS</p>
</li>
<li>
<p>PKCS12</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The keystore filename extension must be either <code>.p12</code> indicating PKCS12 or <code>.bcfks</code> indicating BCFKS.</p>
</div>
<div class="paragraph">
<p>The <code>keytool</code> command can be used to generate an AES-256 Secret Key stored in a PKCS12 file for repository encryption:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>keytool -genseckey -alias primary-key -keyalg AES -keysize 256 -keystore repository.p12 -storetype PKCS12</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>keytool</code> command requires additional arguments specifying the BouncyCastle Security Provider to store
Secret Keys using BCFKS. The arguments must include a reference to the BouncyCastle Security Provider library, which
is available in the <code>lib/bootstrap</code> directory under the NiFi installation.</p>
</div>
<div class="paragraph">
<p>The following command can be used to generate an AES-256 Secret Key stored using BCFKS:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>keytool -genseckey -alias primary-key -keyalg AES -keysize 256 -keystore repository.bcfks -storetype BCFKS -providerclass org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath lib/bootstrap/bcprov-jdk15on-*.jar</pre>
</div>
</div>
<div class="paragraph">
<p>Enter a keystore password when prompted. The same value must be used for both the keystore password and key password.
The keystore password will be used in the provider configuration properties.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="encryption"><a class="anchor" href="administration-guide.html#encryption"></a>Encryption Configuration</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The EncryptContent processor allows for the encryption and decryption of data, both internal to NiFi and integrated with external systems, such as <code>openssl</code> and other data sources and consumers.</p>
</div>
<div class="sect2">
<h3 id="key-derivation-functions"><a class="anchor" href="administration-guide.html#key-derivation-functions"></a>Key Derivation Functions</h3>
<div class="paragraph">
<p>Key Derivation Functions (KDF) are mechanisms by which human-readable information, usually a password or other secret information, is translated into a cryptographic key suitable for data protection. For further information, read the <a href="https://en.wikipedia.org/wiki/Key_derivation_function" target="_blank" rel="noopener">Wikipedia entry on Key Derivation Functions</a>.
Currently, KDFs are ingested by <code>CipherProvider</code> implementations and return a fully-initialized <code>Cipher</code> object to be used for encryption or decryption. Due to the use of a <code>CipherProviderFactory</code>, the KDFs are not customizable at this time. Future enhancements will include the ability to provide custom cost parameters to the KDF at initialization time. As a work-around, <code>CipherProvider</code> instances can be initialized with custom cost parameters in the constructor but this is not currently supported by the <code>CipherProviderFactory</code>.
If you do not have a need for a specific KDF, <strong>Argon2 is recommended</strong> as it is a robust, secure, performant, and user-friendly default and is widely supported on multiple platforms.
Here are the KDFs currently supported by NiFi (primarily in the <code>EncryptContent</code> processor for password-based encryption (PBE)) and relevant notes:</p>
</div>
<div class="sect3">
<h4 id="nifi-legacy-kdf"><a class="anchor" href="administration-guide.html#nifi-legacy-kdf"></a>NiFi Legacy KDF</h4>
<div class="ulist">
<ul>
<li>
<p>The original KDF used by NiFi for internal key derivation for PBE, this is 1000 iterations of the MD5 digest over the concatenation of the password and 8 or 16 bytes of random salt (the salt length depends on the selected cipher block size).</p>
</li>
<li>
<p>This KDF is <strong>deprecated as of NiFi 0.5.0</strong> and should only be used for backwards compatibility to decrypt data that was previously encrypted by a legacy version of NiFi.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="openssl-pkcs5-evp-bytestokey"><a class="anchor" href="administration-guide.html#openssl-pkcs5-evp-bytestokey"></a>OpenSSL PKCS#5 v1.5 EVP_BytesToKey</h4>
<div class="ulist">
<ul>
<li>
<p>This KDF was added in v0.4.0.</p>
</li>
<li>
<p>This KDF is provided for compatibility with data encrypted using OpenSSL&#8217;s default PBE, known as <code>EVP_BytesToKey</code>. This is a single iteration of MD5 over the concatenation of the password and 8 bytes of random ASCII salt. OpenSSL recommends using <code>PBKDF2</code> for key derivation but does not expose the library method necessary to the command-line tool, so this KDF is still the de facto default for command-line encryption.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="bcrypt-kdf"><a class="anchor" href="administration-guide.html#bcrypt-kdf"></a>Bcrypt</h4>
<div class="ulist">
<ul>
<li>
<p>This KDF was added in v0.5.0.</p>
</li>
<li>
<p><a href="https://en.wikipedia.org/wiki/Bcrypt" target="_blank" rel="noopener">Bcrypt</a> is an adaptive function based on the <a href="https://en.wikipedia.org/wiki/Blowfish_(cipher)" target="_blank" rel="noopener">Blowfish</a> cipher. This KDF is recommended as it automatically incorporates a random 16 byte salt, configurable cost parameter (or "work factor"), and is hardened against brute-force attacks using <a href="https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units" target="_blank" rel="noopener">GPGPU</a> (which share memory between cores) by requiring access to "large" blocks of memory during the key derivation. It is less resistant to <a href="https://en.wikipedia.org/wiki/Field-programmable_gate_array" target="_blank" rel="noopener">FPGA</a> brute-force attacks where the gate arrays have access to individual embedded RAM blocks.</p>
</li>
<li>
<p>Because the length of a Bcrypt-derived hash is always 184 bits, the hash output (not including the algorithm, work factor, or salt) is then fed to a <code>SHA-512</code> digest and truncated to the desired key length. This provides the benefit of the avalanche effect over the input. This key stretching mechanism was introduced in Apache NiFi 1.12.0.</p>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Prior to this, the <em>complete output</em> (algorithm, work factor, salt, and hash output for a total of 480 bits) was provided to the SHA-512 digest function. NiFi can transparently handle decrypting data (under 10 MiB) encrypted using a key derived via this legacy process.
</td>
</tr>
</table>
</div>
</li>
<li>
<p>The recommended minimum work factor is 12 (2<sup>12</sup> key derivation rounds) (as of 2/1/2016 on commodity hardware) and should be increased to the threshold at which legitimate systems will encounter detrimental delays (see schedule below or use <code>BcryptCipherProviderGroovyTest#testDefaultConstructorShouldProvideStrongWorkFactor()</code> to calculate safe minimums).</p>
</li>
<li>
<p>The salt format is <code>$2a$10$ABCDEFGHIJKLMNOPQRSTUV</code>. The salt is delimited by <code>$</code> and the three sections are as follows:</p>
<div class="ulist">
<ul>
<li>
<p><code>2a</code> - the version of the format. An extensive explanation can be found <a href="http://blog.ircmaxell.com/2012/12/seven-ways-to-screw-up-bcrypt.html" target="_blank" rel="noopener">here</a>. NiFi currently uses <code>2a</code> for all salts generated internally.</p>
</li>
<li>
<p><code>10</code> - the work factor. This is actually the log<sub>2</sub> value, so the total iteration count would be 2<sup>10</sup> (1024) in this case.</p>
</li>
<li>
<p><code>ABCDEFGHIJKLMNOPQRSTUV</code> - the 22 character, Radix64-encoded, unpadded, raw salt value. This decodes to a 16 byte salt used in the key derivation.</p>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
The Bcrypt Radix64 encoding is <strong>not</strong> compatible with standard MIME Base64 encoding.
</td>
</tr>
</table>
</div>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="scrypt-kdf"><a class="anchor" href="administration-guide.html#scrypt-kdf"></a>Scrypt</h4>
<div class="ulist">
<ul>
<li>
<p>This KDF was added in v0.5.0.</p>
</li>
<li>
<p><a href="https://en.wikipedia.org/wiki/Scrypt" target="_blank" rel="noopener">Scrypt</a> is an adaptive function designed in response to <code>bcrypt</code>. This KDF is recommended as it requires relatively large amounts of memory for each derivation, making it resistant to hardware brute-force attacks.</p>
</li>
<li>
<p>The recommended minimum cost is <code>N</code>=2<sup>14</sup> (16,384), <code>r</code>=8, <code>p</code>=1 (as of 2/1/2016 on commodity hardware). <code>p</code> must be a positive integer and less than <code>(2^32 − 1) * (Hlen/MFlen)</code> where <code>Hlen</code> is the length in octets of the digest function output (32 for SHA-256) and <code>MFlen</code> is the length in octets of the mixing function output, defined as <code>r * 128</code>. These parameters should be increased to the threshold at which legitimate systems will encounter detrimental delays (see schedule below or use <code>ScryptCipherProviderGroovyTest#testDefaultConstructorShouldProvideStrongParameters()</code> to calculate safe minimums).</p>
</li>
<li>
<p>The salt format is <code>$s0$e0101$ABCDEFGHIJKLMNOPQRSTUV</code>. The salt is delimited by <code>$</code> and the three sections are as follows:</p>
<div class="ulist">
<ul>
<li>
<p><code>s0</code> - the version of the format. NiFi currently uses <code>s0</code> for all salts generated internally.</p>
</li>
<li>
<p><code>e0101</code> - the cost parameters. This is actually a hexadecimal encoding of <code>N</code>, <code>r</code>, <code>p</code> using shifts. This can be formed/parsed using <code>Scrypt#encodeParams()</code> and <code>Scrypt#parseParameters()</code>.</p>
<div class="ulist">
<ul>
<li>
<p>Some external libraries encode <code>N</code>, <code>r</code>, and <code>p</code> separately in the form <code>$4000$1$1$</code> (<code>N</code> is stored in hex encoding as <code>0x4000</code>, which is <code>0d16384</code>, or 2<sup>14</sup> as <code>0xe</code> = <code>0d14</code>). A utility method is available at <code>ScryptCipherProvider#translateSalt()</code> which will convert the external form to the internal form.</p>
</li>
</ul>
</div>
</li>
<li>
<p><code>ABCDEFGHIJKLMNOPQRSTUV</code> - the 12-44 character, Base64-encoded, unpadded, raw salt value. This decodes to a 8-32 byte salt used in the key derivation.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="pbkdf2-kdf"><a class="anchor" href="administration-guide.html#pbkdf2-kdf"></a>PBKDF2</h4>
<div class="ulist">
<ul>
<li>
<p>This KDF was added in v0.5.0.</p>
</li>
<li>
<p><a href="https://en.wikipedia.org/wiki/PBKDF2" target="_blank" rel="noopener">Password-Based Key Derivation Function 2</a> is an adaptive derivation function which uses an internal pseudorandom function (PRF) and iterates it many times over a password and salt (at least 16 bytes).</p>
</li>
<li>
<p>The PRF is recommended to be <code>HMAC/SHA-256</code> or <code>HMAC/SHA-512</code>. The use of an HMAC cryptographic hash function mitigates a length extension attack.</p>
</li>
<li>
<p>The recommended minimum number of iterations is 160,000 (as of 2/1/2016 on commodity hardware). This number should be doubled every two years (see schedule below or use <code>PBKDF2CipherProviderGroovyTest#testDefaultConstructorShouldProvideStrongIterationCount()</code> to calculate safe minimums).</p>
</li>
<li>
<p>This KDF is not memory-hard (can be parallelized massively with commodity hardware) but is still recommended as sufficient by <a href="http://csrc.nist.gov/publications/nistpubs/800-132/nist-sp800-132.pdf" target="_blank" rel="noopener">NIST SP 800-132 (PDF)</a> and many cryptographers (when used with a proper iteration count and HMAC cryptographic hash function).</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="none-kdf"><a class="anchor" href="administration-guide.html#none-kdf"></a>None</h4>
<div class="ulist">
<ul>
<li>
<p>This KDF was added in v0.5.0.</p>
</li>
<li>
<p>This KDF performs no operation on the input and is a marker to indicate the raw key is provided to the cipher. The key must be provided in hexadecimal encoding and be of a valid length for the associated cipher/algorithm.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="argon2-kdf"><a class="anchor" href="administration-guide.html#argon2-kdf"></a>Argon2</h4>
<div class="ulist">
<ul>
<li>
<p>This KDF was added in v1.12.0.</p>
</li>
<li>
<p><a href="https://en.wikipedia.org/wiki/Argon2" target="_blank" rel="noopener">Argon2</a> is a key derivation function which won the Password Hashing Competition in 2015. This KDF is recommended as it offers a variety of modes which can be tailored to prevention of GPU attacks, prevention of side-channel attacks, or a combination of both. It allows for a variable output key length.</p>
</li>
<li>
<p>The recommended minimum cost is <code>memory</code>=2<sup>16</sup> (65,536) KiB, <code>iterations</code>=5, <code>parallelism</code>=8 (as of 4/22/2020 on commodity hardware). The <a href="https://password-hashing.net/argon2-specs.pdf" target="_blank" rel="noopener">Argon2 specification paper (PDF)</a> Section 9 describes an algorithm used to determine recommended parameters. These parameters should be increased to the threshold at which legitimate systems will encounter detrimental delays (use <code>Argon2SecureHasherTest#testDefaultCostParamsShouldBeSufficient()</code> to calculate safe minimums).</p>
</li>
<li>
<p>The salt format is <code>$argon2id$v=19$m=65536,t=5,p=8$ABCDEFGHIJKLMNOPQRSTUV</code>. The salt is delimited by <code>$</code> and the four sections are as follows:</p>
<div class="ulist">
<ul>
<li>
<p><code>argon2id</code> - the "type" of algorithm (<code>2i</code>, <code>2d</code>, <code>2id</code>). NiFi currently uses <code>argon2id</code> for all salts generated internally.</p>
</li>
<li>
<p><code>v=19</code> - the version of the algorithm in decimal (<code>0d19</code> = <code>0x13</code>). NiFi currently uses <code>0d19</code> for all salts generated internally.</p>
</li>
<li>
<p><code>m=65536,t=5,p=8</code> - the cost parameters. This contains the memory, iterations, and parallelism in order.</p>
</li>
<li>
<p><code>ABCDEFGHIJKLMNOPQRSTUV</code> - the 12-44 character, Base64-encoded, unpadded, raw salt value. This decodes to a 8-32 byte salt used in the key derivation.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="sect4">
<h5 id="additional-resources"><a class="anchor" href="administration-guide.html#additional-resources"></a>Additional Resources</h5>
<div class="ulist">
<ul>
<li>
<p><a href="http://stackoverflow.com/a/30308723/70465" target="_blank" rel="noopener">Explanation of optimal scrypt cost parameters and relationships</a></p>
</li>
<li>
<p><a href="http://csrc.nist.gov/publications/nistpubs/800-132/nist-sp800-132.pdf" target="_blank" rel="noopener">NIST Special Publication 800-132</a></p>
</li>
<li>
<p><a href="https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet#Work_Factor" target="_blank" rel="noopener">OWASP Password Storage Work Factor Calculations</a></p>
</li>
<li>
<p><a href="http://security.stackexchange.com/a/3993/16485" target="_blank" rel="noopener">PBKDF2 rounds calculations</a></p>
</li>
<li>
<p><a href="http://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scrypt.html" target="_blank" rel="noopener">Scrypt as KDF vs password storage vulnerabilities</a></p>
</li>
<li>
<p><a href="http://security.stackexchange.com/a/26253/16485" target="_blank" rel="noopener">Scrypt vs. Bcrypt (as of 2010)</a></p>
</li>
<li>
<p><a href="http://security.stackexchange.com/a/6415/16485" target="_blank" rel="noopener">Bcrypt vs PBKDF2</a></p>
</li>
<li>
<p><a href="http://wildlyinaccurate.com/bcrypt-choosing-a-work-factor/" target="_blank" rel="noopener">Choosing a work factor for Bcrypt</a></p>
</li>
<li>
<p><a href="https://docs.spring.io/spring-security/site/docs/current/api/org/springframework/security/crypto/bcrypt/BCrypt.html" target="_blank" rel="noopener">Spring Security Bcrypt</a></p>
</li>
<li>
<p><a href="https://www.openssl.org/docs/man3.0/man3/EVP_BytesToKey.html" target="_blank" rel="noopener">OpenSSL EVP BytesToKey</a></p>
</li>
<li>
<p><a href="https://en.wikipedia.org/wiki/PBKDF2" target="_blank" rel="noopener">Wikipedia PBKDF2</a></p>
</li>
<li>
<p><a href="http://security.stackexchange.com/a/29139/16485" target="_blank" rel="noopener">OpenSSL KDF flaws description</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="salt-and-iv-encoding"><a class="anchor" href="administration-guide.html#salt-and-iv-encoding"></a>Salt and IV Encoding</h3>
<div class="paragraph">
<p>Initially, the <code>EncryptContent</code> processor had a single method of deriving the encryption key from a user-provided password. This is now referred to as <code>NiFiLegacy</code> mode, effectively <code>MD5 digest, 1000 iterations</code>. In v0.4.0, another method of deriving the key, <code>OpenSSL PKCS#5 v1.5 EVP_BytesToKey</code> was added for compatibility with content encrypted outside of NiFi using the <code>openssl</code> command-line tool. Both of these <a href="administration-guide.html#key-derivation-functions">Key Derivation Functions</a> (KDF) had hard-coded digest functions and iteration counts, and the salt format was also hard-coded. With v0.5.0, additional KDFs are introduced with variable iteration counts, work factors, and salt formats. In addition, <em>raw keyed encryption</em> was also introduced. This required the capacity to encode arbitrary salts and Initialization Vectors (IV) into the cipher stream in order to be recovered by NiFi or a follow-on system to decrypt these messages.</p>
</div>
<div class="paragraph">
<p>For the existing KDFs, the salt format has not changed.</p>
</div>
<div class="sect3">
<h4 id="nifi-legacy"><a class="anchor" href="administration-guide.html#nifi-legacy"></a>NiFi Legacy</h4>
<div class="paragraph">
<p>The first 8 or 16 bytes of the input are the salt. The salt length is determined based on the selected algorithm&#8217;s cipher block length. If the cipher block size cannot be determined (such as with a stream cipher like <code>RC4</code>), the default value of 8 bytes is used. On decryption, the salt is read in and combined with the password to derive the encryption key and IV.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/nifi-legacy-salt.png" alt="NiFi Legacy Salt Encoding"></span></p>
</div>
</div>
<div class="sect3">
<h4 id="openssl-pkcs5-v1-5-evp_bytestokey"><a class="anchor" href="administration-guide.html#openssl-pkcs5-v1-5-evp_bytestokey"></a>OpenSSL PKCS#5 v1.5 EVP_BytesToKey</h4>
<div class="paragraph">
<p>OpenSSL allows for salted or unsalted key derivation. <em>*Unsalted key derivation is a security risk and is not recommended.*</em> If a salt is present, the first 8 bytes of the input are the ASCII string &#8220;Salted__&#8221; (<code>0x53 61 6C 74 65 64 5F 5F</code>) and the next 8 bytes are the ASCII-encoded salt. On decryption, the salt is read in and combined with the password to derive the encryption key and IV. If there is no salt header, the entire input is considered to be the cipher text.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/openssl-salt.png" alt="OpenSSL Salt Encoding"></span></p>
</div>
<div class="paragraph">
<p>For new KDFs, each of which allow for non-deterministic IVs, the IV must be stored alongside the cipher text. This is not a vulnerability, as the IV is not required to be secret, but simply to be unique for messages encrypted using the same key to reduce the success of cryptographic attacks. For these KDFs, the output consists of the salt, followed by the salt delimiter, UTF-8 string &#8220;NiFiSALT&#8221; (<code>0x4E 69 46 69 53 41 4C 54</code>) and then the IV, followed by the IV delimiter, UTF-8 string &#8220;NiFiIV&#8221; (<code>0x4E 69 46 69 49 56</code>), followed by the cipher text.</p>
</div>
</div>
<div class="sect3">
<h4 id="bcrypt-scrypt-pbkdf2-argon2"><a class="anchor" href="administration-guide.html#bcrypt-scrypt-pbkdf2-argon2"></a>Bcrypt, Scrypt, PBKDF2, Argon2</h4>
<div class="paragraph">
<p><span class="image"><img src="images/bcrypt-salt.png" alt="Bcrypt Salt &amp; IV Encoding"></span></p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/scrypt-salt.png" alt="Scrypt Salt &amp; IV Encoding"></span></p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/pbkdf2-salt.png" alt="PBKDF2 Salt &amp; IV Encoding"></span></p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/argon2-salt.png" alt="Argon2 Salt &amp; IV Encoding"></span></p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="java-cryptography-extension-jce-limited-strength-jurisdiction-policies"><a class="anchor" href="administration-guide.html#java-cryptography-extension-jce-limited-strength-jurisdiction-policies"></a>Java Cryptography Extension (JCE) Limited Strength Jurisdiction Policies</h3>
<div class="paragraph">
<p>Because of US export regulations, default JVMs have <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#importlimits" target="_blank" rel="noopener">limits imposed on the strength of cryptographic operations</a> available to them. For example, AES operations are limited to <code>128 bit keys</code> by default. While <code>AES-128</code> is cryptographically safe, this can have unintended consequences, specifically on Password-based Encryption (PBE).</p>
</div>
<div class="paragraph">
<p>PBE is the process of deriving a cryptographic key for encryption or decryption from <em>user-provided secret material</em>, usually a password. Rather than a human remembering a (random-appearing) 32 or 64 character hexadecimal string, a password or passphrase is used.</p>
</div>
<div class="paragraph">
<p>A number of PBE algorithms provided by NiFi impose strict limits on the length of the password due to the underlying key length checks. Below is a table listing the maximum password length on a JVM with limited cryptographic strength.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<caption class="title">Table 1. Maximum Password Length on Limited Cryptographic Strength JVM</caption>
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Algorithm</th>
<th class="tableblock halign-left valign-top">Max Password Length</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHMD5AND128BITAES-CBC-OPENSSL</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">16</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHMD5AND192BITAES-CBC-OPENSSL</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">16</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHMD5AND256BITAES-CBC-OPENSSL</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">16</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHMD5ANDDES</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">16</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHMD5ANDRC2</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">16</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHA1ANDRC2</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">16</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHA1ANDDES</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">16</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND128BITAES-CBC-BC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND192BITAES-CBC-BC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND256BITAES-CBC-BC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND40BITRC2-CBC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND128BITRC2-CBC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND40BITRC4</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND128BITRC4</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHA256AND128BITAES-CBC-BC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHA256AND192BITAES-CBC-BC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHA256AND256BITAES-CBC-BC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND2-KEYTRIPLEDES-CBC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAAND3-KEYTRIPLEDES-CBC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PBEWITHSHAANDTWOFISH-CBC</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="allow-insecure-cryptographic-modes"><a class="anchor" href="administration-guide.html#allow-insecure-cryptographic-modes"></a>Allow Insecure Cryptographic Modes</h3>
<div class="paragraph">
<p>By default, the <code>Allow Insecure Cryptographic Modes</code> property in <code>EncryptContent</code> processor settings is set to <code>not-allowed</code>. This means that if a password of fewer than <code>10</code> characters is provided, a validation error will occur. 10 characters is a conservative estimate and does not take into consideration full entropy calculations, patterns, etc.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/allow-weak-crypto.png" alt="Allow Insecure Cryptographic Modes" width="940"></span></p>
</div>
<div class="paragraph">
<p>On a JVM with limited strength cryptography, some PBE algorithms limit the maximum password length to 7, and in this case it will not be possible to provide a "safe" password. It is recommended to install the JCE Unlimited Strength Jurisdiction Policy files for the JVM to mitigate this issue.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html" target="_blank" rel="noopener">JCE Unlimited Strength Jurisdiction Policy files for Java 8</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>If on a system where the unlimited strength policies cannot be installed, it is recommended to switch to an algorithm that supports longer passwords (see table above).</p>
</div>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
<div class="title">Allowing Weak Crypto</div>
<div class="paragraph">
<p>If it is not possible to install the unlimited strength jurisdiction policies, the <code>Allow Weak Crypto</code> setting can be changed to <code>allowed</code>, but <strong>this is <em>not</em> recommended</strong>. Changing this setting explicitly acknowledges the inherent risk in using weak cryptographic configurations.</p>
</div>
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>It is preferable to request upstream/downstream systems to switch to <a href="https://cwiki.apache.org/confluence/display/NIFI/Encryption+Information" target="_blank" rel="noopener">keyed encryption</a> or use a "strong" <a href="administration-guide.html#key-derivation-functions">Key Derivation Function (KDF) supported by NiFi</a>.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="nifi_sensitive_props_key"><a class="anchor" href="administration-guide.html#nifi_sensitive_props_key"></a>Encrypted Passwords in Flows</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi always stores all sensitive values (passwords, tokens, and other credentials) populated into a flow in an encrypted format on disk.
The encryption algorithm used is specified by <code>nifi.sensitive.props.algorithm</code> and the password from which the encryption key is derived is specified by <code>nifi.sensitive.props.key</code> in <em>nifi.properties</em> (see <a href="administration-guide.html#security_configuration">Security Configuration</a> for additional information).
Prior to version 1.12.0, the list of available algorithms was all password-based encryption (PBE) algorithms supported by the <code>EncryptionMethod</code> enum in that version.
Unfortunately many of these algorithms are provided for legacy compatibility, and use weak key derivation functions and block cipher algorithms &amp; modes of operation.
In 1.12.0, a pair of custom algorithms was introduced for security-conscious users looking for more robust protection of the flow sensitive values.</p>
</div>
<div class="paragraph">
<p>NiFi supports several configuration options to provide authenticated encryption with associated data (AEAD) using AES Galois/Counter Mode (AES-GCM).
These algorithms use a strong Key Derivation Function to derive a secret key of specified length based on the sensitive properties key configured.
Each Key Derivation Function uses a static salt in order to support flow configuration comparison across cluster nodes.
Each Key Derivation Function also uses default iteration and cost parameters as defined in the associated secure hashing implementation class.</p>
</div>
<div class="sect2">
<h3 id="property-encryption-algorithms"><a class="anchor" href="administration-guide.html#property-encryption-algorithms"></a>Property Encryption Algorithms</h3>
<div class="paragraph">
<p>The following strong encryption methods can be configured in the <code>nifi.sensitive.props.algorithm</code> property:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>NIFI_ARGON2_AES_GCM_128</code></p>
</li>
<li>
<p><code>NIFI_ARGON2_AES_GCM_256</code></p>
</li>
<li>
<p><code>NIFI_BCRYPT_AES_GCM_128</code></p>
</li>
<li>
<p><code>NIFI_BCRYPT_AES_GCM_256</code></p>
</li>
<li>
<p><code>NIFI_PBKDF2_AES_GCM_128</code></p>
</li>
<li>
<p><code>NIFI_PBKDF2_AES_GCM_256</code></p>
</li>
<li>
<p><code>NIFI_SCRYPT_AES_GCM_128</code></p>
</li>
<li>
<p><code>NIFI_SCRYPT_AES_GCM_256</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Each Key Derivation Function uses the following default parameters:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Argon2</p>
<div class="ulist">
<ul>
<li>
<p>Iterations: 5</p>
</li>
<li>
<p>Memory: 65536 KB</p>
</li>
<li>
<p>Parallelism: 8</p>
</li>
</ul>
</div>
</li>
<li>
<p>Bcrypt</p>
<div class="ulist">
<ul>
<li>
<p>Cost: 12</p>
</li>
<li>
<p>Derived Key Digest Algorithm: SHA-512</p>
</li>
</ul>
</div>
</li>
<li>
<p>PBKDF2</p>
<div class="ulist">
<ul>
<li>
<p>Iterations: 160,000</p>
</li>
<li>
<p>Pseudorandom Function Family: SHA-512</p>
</li>
</ul>
</div>
</li>
<li>
<p>Scrypt</p>
<div class="ulist">
<ul>
<li>
<p>Cost Factor (N): 16384</p>
</li>
<li>
<p>Block Size Factor (r): 8</p>
</li>
<li>
<p>Parallelization Factor (p): 1</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>All options require a password (<code>nifi.sensitive.props.key</code> value) of <strong>at least 12 characters</strong>.</p>
</div>
<div class="paragraph">
<p>In new standalone installations of 1.14.0 or later, NiFi generates a random value when <code>nifi.sensitive.props.key</code> is
empty. NiFi writes the generated value to <code>nifi.properties</code> and logs a warning.</p>
</div>
<div class="paragraph">
<p>Clustered installations of NiFi require the same value to be configured on all nodes.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="encrypt-config_tool"><a class="anchor" href="administration-guide.html#encrypt-config_tool"></a>Encrypted Passwords in Configuration Files</h2>
<div class="sectionbody">
<div class="paragraph">
<p>In order to facilitate the secure setup of NiFi, you can use the <code>encrypt-config</code> command line utility to encrypt raw configuration values that NiFi decrypts in memory on startup. This extensible protection scheme transparently allows NiFi to use raw values in operation, while protecting them at rest.</p>
</div>
<div class="paragraph">
<p>This is a change in behavior; prior to 1.0, all configuration values were stored in plaintext on the file system. POSIX file permissions were recommended to limit unauthorized access to these files.</p>
</div>
<div class="paragraph">
<p>If no administrator action is taken, the configuration values remain unencrypted.</p>
</div>
<div class="paragraph">
<p>For more information, see the <a href="toolkit-guide.html#encrypt_config_tool">Encrypt-Config Tool</a> section in the <a href="toolkit-guide.html">NiFi Toolkit Guide</a>.</p>
</div>
<div class="paragraph">
<p>Configuring each Sensitive Property Provider requires including the appropriate file reference property in <code>bootstrap.conf</code>. The default <code>bootstrap.conf</code> includes commented file reference properties for available providers.</p>
</div>
<div class="sect2">
<h3 id="hashicorp-vault-providers"><a class="anchor" href="administration-guide.html#hashicorp-vault-providers"></a>HashiCorp Vault providers</h3>
<div class="paragraph">
<p>Two encryption providers are currently configurable in the <code>bootstrap-hashicorp-vault.conf</code> file:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Provider</th>
<th class="tableblock halign-left valign-top">Provider Identifier</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">HashiCorp Vault Transit provider</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>hashicorp/vault/kv/{vault.transit.path}</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Uses HashiCorp Vault&#8217;s Transit Secrets Engine to decrypt sensitive properties.</p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">HashiCorp Vault Key/Value provider</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>hashicorp/vault/kv/{vault.kv.path}</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Retrieves sensitive values from Secrets stored in a HashiCorp Vault Key/Value (unversioned) Secrets Engine.</p></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>Note that all HashiCorp Vault encryption providers require a running Vault instance in order to decrypt these values at NiFi&#8217;s startup.</p>
</div>
<div class="paragraph">
<p>Following are the configuration properties available inside the <code>bootstrap-hashicorp-vault.conf</code> file:</p>
</div>
<div class="sect3">
<h4 id="required-properties"><a class="anchor" href="administration-guide.html#required-properties"></a>Required properties</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.uri</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The HashiCorp Vault URI (e.g., <code><a href="https://vault-server:8200" class="bare">https://vault-server:8200</a></code>). If not set, all HashiCorp Vault providers will be disabled.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.authentication.properties.file</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filename of a properties file containing Vault authentication properties. See the <code>Authentication-specific property keys</code> section of <a href="https://docs.spring.io/spring-vault/docs/2.3.x/reference/html/#vault.core.environment-vault-configuration" class="bare">https://docs.spring.io/spring-vault/docs/2.3.x/reference/html/#vault.core.environment-vault-configuration</a> for all authentication property keys. If not set, all Spring Vault authentication properties must be configured directly in bootstrap-hashicorp-vault.conf.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.transit.path</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set, enables the HashiCorp Vault Transit provider. The value should be the Vault <code>path</code> of a Transit Secrets Engine (e.g., <code>nifi-transit</code>). Valid characters include alphanumeric, dash, and underscore.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.kv.path</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set, enables the HashiCorp Vault Key/Value provider. The value should be the Vault <code>path</code> of a K/V (v1) Secrets Engine (e.g., <code>nifi-kv</code>). Valid characters include alphanumeric, dash, and underscore.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
<div class="sect3">
<h4 id="optional-properties"><a class="anchor" href="administration-guide.html#optional-properties"></a>Optional properties</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.kv.version</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Key/Value Secrets Engine version: <code>1</code> for unversioned, and <code>2</code> for versioned. This must match the versioned enabled in Vault.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>1</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.connection.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The connection timeout of the Vault client</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>5 secs</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.read.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The read timeout of the Vault client</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>15 secs</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.enabledCipherSuites</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma-separated list of the enabled TLS cipher suites</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.enabledProtocols</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma-separated list of the enabled TLS protocols</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.key-store</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Path to a keystore. Required if the Vault server is TLS-enabled</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.key-store-type</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Keystore type (JKS, BCFKS or PKCS12). Required if the Vault server is TLS-enabled</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.key-store-password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Keystore password. Required if the Vault server is TLS-enabled</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.trust-store</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Path to a truststore. Required if the Vault server is TLS-enabled</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.trust-store-type</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Truststore type (JKS, BCFKS or PKCS12). Required if the Vault server is TLS-enabled</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>vault.ssl.trust-store-password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Truststore password. Required if the Vault server is TLS-enabled</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
</div>
<div class="sect2">
<h3 id="aws-kms-provider"><a class="anchor" href="administration-guide.html#aws-kms-provider"></a>AWS KMS provider</h3>
<div class="paragraph">
<p>This provider uses <a href="https://aws.amazon.com/kms/">AWS Key Management Service</a> for decryption. AWS KMS configuration properties can be stored in the <code>bootstrap-aws.conf</code> file, as referenced in <code>bootstrap.conf</code>. If the configuration properties are not specified in <code>bootstrap-aws.conf</code>, then the provider will attempt to use the AWS default credentials provider, which checks standard environment variables and system properties.</p>
</div>
<div class="sect3">
<h4 id="required-properties-2"><a class="anchor" href="administration-guide.html#required-properties-2"></a>Required properties</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>aws.kms.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identifier or ARN that the AWS KMS client uses for encryption and decryption.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
<div class="sect3">
<h4 id="optional-properties-2"><a class="anchor" href="administration-guide.html#optional-properties-2"></a>Optional properties</h4>
<div class="sect4">
<h5 id="all-of-the-following-must-be-configured-or-will-be-ignored-entirely"><a class="anchor" href="administration-guide.html#all-of-the-following-must-be-configured-or-will-be-ignored-entirely"></a>All of the following must be configured, or will be ignored entirely.</h5>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>aws.region</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The AWS region used to configure the AWS KMS Client.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>aws.access.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The access key ID credential used to access AWS KMS.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>aws.secret.access.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The secret access key used to access AWS KMS.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
</div>
</div>
<div class="sect2">
<h3 id="aws-secrets-manager-provider"><a class="anchor" href="administration-guide.html#aws-secrets-manager-provider"></a>AWS Secrets Manager provider</h3>
<div class="paragraph">
<p>This provider uses <a href="https://aws.amazon.com/secrets-manager/">AWS Secrets Manager Service</a> to store and retrieve AWS Secrets. AWS Secrets Manager configuration properties can be stored in the <code>bootstrap-aws.conf</code> file, as referenced in <code>bootstrap.conf</code>. If the configuration properties are not specified in <code>bootstrap-aws.conf</code>, then the provider will attempt to use the AWS default credentials provider, which checks standard environment variables and system properties.</p>
</div>
<div class="sect3">
<h4 id="optional-properties-3"><a class="anchor" href="administration-guide.html#optional-properties-3"></a>Optional properties</h4>
<div class="sect4">
<h5 id="all-of-the-following-must-be-configured-or-will-be-ignored-entirely-2"><a class="anchor" href="administration-guide.html#all-of-the-following-must-be-configured-or-will-be-ignored-entirely-2"></a>All of the following must be configured, or will be ignored entirely.</h5>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>aws.region</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The AWS region used to configure the AWS Secrets Manager Client.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>aws.access.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The access key ID credential used to access AWS Secrets Manager.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>aws.secret.access.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The secret access key used to access AWS Secrets Manager.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
</div>
</div>
<div class="sect2">
<h3 id="azure-key-vault-key-provider"><a class="anchor" href="administration-guide.html#azure-key-vault-key-provider"></a>Azure Key Vault Key Provider</h3>
<div class="paragraph">
<p>This protection scheme uses keys managed by
<a href="https://docs.microsoft.com/en-us/azure/key-vault/keys/about-keys">Azure Key Vault Keys</a> for encryption and decryption.</p>
</div>
<div class="paragraph">
<p>Azure Key Vault configuration properties can be stored in the <code>bootstrap-azure.conf</code> file, as referenced in the
<code>bootstrap.conf</code> of NiFi or NiFi Registry.
The provider will use the
<a href="https://docs.microsoft.com/en-us/java/api/com.azure.identity.defaultazurecredential">DefaultAzureCredential</a>
for authentication.
The <a href="https://docs.microsoft.com/en-us/java/api/overview/azure/identity-readme#key-concepts">Azure Identity</a> client library
describes the process for credentials resolution, which leverages environment variables, system properties, and falls
back to
<a href="https://docs.microsoft.com/en-us/java/api/overview/azure/identity-readme#managed-identity-support">Managed Identity</a>
authentication.</p>
</div>
<div class="sect3">
<h4 id="required-properties-3"><a class="anchor" href="administration-guide.html#required-properties-3"></a>Required properties</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>azure.keyvault.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The identifier of the key that the Azure Key Vault client uses for encryption and decryption.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>azure.keyvault.encryption.algorithm</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The encryption algorithm that the Azure Key Vault client uses for encryption and decryption.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
</div>
<div class="sect2">
<h3 id="azure-key-vault-secret-provider"><a class="anchor" href="administration-guide.html#azure-key-vault-secret-provider"></a>Azure Key Vault Secret Provider</h3>
<div class="paragraph">
<p>This protection scheme uses secrets managed by
<a href="https://docs.microsoft.com/en-us/azure/key-vault/secrets/about-secrets">Azure Key Vault Secrets</a> for storing and
retrieving protected properties.</p>
</div>
<div class="paragraph">
<p>Azure Key Vault configuration properties can be stored in the <code>bootstrap-azure.conf</code> file, as referenced in the
<code>bootstrap.conf</code> of NiFi or NiFi Registry.
The provider will use the
<a href="https://docs.microsoft.com/en-us/java/api/com.azure.identity.defaultazurecredential">DefaultAzureCredential</a>
for authentication.
The <a href="https://docs.microsoft.com/en-us/java/api/overview/azure/identity-readme#key-concepts">Azure Identity</a> client library
describes the process for credentials resolution, which leverages environment variables, system properties, and falls
back to
<a href="https://docs.microsoft.com/en-us/java/api/overview/azure/identity-readme#managed-identity-support">Managed Identity</a>
authentication.</p>
</div>
<div class="paragraph">
<p>Names of secrets stored in Azure Key Vault support alphanumeric and dash characters, but do not support characters such as <code>/</code> or <code>.</code>.
For this reason, NiFi replaces these characters with <code>-</code> when storing and retrieving secrets. The following table provides an example property name mapping:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Context</th>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Secret Name</th>
</tr>
</thead>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>default</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keystorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>default-nifi-security-keystorePasswd</code></p></td>
</tr>
</tfoot>
</table>
<div class="sect3">
<h4 id="required-properties-4"><a class="anchor" href="administration-guide.html#required-properties-4"></a>Required properties</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>azure.keyvault.uri</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">URI for the Azure Key Vault service such as <code><a href="https://{value-name}.vault.azure.net/" class="bare">https://{value-name}.vault.azure.net/</a></code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
</div>
<div class="sect2">
<h3 id="google-cloud-kms-provider"><a class="anchor" href="administration-guide.html#google-cloud-kms-provider"></a>Google Cloud KMS provider</h3>
<div class="paragraph">
<p>This protection scheme uses Google Cloud Key Management Service (<a href="https://cloud.google.com/security-key-management">Google Cloud Key Management Service</a>) for encryption and decryption. Google Cloud KMS configuration properties are to be stored in the <code>bootstrap-gcp.conf</code> file, as referenced in the <code>bootstrap.conf</code> of NiFi or NiFi Registry. Credentials must be configured as per the following documentation: <a href="https://cloud.google.com/kms/docs/reference/libraries">Google Cloud KMS documentation</a></p>
</div>
<div class="sect3">
<h4 id="required-properties-5"><a class="anchor" href="administration-guide.html#required-properties-5"></a>Required properties</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>gcp.kms.project</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The project containing the key that the Google Cloud KMS client uses for encryption and decryption.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>gcp.kms.location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The geographic region of the project containing the key that the Google Cloud KMS client uses for encryption and decryption.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>gcp.kms.keyring</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The keyring containing the key that the Google Cloud KMS client uses for encryption and decryption.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>gcp.kms.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The key identifier that the Google Cloud KMS client uses for encryption and decryption.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
</div>
</div>
<div class="sect2">
<h3 id="property-context-mapping"><a class="anchor" href="administration-guide.html#property-context-mapping"></a>Property Context Mapping</h3>
<div class="paragraph">
<p>Some encryption providers store protected values in an external service instead of persisting the encrypted values directly in the configuration file. To support this use case, a property context is defined for each protected property in NiFi&#8217;s configuration files, in the format: <code>{context-name}/{property-name}</code></p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>context-name</code> - represents a namespace for properties in order to disambiguate properties with the same name. Without additional configuration, all protected properties are assigned the <code>default</code> context.</p>
</li>
<li>
<p><code>property-name</code> - contains the name of the property.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>In order to support logical context names, mapping properties may be provided in <code>bootstrap.conf</code>, as follows:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>nifi.bootstrap.protection.context.mapping.&lt;context-name&gt;=&lt;identifier matching regex&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Here, <code>context-name</code> would determine the context name above, and <code>&lt;identifier matching regex&gt;</code> would map any property whose <strong>group identifier</strong> matched the provided Regular Expression. <strong>Group identifiers</strong> are defined per configuration file type, and are described as follows:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Configuration File</th>
<th class="tableblock halign-left valign-top">Group Identifier Description</th>
<th class="tableblock halign-left valign-top">Assigned Context</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.properties</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">There is no concept of a group identifier here, since all property names should be unique.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>default</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>authorizers.xml</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The <code>&lt;identifier&gt;</code> value of the XML block surrounding the property.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The mapped context name if RegEx matches the identifier, otherwise <em>default</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>login-identity-providers.xml</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The <code>&lt;identifier&gt;</code> value of the XML block surrounding the property.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The mapped context name if RegEx matches the identifier, otherwise <em>default</em></p></td>
</tr>
</tfoot>
</table>
<div class="sect3">
<h4 id="example"><a class="anchor" href="administration-guide.html#example"></a>Example</h4>
<div class="paragraph">
<p>In the NiFi binary distribution, the <code>login-identity-providers.xml</code> file comes with a provider with the identifier <code>ldap-provider</code> and a property called <code>Manager Password</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code> &lt;provider&gt;
&lt;identifier&gt;ldap-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.ldap.LdapProvider&lt;/class&gt;
...
&lt;property name="Manager Password"/&gt;
...
&lt;/provider&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Similarly, the <code>authorizers.xml</code> file comes with a <code>ldap-user-group-provider</code> and a property also called <code>Manager Password</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code> &lt;userGroupProvider&gt;
&lt;identifier&gt;ldap-user-group-provider&lt;/identifier&gt;
&lt;class&gt;org.apache.nifi.ldap.tenants.LdapUserGroupProvider&lt;/class&gt;
...
&lt;property name="Manager Password"/&gt;
...
&lt;/userGroupProvider&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>If the Manager Password is desired to reference the same exact property (e.g., the same Secret in the HashiCorp Vault K/V provider) but still be distinguished from any other <code>Manager Password</code> property unrelated to LDAP, the following mapping could be added:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>nifi.bootstrap.protection.context.mapping.ldap=ldap-.*</code></pre>
</div>
</div>
<div class="paragraph">
<p>This would cause both of the above to be assigned a context of <code>"ldap/Manager Password"</code> instead of <code>"default/Manager Password"</code>.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="admin-toolkit"><a class="anchor" href="administration-guide.html#admin-toolkit"></a>NiFi Toolkit Administrative Tools</h2>
<div class="sectionbody">
<div class="paragraph">
<p>In addition to <code>tls-toolkit</code> and <code>encrypt-config</code>, the NiFi Toolkit also contains command line utilities for administrators to support NiFi maintenance in standalone and clustered environments. These utilities include:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>CLI&#8201;&#8212;&#8201;The <code>cli</code> tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes.</p>
</li>
<li>
<p>File Manager&#8201;&#8212;&#8201;The <code>file-manager</code> tool enables administrators to backup, install or restore a NiFi installation from backup.</p>
</li>
<li>
<p>Flow Analyzer&#8201;&#8212;&#8201;The <code>flow-analyzer</code> tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow.</p>
</li>
<li>
<p>Node Manager&#8201;&#8212;&#8201;The <code>node-manager</code> tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster.</p>
</li>
<li>
<p>Notify&#8201;&#8212;&#8201;The <code>notify</code> tool enables administrators to send bulletins to the NiFi UI.</p>
</li>
<li>
<p>S2S&#8201;&#8212;&#8201;The <code>s2s</code> tool enables administrators to send data into or out of NiFi flows over site-to-site.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For more information about each utility, see the <a href="toolkit-guide.html">NiFi Toolkit Guide</a>.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="clustering"><a class="anchor" href="administration-guide.html#clustering"></a>Clustering Configuration</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This section provides a quick overview of NiFi Clustering and instructions on how to set up a basic cluster.
In the future, we hope to provide supplemental documentation that covers the NiFi Cluster Architecture in depth.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="images/zero-leader-cluster-http-access.png" alt="NiFi Cluster HTTP Access">
</div>
</div>
<div class="sect2">
<h3 id="zero-leader-clustering"><a class="anchor" href="administration-guide.html#zero-leader-clustering"></a>Zero-Leader Clustering</h3>
<div class="paragraph">
<p>NiFi employs a Zero-Leader Clustering paradigm. Each node in the cluster has an identical flow and performs the same tasks on
the data, but each operates on a different set of data. The cluster automatically distributes the data throughout all the active nodes.</p>
</div>
<div class="paragraph">
<p>One of the nodes is automatically elected (via Apache
ZooKeeper) as the Cluster Coordinator. All nodes in the cluster will then send heartbeat/status information
to this node, and this node is responsible for disconnecting nodes that do not report any heartbeat status
for some amount of time. Additionally, when a new node elects to join the cluster, the new node must first
connect to the currently-elected Cluster Coordinator in order to obtain the most up-to-date flow. If the Cluster
Coordinator determines that the node is allowed to join (based on its configured <a href="administration-guide.html#cluster_firewall_configuration">Firewall</a> file), the current
flow is provided to that node, and that node is able to join the cluster, assuming that the node&#8217;s copy of the
flow matches the copy provided by the Cluster Coordinator. If the node&#8217;s version of the flow configuration differs
from that of the Cluster Coordinator&#8217;s, the node will not join the cluster.</p>
</div>
</div>
<div class="sect2">
<h3 id="why-cluster"><a class="anchor" href="administration-guide.html#why-cluster"></a>Why Cluster?</h3>
<div class="paragraph">
<p>NiFi Administrators or DataFlow Managers (DFMs) may find that using one instance of NiFi on a single server is not
enough to process the amount of data they have. So, one solution is to run the same dataflow on multiple NiFi servers.
However, this creates a management problem, because each time DFMs want to change or update the dataflow, they must make
those changes on each server and then monitor each server individually. By clustering the NiFi servers, it&#8217;s possible to
have that increased processing capability along with a single interface through which to make dataflow changes and monitor
the dataflow. Clustering allows the DFM to make each change only once, and that change is then replicated to all the nodes
of the cluster. Through the single interface, the DFM may also monitor the health and status of all the nodes.</p>
</div>
</div>
<div class="sect2">
<h3 id="terminology"><a class="anchor" href="administration-guide.html#terminology"></a>Terminology</h3>
<div class="paragraph">
<p>NiFi Clustering is unique and has its own terminology. It&#8217;s important to understand the following terms before setting up a cluster:</p>
</div>
<div class="paragraph">
<p><strong>NiFi Cluster Coordinator</strong>: A NiFi Cluster Coordinator is the node in a NiFi cluster that is responsible for carrying out
tasks to manage which nodes are allowed in the cluster and providing the most up-to-date flow to newly joining nodes. When a
DataFlow Manager manages a dataflow in a cluster, they are able to do so through the User Interface of any node in the cluster. Any
change made is then replicated to all nodes in the cluster.</p>
</div>
<div class="paragraph">
<p><strong>Nodes</strong>: Each cluster is made up of one or more nodes. The nodes do the actual data processing.</p>
</div>
<div class="paragraph">
<p><strong>Primary Node</strong>: Every cluster has one Primary Node. On this node, it is possible to run "Isolated Processors" (see below).
ZooKeeper is used to automatically elect a Primary Node. If that node disconnects from the cluster for any reason, a new
Primary Node will automatically be elected. Users can determine which node is currently elected as the Primary Node by
looking at the Cluster Management page of the User Interface.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="images/primary-node-cluster-mgt.png" alt="Primary Node in Cluster Management UI">
</div>
</div>
<div class="paragraph">
<p><strong>Isolated Processors</strong>: In a NiFi cluster, the same dataflow runs on all the nodes. As a result, every component in the flow
runs on every node. However, there may be cases when the DFM would not want every processor to run on every node. The most
common case is when using a processor that communicates with an external service using a protocol that does not scale well.
For example, the GetSFTP processor pulls from a remote directory. If the GetSFTP Processor runs on every node in the
cluster and tries simultaneously to pull from the same remote directory, there could be race conditions. Therefore, the DFM could
configure the GetSFTP on the Primary Node to run in isolation, meaning that it only runs on that node. With the proper dataflow configuration, it could pull in data and load-balance it across the rest of the nodes in the cluster. Note that while this
feature exists, it is also very common to simply use a standalone NiFi instance to pull data and feed it to the cluster.
It just depends on the resources available and how the Administrator decides to configure the cluster.</p>
</div>
<div class="paragraph">
<p><strong>Heartbeats</strong>: The nodes communicate their health and status to the currently elected Cluster Coordinator via "heartbeats",
which let the Coordinator know they are still connected to the cluster and working properly. By default, the nodes emit
heartbeats every 5 seconds, and if the Cluster Coordinator does not receive a heartbeat from a node within 40 seconds (= 5 seconds * 8), it
disconnects the node due to "lack of heartbeat". The 5-second and 8 times settings are configurable in the <em>nifi.properties</em> file (see
the <a href="administration-guide.html#cluster_common_properties">Cluster Common Properties</a> section for more information). The reason that the Cluster Coordinator
disconnects the node is because the Coordinator needs to ensure that every node in the cluster is in sync, and if a node
is not heard from regularly, the Coordinator cannot be sure it is still in sync with the rest of the cluster. If, after
40 seconds, the node does send a new heartbeat, the Coordinator will automatically request that the node re-join the cluster,
to include the re-validation of the node&#8217;s flow.
Both the disconnection due to lack of heartbeat and the reconnection once a heartbeat is received are reported to the DFM
in the User Interface.</p>
</div>
</div>
<div class="sect2">
<h3 id="communication-within-the-cluster"><a class="anchor" href="administration-guide.html#communication-within-the-cluster"></a>Communication within the Cluster</h3>
<div class="paragraph">
<p>As noted, the nodes communicate with the Cluster Coordinator via heartbeats. When a Cluster Coordinator is elected, it updates
a well-known ZNode in Apache ZooKeeper with its connection information so that nodes understand where to send heartbeats. If one
of the nodes goes down, the other nodes in the cluster will not automatically pick up the load of the missing node. It is possible
for the DFM to configure the dataflow for failover contingencies; however, this is dependent on the dataflow design and does not
happen automatically.</p>
</div>
<div class="paragraph">
<p>When the DFM makes changes to the dataflow, the node that receives the request to change the flow communicates those changes to all
nodes and waits for each node to respond, indicating that it has made the change on its local flow.</p>
</div>
</div>
<div class="sect2">
<h3 id="managing_nodes"><a class="anchor" href="administration-guide.html#managing_nodes"></a>Managing Nodes</h3>
<div class="sect3">
<h4 id="disconnect-nodes"><a class="anchor" href="administration-guide.html#disconnect-nodes"></a>Disconnect Nodes</h4>
<div class="paragraph">
<p>A DFM may manually disconnect a node from the cluster. A node may also become disconnected for other reasons, such as due to a lack of heartbeat. The Cluster Coordinator will show a bulletin on the User Interface when a node is disconnected. The DFM will not be able to make any changes to the dataflow until the issue of the disconnected node is resolved. The DFM or the Administrator will need to troubleshoot the issue with the node and resolve it before any new changes can be made to the dataflow. However, it is worth noting that just because a node is disconnected does not mean that it is not working. This may happen for a few reasons, for example when the node is unable to communicate with the Cluster Coordinator due to network problems.</p>
</div>
<div class="paragraph">
<p>To manually disconnect a node, select the "Disconnect" icon (<span class="image"><img src="images/iconDisconnect.png" alt="Disconnect Icon"></span>) from the node&#8217;s row.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="images/disconnected-node-cluster-mgt.png" alt="Disconnected Node in Cluster Management UI">
</div>
</div>
<div class="paragraph">
<p>A disconnected node can be connected (<span class="image"><img src="images/iconConnect.png" alt="Connect Icon"></span>), offloaded (<span class="image"><img src="images/iconOffload.png" alt="Offload Icon"></span>) or deleted (<span class="image"><img src="images/iconDelete.png" alt="Delete Icon"></span>).</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Not all nodes in a "Disconnected" state can be offloaded. If the node is disconnected and unreachable, the offload request can not be received by the node to start the offloading. Additionally, offloading may be interrupted or prevented due to firewall rules.
</td>
</tr>
</table>
</div>
</div>
<div class="sect3">
<h4 id="offload-nodes"><a class="anchor" href="administration-guide.html#offload-nodes"></a>Offload Nodes</h4>
<div class="paragraph">
<p>Flowfiles that remain on a disconnected node can be rebalanced to other active nodes in the cluster via offloading. In the Cluster Management dialog, select the "Offload" icon (<span class="image"><img src="images/iconOffload.png" alt="Offload Icon"></span>) for a Disconnected node. This will stop all processors, terminate all processors, stop transmitting on all remote process groups and rebalance flowfiles to the other connected nodes in the cluster.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="images/offloading-node-cluster-mgt.png" alt="Offloading Node in Cluster Management UI">
</div>
</div>
<div class="paragraph">
<p>Nodes that remain in "Offloading" state due to errors encountered (out of memory, no network connection, etc.) can be reconnected to the cluster by restarting NiFi on the node. Offloaded nodes can be either reconnected to the cluster (by selecting Connect or restarting NiFi on the node) or deleted from the cluster.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="images/offloaded-node-cluster-mgt.png" alt="Offloaded Node in Cluster Management UI">
</div>
</div>
</div>
<div class="sect3">
<h4 id="delete-nodes"><a class="anchor" href="administration-guide.html#delete-nodes"></a>Delete Nodes</h4>
<div class="paragraph">
<p>There are cases where a DFM may wish to continue making changes to the flow, even though a node is not connected to the cluster. In this case, the DFM may elect to delete the node from the cluster entirely. In the Cluster Management dialog, select the "Delete" icon (<span class="image"><img src="images/iconDelete.png" alt="Delete Icon"></span>) for a Disconnected or Offloaded node. Once deleted, the node cannot be rejoined to the cluster until it has been restarted.</p>
</div>
</div>
<div class="sect3">
<h4 id="decommission-nodes"><a class="anchor" href="administration-guide.html#decommission-nodes"></a>Decommission Nodes</h4>
<div class="paragraph">
<p>The steps to decommission a node and remove it from a cluster are as follows:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Disconnect the node.</p>
</li>
<li>
<p>Once disconnect completes, offload the node.</p>
</li>
<li>
<p>Once offload completes, delete the node.</p>
</li>
<li>
<p>Once the delete request has finished, stop/remove the NiFi service on the host.</p>
</li>
</ol>
</div>
</div>
<div class="sect3">
<h4 id="nifi-cli-node-commands"><a class="anchor" href="administration-guide.html#nifi-cli-node-commands"></a>NiFi CLI Node Commands</h4>
<div class="paragraph">
<p>As an alternative to the UI, the following NiFi CLI commands can be used for retrieving a single node, retrieving a list of nodes, and connecting/disconnecting/offloading/deleting nodes:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>nifi get-node</code></p>
</li>
<li>
<p><code>nifi get-nodes</code></p>
</li>
<li>
<p><code>nifi connect-node</code></p>
</li>
<li>
<p><code>nifi disconnect-node</code></p>
</li>
<li>
<p><code>nifi offload-node</code></p>
</li>
<li>
<p><code>nifi delete-node</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For more information, see the <a href="toolkit-guide.html#nifi_CLI">NiFi CLI</a> section in the <a href="toolkit-guide.html">NiFi Toolkit Guide</a>.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="flow-election"><a class="anchor" href="administration-guide.html#flow-election"></a>Flow Election</h3>
<div class="paragraph">
<p>When a cluster first starts up, NiFi must determine which of the nodes have the
"correct" version of the flow. This is done by voting on the flows that each of the nodes has. When a node
attempts to connect to a cluster, it provides a copy of its local flow and (if the policy provider allows for configuration via NiFi)
its users, groups, and policies, to the Cluster Coordinator. If no flow
has yet been elected the "correct" flow, the node&#8217;s flow is compared to each of the other Nodes' flows. If another
Node&#8217;s flow matches this one, a vote is cast for this flow. If no other Node has reported the same flow yet, this
flow will be added to the pool of possibly elected flows with one vote. After
some amount of time has elapsed (configured by setting the <code>nifi.cluster.flow.election.max.wait.time</code> property) or
some number of Nodes have cast votes (configured by setting the <code>nifi.cluster.flow.election.max.candidates</code> property),
a flow is elected to be the "correct" copy of the flow.</p>
</div>
<div class="paragraph">
<p>Any node whose dataflow, users, groups, and policies conflict with those elected will backup any conflicting resources and replace the local
resources with those from the cluster. How the backup is performed depends on the configured Access Policy Provider and User Group Provider.
For file-based access policy providers, the backup will be written to the same directory as the existing file (e.g., $NIFI_HOME/conf) and bear the same
name but with a suffix of "." and a timestamp. For example, if the flow itself conflicts with the cluster&#8217;s flow at 12:05:03 on January 1, 2020,
the node&#8217;s <code>flow.json.gz</code> file will be copied to <code>flow.json.gz.2020-01-01-12-05-03</code> and the cluster&#8217;s flow will then be written to <code>flow.json.gz</code>.
Similarly, this will happen for the <code>users.xml</code> and <code>authorizations.xml</code> file. This is done so that the flow can be manually reverted if necessary
by renaming the backup file back to <code>flow.json.gz</code>, for example.</p>
</div>
<div class="paragraph">
<p>It is important to note that before inheriting the elected flow, NiFi will first read through the FlowFile repository and any swap files to determine which
queues in the dataflow currently hold data. If there exists any queue in the dataflow that contains a FlowFile, that queue must also exist in the elected
dataflow. If that queue does not exist in the elected dataflow, the node will not inherit the dataflow, users, groups, and policies. Instead, NiFi will
log errors to that effect and will fail to startup. This ensures that even if the node has data stored in a connection, and the cluster&#8217;s dataflow is different,
restarting the node will not result in data loss.</p>
</div>
<div class="paragraph">
<p>Election is performed according to the "popular vote" with the caveat that the winner will never be an "empty flow" unless all flows are empty. This
allows an administrator to remove a node&#8217;s <em>flow.json.gz</em> file and restart the node, knowing that the node&#8217;s flow will
not be voted to be the "correct" flow unless no other flow is found. If there are two non-empty flows that receive the same number of votes, one of those
flows will be chosen. The methodology used to determine which of those flows is undefined and may change at any time without notice.</p>
</div>
</div>
<div class="sect2">
<h3 id="basic-cluster-setup"><a class="anchor" href="administration-guide.html#basic-cluster-setup"></a>Basic Cluster Setup</h3>
<div class="paragraph">
<p>This section describes the setup for a simple three-node, non-secure cluster comprised of three instances of NiFi.</p>
</div>
<div class="paragraph">
<p>For each instance, certain properties in the <em>nifi.properties</em> file will need to be updated. In particular, the Web and Clustering properties
should be evaluated for your situation and adjusted accordingly. All the properties are described in the <a href="administration-guide.html#system_properties">System Properties</a> section of this
guide; however, in this section, we will focus on the minimum properties that must be set for a simple cluster.</p>
</div>
<div class="paragraph">
<p>For all three instances, the <a href="administration-guide.html#cluster_common_properties">Cluster Common Properties</a> can be left with the default settings. Note, however, that if you change these settings,
they must be set the same on every instance in the cluster.</p>
</div>
<div class="paragraph">
<p>For each Node, the minimum properties to configure are as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Under the <em>Web Properties</em> section, set either the HTTP or HTTPS port that you want the Node to run on.
Also, consider whether you need to set the HTTP or HTTPS host property. All nodes in the cluster should use the same protocol setting.</p>
</li>
<li>
<p>Under the <em>State Management section</em>, set the <code>nifi.state.management.provider.cluster</code> property
to the identifier of the Cluster State Provider. Ensure that the Cluster State Provider has been
configured in the <em>state-management.xml</em> file. See <a href="administration-guide.html#state_providers">Configuring State Providers</a> for more information.</p>
</li>
<li>
<p>Under <em>Cluster Node Properties</em>, set the following:</p>
<div class="ulist">
<ul>
<li>
<p><code>nifi.cluster.is.node</code> - Set this to <em>true</em>.</p>
</li>
<li>
<p><code>nifi.cluster.node.address</code> - Set this to the fully qualified hostname of the node. If left blank, it defaults to <code>localhost</code>.</p>
</li>
<li>
<p><code>nifi.cluster.node.protocol.port</code> - Set this to an open port that is higher than 1024 (anything lower requires root).</p>
</li>
<li>
<p><code>nifi.cluster.node.protocol.max.threads</code> - The maximum number of threads that should be used to communicate with other nodes in the cluster. This property
defaults to <code>50</code>. A thread pool is used for replicating requests to all nodes. The thread pool will increase the number of active threads to the limit
set by this property. It is typically recommended that this property be set to 4-8 times the number of nodes in your cluster. There could be up to <code>n+2</code> threads for a given request, where <code>n</code> = number of nodes in your cluster. As an example, if 4 requests are made, a 5 node cluster will use <code>4 * 7 = 28</code> threads.</p>
</li>
<li>
<p><code>nifi.zookeeper.connect.string</code> - The Connect String that is needed to connect to Apache ZooKeeper. This is a comma-separated list
of hostname:port pairs. For example, <code>localhost:2181,localhost:2182,localhost:2183</code>. This should contain a list of all ZooKeeper
instances in the ZooKeeper quorum.</p>
</li>
<li>
<p><code>nifi.zookeeper.root.node</code> - The root ZNode that should be used in ZooKeeper. ZooKeeper provides a directory-like structure
for storing data. Each 'directory' in this structure is referred to as a ZNode. This denotes the root ZNode, or 'directory',
that should be used for storing data. The default value is <code>/root</code>. This is important to set correctly, as which cluster
the NiFi instance attempts to join is determined by which ZooKeeper instance it connects to and the ZooKeeper Root Node
that is specified.</p>
</li>
<li>
<p><code>nifi.cluster.flow.election.max.wait.time</code> - Specifies the amount of time to wait before electing a Flow as the "correct" Flow.
If the number of Nodes that have voted is equal to the number specified by the <code>nifi.cluster.flow.election.max.candidates</code>
property, the cluster will not wait this long. The default value is <code>5 mins</code>. Note that the time starts as soon as the first vote
is cast.</p>
</li>
<li>
<p><code>nifi.cluster.flow.election.max.candidates</code> - Specifies the number of Nodes required in the cluster to cause early election
of Flows. This allows the Nodes in the cluster to avoid having to wait a long time before starting processing if we reach
at least this number of nodes in the cluster.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Now, it is possible to start up the cluster. It does not matter which order the instances start up. Navigate to the URL for
one of the nodes, and the User Interface should look similar to the following:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/ncm.png" alt="Clustered User Interface"></span></p>
</div>
</div>
<div class="sect2">
<h3 id="cluster_firewall_configuration"><a class="anchor" href="administration-guide.html#cluster_firewall_configuration"></a>Cluster Firewall Configuration</h3>
<div class="paragraph">
<p>NiFi clustering supports network access restrictions using a custom firewall configuration.
The <code>nifi.cluster.firewall.file</code> property can be configured with a path to a file containing hostnames, IP addresses, or
subnets of permitted nodes. The Cluster Coordinator uses the configuration to determine whether to accept or reject
heartbeats and connection requests from potential cluster members.</p>
</div>
<div class="paragraph">
<p>The configuration file format expects one entry per line and ignores lines beginning with the <code>#</code> character. NiFi uses
standard Java host name resolution to convert names to IP addresses. Java host name resolution leverages a combination
of local machine configuration and network services, such as DNS. The configuration file supports IPv4 addresses or subnet
ranges using CIDR notation. The following example cluster firewall configuration includes a combination of supported entries:</p>
</div>
<div class="listingblock">
<div class="content">
<pre># Cluster Node Hostnames
nifi0.example.com
nifi1.example.com
nifi3.example.com
# Cluster Node Addresses
192.168.0.1
192.168.0.2
192.168.0.3
# Cluster Subnet Address
192.168.0.0/29 # Address Range from 192.168.0.1 to 192.168.0.6</pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="troubleshooting"><a class="anchor" href="administration-guide.html#troubleshooting"></a>Troubleshooting</h3>
<div class="paragraph">
<p>If you encounter issues and your cluster does not work as described, investigate the <em>nifi-app.log</em> and <em>nifi-user.log</em>
files on the nodes. If needed, you can change the logging level to DEBUG by editing the <code>conf/logback.xml</code> file. Specifically,
set the <code>level="DEBUG"</code> in the following line (instead of <code>"INFO"</code>):</p>
</div>
<div class="listingblock">
<div class="content">
<pre> &lt;logger name="org.apache.nifi.web.api.config" level="INFO" additivity="false"&gt;
&lt;appender-ref ref="USER_FILE"/&gt;
&lt;/logger&gt;</pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="state_management"><a class="anchor" href="administration-guide.html#state_management"></a>State Management</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi provides a mechanism for Processors, Reporting Tasks, Controller Services, and the framework itself to persist state. This
allows a Processor, for example, to resume from the place where it left off after NiFi is restarted. Additionally, it allows for
a Processor to store some piece of information so that the Processor can access that information from all of the different nodes
in the cluster. This allows one node to pick up where another node left off, or to coordinate across all of the nodes in a cluster.</p>
</div>
<div class="sect2">
<h3 id="state_providers"><a class="anchor" href="administration-guide.html#state_providers"></a>Configuring State Providers</h3>
<div class="paragraph">
<p>When a component decides to store or retrieve state, it does so by providing a "Scope" - either Node-local or Cluster-wide. The
mechanism that is used to store and retrieve this state is then determined based on this Scope, as well as the configured State
Providers. The <em>nifi.properties</em> file contains three different properties that are relevant to configuring these State Providers.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.configuration.file</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The first is the property that specifies an external XML file that is used for configuring the local and/or cluster-wide State Providers. This XML file may contain configurations for multiple providers</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.provider.local</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The property that provides the identifier of the local State Provider configured in this XML file</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.provider.cluster</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Similarly, the property provides the identifier of the cluster-wide State Provider configured in this XML file.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>This XML file consists of a top-level <code>state-management</code> element, which has one or more <code>local-provider</code> and zero or more <code>cluster-provider</code>
elements. Each of these elements then contains an <code>id</code> element that is used to specify the identifier that can be referenced in the
<em>nifi.properties</em> file, as well as a <code>class</code> element that specifies the fully-qualified class name to use in order to instantiate the State
Provider. Finally, each of these elements may have zero or more <code>property</code> elements. Each <code>property</code> element has an attribute, <code>name</code> that is the name
of the <code>property</code> that the State Provider supports. The textual content of the property element is the value of the property.</p>
</div>
<div class="paragraph">
<p>Once these State Providers have been configured in the <em>state-management.xml</em> file (or whatever file is configured), those Providers may be
referenced by their identifiers.</p>
</div>
<div class="paragraph">
<p>By default, the Local State Provider is configured to be a <code>WriteAheadLocalStateProvider</code> that persists the data to the
<code>$NIFI_HOME/state/local</code> directory. The default Cluster State Provider is configured to be a <code>ZooKeeperStateProvider</code>. The default
ZooKeeper-based provider must have its <code>Connect String</code> property populated before it can be used. It is also advisable, if multiple NiFi instances
will use the same ZooKeeper instance, that the value of the <code>Root Node</code> property be changed. For instance, one might set the value to
<code>/nifi/&lt;team name&gt;/production</code>. A <code>Connect String</code> takes the form of comma separated &lt;host&gt;:&lt;port&gt; tuples, such as
<code>my-zk-server1:2181,my-zk-server2:2181,my-zk-server3:2181</code>. In the event a port is not specified for any of the hosts, the ZooKeeper default of
<code>2181</code> is assumed.</p>
</div>
<div class="paragraph">
<p>When adding data to ZooKeeper, there are two options for Access Control: <code>Open</code> and <code>CreatorOnly</code>. If the <code>Access Control</code> property is
set to <code>Open</code>, then anyone is allowed to log into ZooKeeper and have full permissions to see, change, delete, or administer the data.
If <code>CreatorOnly</code> is specified, then only the user that created the data is allowed to read, change, delete, or administer the data.
In order to use the <code>CreatorOnly</code> option, NiFi must provide some form of authentication. See the <a href="administration-guide.html#zk_access_control">ZooKeeper Access Control</a>
section below for more information on how to configure authentication.</p>
</div>
<div class="paragraph">
<p>If NiFi is configured to run in a standalone mode, the <code>cluster-provider</code> element need not be populated in the <em>state-management.xml</em>
file and will actually be ignored if they are populated. However, the <code>local-provider</code> element must always be present and populated.
Additionally, if NiFi is run in a cluster, each node must also have the <code>cluster-provider</code> element present and properly configured.
Otherwise, NiFi will fail to startup.</p>
</div>
<div class="paragraph">
<p>While there are not many properties that need to be configured for these providers, they were externalized into a separate <em>state-management.xml</em>
file, rather than being configured via the <em>nifi.properties</em> file, simply because different implementations may require different properties,
and it is easier to maintain and understand the configuration in an XML-based file such as this, than to mix the properties of the Provider
in with all of the other NiFi framework-specific properties.</p>
</div>
<div class="paragraph">
<p>It should be noted that if Processors and other components save state using the Clustered scope, the Local State Provider will be used
if the instance is a standalone instance (not in a cluster) or is disconnected from the cluster. This also means that if a standalone instance
is migrated to become a cluster, then that state will no longer be available, as the component will begin using the Clustered State Provider
instead of the Local State Provider.</p>
</div>
</div>
<div class="sect2">
<h3 id="embedded_zookeeper"><a class="anchor" href="administration-guide.html#embedded_zookeeper"></a>Embedded ZooKeeper Server</h3>
<div class="paragraph">
<p>As mentioned above, the default State Provider for cluster-wide state is the <code>ZooKeeperStateProvider</code>. At the time of this writing, this is the
only State Provider that exists for handling cluster-wide state. What this means is that NiFi has dependencies on ZooKeeper in order to
behave as a cluster. However, there are many environments in which NiFi is deployed where there is no existing ZooKeeper ensemble being maintained.
In order to avoid the burden of forcing administrators to also maintain a separate ZooKeeper instance, NiFi provides the option of starting an
embedded ZooKeeper server.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.embedded.zookeeper.start</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.embedded.zookeeper.properties</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Properties file that provides the ZooKeeper properties to use if <code>nifi.state.management.embedded.zookeeper.start</code> is set to <code>true</code></p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>This can be accomplished by setting the <code>nifi.state.management.embedded.zookeeper.start</code> property in <em>nifi.properties</em> to <code>true</code> on those nodes
that should run the embedded ZooKeeper server. Generally, it is advisable to run ZooKeeper on either 3 or 5 nodes. Running on fewer than 3 nodes
provides less durability in the face of failure. Running on more than 5 nodes generally produces more network traffic than is necessary. Additionally,
running ZooKeeper on 4 nodes provides no more benefit than running on 3 nodes, ZooKeeper requires a majority of nodes be active in order to function.
However, it is up to the administrator to determine the number of nodes most appropriate to the particular deployment of NiFi.</p>
</div>
<div class="paragraph">
<p>If the <code>nifi.state.management.embedded.zookeeper.start</code> property is set to <code>true</code>, the <code>nifi.state.management.embedded.zookeeper.properties</code> property
in <em>nifi.properties</em> also becomes relevant. This specifies the ZooKeeper properties file to use. At a minimum, this properties file needs to be populated
with the list of ZooKeeper servers. The servers are specified as properties in the form of <code>server.1</code>, <code>server.2</code>, to <code>server.n</code>. As of NiFi 1.10.x, ZooKeeper
has been upgraded to 3.5.5 and servers are now defined with the client port appended at the end as per the <a href="https://zookeeper.apache.org/doc/r3.5.2-alpha/zookeeperReconfig.html#sc_reconfig_clientport">ZooKeeper Documentation</a>.
As such, each of these servers is configured as &lt;hostname&gt;:&lt;quorum port&gt;[:&lt;leader election port&gt;][:role];[&lt;client port address&gt;:]&lt;client port&gt;.
As a simple example this would be <code>server.1 = myhost:2888:3888;2181</code>. This list of nodes should be the same nodes in the NiFi cluster that have the <code>nifi.state.management.embedded.zookeeper.start</code> property set to <code>true</code>.
Also note that because ZooKeeper will be listening on these ports, the firewall may need to be configured to open these ports for incoming traffic, at least between nodes in the cluster.</p>
</div>
<div class="paragraph">
<p>When using an embedded ZooKeeper, the ./<em>conf/zookeeper.properties</em> file has a property named <code>dataDir</code>. By default, this value is set to <code>./state/zookeeper</code>.
If more than one NiFi node is running an embedded ZooKeeper, it is important to tell the server which one it is. This is accomplished by creating a file named
<em>myid</em> and placing it in ZooKeeper’s data directory. The contents of this file should be the index of the server as specific by the <code>server.&lt;number&gt;</code>. So for
one of the ZooKeeper servers, we will accomplish this by performing the following commands:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>cd $NIFI_HOME
mkdir state
mkdir state/zookeeper
echo 1 &gt; state/zookeeper/myid</code></pre>
</div>
</div>
<div class="paragraph">
<p>For the next NiFi Node that will run ZooKeeper, we can accomplish this by performing the following commands:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>cd $NIFI_HOME
mkdir state
mkdir state/zookeeper
echo 2 &gt; state/zookeeper/myid</code></pre>
</div>
</div>
<div class="paragraph">
<p>And so on.</p>
</div>
<div class="paragraph">
<p>For more information on the properties used to administer ZooKeeper, see the
<a href="https://zookeeper.apache.org/doc/current/zookeeperAdmin.html" target="_blank" rel="noopener">ZooKeeper Admin Guide</a>.</p>
</div>
<div class="paragraph">
<p>For information on securing the embedded ZooKeeper Server, see the <a href="administration-guide.html#securing_zookeeper">Securing ZooKeeper with Kerberos</a> section below.</p>
</div>
</div>
<div class="sect2">
<h3 id="zk_access_control"><a class="anchor" href="administration-guide.html#zk_access_control"></a>ZooKeeper Access Control</h3>
<div class="paragraph">
<p>ZooKeeper provides Access Control to its data via an Access Control List (ACL) mechanism. When data is written to ZooKeeper, NiFi will provide an ACL
that indicates that any user is allowed to have full permissions to the data, or an ACL that indicates that only the user that created the data is
allowed to access the data. Which ACL is used depends on the value of the <code>Access Control</code> property for the <code>ZooKeeperStateProvider</code> (see the
<a href="administration-guide.html#state_providers">Configuring State Providers</a> section for more information).</p>
</div>
<div class="paragraph">
<p>In order to use an ACL that indicates that only the Creator is allowed to access the data, we need to tell ZooKeeper who the Creator is. There are three
mechanisms for accomplishing this. The first mechanism is to provide authentication using Kerberos. See <a href="administration-guide.html#zk_kerberos_client">Kerberizing NiFi&#8217;s ZooKeeper Client</a> for more information.</p>
</div>
<div class="paragraph">
<p>The second option, which additionally ensures that network communication is encrypted, is to authenticate using an X.509 certificate on a TLS-enabled ZooKeeper
server. See <a href="administration-guide.html#zk_tls_client">Securing ZooKeeper with TLS</a> for more information.</p>
</div>
<div class="paragraph">
<p>The third option is to use a username and password. This is configured by specifying a value for the <code>Username</code> and a value for the <code>Password</code> properties
for the <code>ZooKeeperStateProvider</code> (see the <a href="administration-guide.html#state_providers">Configuring State Providers</a> section for more information). The important thing to keep in mind here, though, is that ZooKeeper
will pass around the password in plain text. This means that using a username and password should not be used unless ZooKeeper is running on localhost as a
one-instance cluster, or if communications with ZooKeeper occur only over encrypted communications, such as a VPN or an SSL connection.</p>
</div>
</div>
<div class="sect2">
<h3 id="securing_zookeeper"><a class="anchor" href="administration-guide.html#securing_zookeeper"></a>Securing ZooKeeper with Kerberos</h3>
<div class="paragraph">
<p>When NiFi communicates with ZooKeeper, all communications, by default, are non-secure, and anyone who logs into ZooKeeper is able to view and manipulate all
of the NiFi state that is stored in ZooKeeper. To prevent this, one option is to use Kerberos to manage authentication.</p>
</div>
<div class="paragraph">
<p>In order to secure the communications with Kerberos, we need to ensure that both the client and the server support the same configuration. Instructions for configuring the
NiFi ZooKeeper client and embedded ZooKeeper server to use Kerberos are provided below.</p>
</div>
<div class="paragraph">
<p>If Kerberos is not already setup in your environment, you can find information on installing and setting up a Kerberos Server at
<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html">Red Hat Customer Portal: Configuring a Kerberos 5 Server</a>. This guide assumes that Kerberos already has been installed in the environment in which NiFi is running.</p>
</div>
<div class="paragraph">
<p>Note, the following procedures for kerberizing an Embedded ZooKeeper server in your NiFi Node and kerberizing a ZooKeeper NiFi client will require that
Kerberos client libraries be installed. This is accomplished in Fedora-based Linux distributions via:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>yum install krb5-workstation</code></pre>
</div>
</div>
<div class="paragraph">
<p>Once this is complete, the <em>/etc/krb5.conf</em> will need to be configured appropriately for your organization’s Kerberos environment.</p>
</div>
<div class="sect3">
<h4 id="zk_kerberos_server"><a class="anchor" href="administration-guide.html#zk_kerberos_server"></a>Kerberizing Embedded ZooKeeper Server</h4>
<div class="paragraph">
<p>The <em>krb5.conf</em> file on the systems with the embedded zookeeper servers should be identical to the one on the system where the krb5kdc service is running.
When using the embedded ZooKeeper server, we may choose to secure the server by using Kerberos. All nodes configured to launch an embedded ZooKeeper and
using Kerberos should follow these steps. When using the embedded ZooKeeper server, we may choose to secure the server by using Kerberos. All nodes
configured to launch an embedded ZooKeeper and using Kerberos should follow these steps.</p>
</div>
<div class="paragraph">
<p>In order to use Kerberos, we first need to generate a Kerberos Principal for our ZooKeeper servers. The following command is run on the server where the
krb5kdc service is running. This is accomplished via the kadmin tool:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>kadmin: addprinc "zookeeper/myHost.example.com@EXAMPLE.COM"</code></pre>
</div>
</div>
<div class="paragraph">
<p>Here, we are creating a Principal with the primary <code>zookeeper/myHost.example.com</code>, using the realm <code>EXAMPLE.COM</code>. We need to use a Principal whose
name is <code>&lt;service name&gt;/&lt;instance name&gt;</code>. In this case, the service is <code>zookeeper</code> and the instance name is <code>myHost.example.com</code> (the fully qualified name of our host).</p>
</div>
<div class="paragraph">
<p>Next, we will need to create a KeyTab for this Principal, this command is run on the server with the NiFi instance with an embedded zookeeper server:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>kadmin: xst -k zookeeper-server.keytab zookeeper/myHost.example.com@EXAMPLE.COM</code></pre>
</div>
</div>
<div class="paragraph">
<p>This will create a file in the current directory named <code>zookeeper-server.keytab</code>. We can now copy that file into the <code>$NIFI_HOME/conf/</code> directory. We should ensure
that only the user that will be running NiFi is allowed to read this file.</p>
</div>
<div class="paragraph">
<p>We will need to repeat the above steps for each of the instances of NiFi that will be running the embedded ZooKeeper server, being sure to replace <code>myHost.example.com</code> with
<code>myHost2.example.com</code>, or whatever fully qualified hostname the ZooKeeper server will be run on.</p>
</div>
<div class="paragraph">
<p>Now that we have our KeyTab for each of the servers that will be running NiFi, we will need to configure NiFi’s embedded ZooKeeper server to use this configuration.
ZooKeeper uses the Java Authentication and Authorization Service (JAAS), so we need to create a JAAS-compatible file In the <code>$NIFI_HOME/conf/</code> directory, create a file
named <em>zookeeper-jaas.conf</em> (this file will already exist if the Client has already been configured to authenticate via Kerberos. That’s okay, just add to the file).
We will add to this file, the following snippet:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>Server {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="./conf/zookeeper-server.keytab"
storeKey=true
useTicketCache=false
principal="zookeeper/myHost.example.com@EXAMPLE.COM";
};</code></pre>
</div>
</div>
<div class="paragraph">
<p>Be sure to replace the value of <code>principal</code> above with the appropriate Principal, including the fully qualified domain name of the server.</p>
</div>
<div class="paragraph">
<p>Next, we need to tell NiFi to use this as our JAAS configuration. This is done by setting a JVM System Property, so we will edit the <em>conf/bootstrap.conf</em> file.
If the Client has already been configured to use Kerberos, this is not necessary, as it was done above. Otherwise, we will add the following line to our <em>bootstrap.conf</em> file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>java.arg.15=-Djava.security.auth.login.config=./conf/zookeeper-jaas.conf</code></pre>
</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
This additional line in the file doesn’t have to be number 15, it just has to be added to the <em>bootstrap.conf</em> file. Use whatever number is appropriate for your configuration.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>We will want to initialize our Kerberos ticket by running the following command:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>kinit –kt zookeeper-server.keytab "zookeeper/myHost.example.com@EXAMPLE.COM"</code></pre>
</div>
</div>
<div class="paragraph">
<p>Again, be sure to replace the Principal with the appropriate value, including your realm and your fully qualified hostname.</p>
</div>
<div class="paragraph">
<p>Finally, we need to tell the Kerberos server to use the SASL Authentication Provider. To do this, we edit the <em>$NIFI_HOME/conf/zookeeper.properties</em> file and add the following
lines:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
kerberos.removeHostFromPrincipal=true
kerberos.removeRealmFromPrincipal=true
jaasLoginRenew=3600000
requireClientAuthScheme=sasl</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <code>kerberos.removeHostFromPrincipal</code> and the <code>kerberos.removeRealmFromPrincipal</code> properties are used to normalize the user principal name before comparing an identity to acls
applied on a Znode. By default the full principal is used however setting the <code>kerberos.removeHostFromPrincipal</code> and the <code>kerberos.removeRealmFromPrincipal</code> properties to true will instruct
ZooKeeper to remove the host and the realm from the logged in user&#8217;s identity for comparison. In cases where NiFi nodes (within the same cluster) use principals that
have different host(s)/realm(s) values, these kerberos properties can be configured to ensure that the nodes' identity will be normalized and that the nodes will have
appropriate access to shared Znodes in ZooKeeper.</p>
</div>
<div class="paragraph">
<p>The last line is optional but specifies that clients MUST use Kerberos to communicate with our ZooKeeper instance.</p>
</div>
<div class="paragraph">
<p>Now, we can start NiFi, and the embedded ZooKeeper server will use Kerberos as the authentication mechanism.</p>
</div>
</div>
<div class="sect3">
<h4 id="zk_kerberos_client"><a class="anchor" href="administration-guide.html#zk_kerberos_client"></a>Kerberizing NiFi&#8217;s ZooKeeper Client</h4>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
The NiFi nodes running the embedded zookeeper server will also need to follow the below procedure since they will also be acting as a client at
the same time.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The preferred mechanism for authenticating users with ZooKeeper is to use Kerberos. In order to use Kerberos to authenticate, we must configure a few
system properties, so that the ZooKeeper client knows who the user is and where the KeyTab file is. All nodes configured to store cluster-wide state
using <code>ZooKeeperStateProvider</code> and using Kerberos should follow these steps.</p>
</div>
<div class="paragraph">
<p>First, we must create the Principal that we will use when communicating with ZooKeeper. This is generally done via the <code>kadmin</code> tool:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>kadmin: addprinc "nifi@EXAMPLE.COM"</code></pre>
</div>
</div>
<div class="paragraph">
<p>A Kerberos Principal is made up of three parts: the primary, the instance, and the realm. Here, we are creating a Principal with the primary <code>nifi</code>,
no instance, and the realm <code>EXAMPLE.COM</code>. The primary (<code>nifi</code>, in this case) is the identifier that will be used to identify the user when authenticating
via Kerberos.</p>
</div>
<div class="paragraph">
<p>After we have created our Principal, we will need to create a KeyTab for the Principal:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>kadmin: xst -k nifi.keytab nifi@EXAMPLE.COM</code></pre>
</div>
</div>
<div class="paragraph">
<p>This keytab file can be copied to the other NiFi nodes with embedded zookeeper servers.</p>
</div>
<div class="paragraph">
<p>This will create a file in the current directory named <em>nifi.keytab</em>. We can now copy that file into the <code>$NIFI_HOME/conf/</code> directory. We should ensure
that only the user that will be running NiFi is allowed to read this file.</p>
</div>
<div class="paragraph">
<p>Next, we need to configure NiFi to use this KeyTab for authentication. Since ZooKeeper uses the Java Authentication and Authorization Service (JAAS), we need to
create a JAAS-compatible file. In the <code>$NIFI_HOME/conf/</code> directory, create a file named <em>zookeeper-jaas.conf</em> and add to it the following snippet:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="./conf/nifi.keytab"
storeKey=true
useTicketCache=false
principal="nifi@EXAMPLE.COM";
};</code></pre>
</div>
</div>
<div class="paragraph">
<p>We then need to tell NiFi to use this as our JAAS configuration. This is done by setting a JVM System Property, so we will edit the <em>conf/bootstrap.conf</em> file.
We add the following line anywhere in this file in order to tell the NiFi JVM to use this configuration:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>java.arg.15=-Djava.security.auth.login.config=./conf/zookeeper-jaas.conf</code></pre>
</div>
</div>
<div class="paragraph">
<p>Finally we need to update <em>nifi.properties</em> to ensure that NiFi knows to apply SASL specific ACLs for the Znodes it will create in ZooKeeper for cluster management.
To enable this, in the <em>$NIFI_HOME/conf/nifi.properties</em> file and edit the following properties as shown below:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>nifi.zookeeper.auth.type=sasl
nifi.zookeeper.kerberos.removeHostFromPrincipal=true
nifi.zookeeper.kerberos.removeRealmFromPrincipal=true</code></pre>
</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
The <code>kerberos.removeHostFromPrincipal</code> and <code>kerberos.removeRealmFromPrincipal</code> should be consistent with what is set in ZooKeeper configuration.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>We can initialize our Kerberos ticket by running the following command:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>kinit -kt nifi.keytab nifi@EXAMPLE.COM</code></pre>
</div>
</div>
<div class="paragraph">
<p>Now, when we start NiFi, it will use Kerberos to authentication as the <code>nifi</code> user when communicating with ZooKeeper.</p>
</div>
</div>
<div class="sect3">
<h4 id="troubleshooting_kerberos"><a class="anchor" href="administration-guide.html#troubleshooting_kerberos"></a>Troubleshooting Kerberos Configuration</h4>
<div class="paragraph">
<p>When using Kerberos, it is import to use fully-qualified domain names and not use <em>localhost</em>. Please ensure that the fully qualified hostname of each server is used
in the following locations:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><em>conf/zookeeper.properties</em> file should use FQDN for <code>server.1</code>, <code>server.2</code>, &#8230;&#8203;, <code>server.N</code> values.</p>
</li>
<li>
<p>The <code>Connect String</code> property of the ZooKeeperStateProvider</p>
</li>
<li>
<p>The <em>/etc/hosts</em> file should also resolve the FQDN to an IP address that is <strong>not</strong> <code>127.0.0.1</code>.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Failure to do so, may result in errors similar to the following:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>2016-01-08 16:08:57,888 ERROR [pool-26-thread-1-SendThread(localhost:2181)] o.a.zookeeper.client.ZooKeeperSaslClient An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating ZooKeeper Quorum Member's received SASL token. ZooKeeper Client will go to AUTH_FAILED state.</code></pre>
</div>
</div>
<div class="paragraph">
<p>If there are problems communicating or authenticating with Kerberos, this
<a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html" target="_blank" rel="noopener">Troubleshooting Guide</a> may be of value.</p>
</div>
<div class="paragraph">
<p>One of the most important notes in the above Troubleshooting guide is the mechanism for turning on Debug output for Kerberos.
This is done by setting the <code>sun.security.krb5.debug</code> environment variable.
In NiFi, this is accomplished by adding the following line to the <em>$NIFI_HOME/conf/bootstrap.conf</em> file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>java.arg.16=-Dsun.security.krb5.debug=true</code></pre>
</div>
</div>
<div class="paragraph">
<p>This will cause the debug output to be written to the NiFi Bootstrap log file. By default, this is located at <em>$NIFI_HOME/logs/nifi-bootstrap.log</em>.
This output can be rather verbose but provides extremely valuable information for troubleshooting Kerberos failures.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="zk_tls_client"><a class="anchor" href="administration-guide.html#zk_tls_client"></a>Securing ZooKeeper with TLS</h3>
<div class="paragraph">
<p>As discussed above, communications with ZooKeeper are insecure by default. The second option for securely authenticating to and communicating with ZooKeeper is to use
certificate-based authentication with a TLS-enabled ZooKeeper server (available since ZooKeeper&#8217;s 3.5.x releases). Instructions for enabling TLS on an external
ZooKeeper ensemble can be found in the <a href="https://zookeeper.apache.org/doc/r3.5.5/zookeeperAdmin.html#sc_authOptions">ZooKeeper Administrator&#8217;s Guide</a>.</p>
</div>
<div class="paragraph">
<p>Once you have a TLS-enabled instance of ZooKeeper, TLS can be enabled for the NiFi client by setting <code>nifi.zookeeper.client.secure=true</code>. By default, the ZooKeeper client will use the existing <code>nifi.security.*</code> properties for the keystore and truststore. If you require separate TLS configuration for ZooKeeper, you can create a separate keystore and truststore and configure the following properties
in the <em>$NIFI_HOME/conf/nifi.properties</em> file:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Description</th>
<th class="tableblock halign-left valign-top">Default</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.client.secure</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to acccess ZooKeeper using client TLS.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">false</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.keystore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filename of the Keystore containing the private key to use when communicating with ZooKeeper.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.keystoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. The type of the Keystore. Must be <code>PKCS12</code>, <code>JKS</code>, or <code>PEM</code>. If not specified the type will be determined from the file extension (<code>.p12</code>, <code>.jks</code>, <code>.pem</code>).</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.keystorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Keystore.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.truststore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filename of the Truststore that will be used to verify the ZooKeeper server(s).</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.truststoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. The type of the Truststore. Must be <code>PKCS12</code>, <code>JKS</code>, or <code>PEM</code>. If not specified the type will be determined from the file extension (<code>.p12</code>, <code>.jks</code>, <code>.pem</code>).</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.truststorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Truststore.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>none</em></p></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>Whether using the default security properties or the ZooKeeper specific properties, the keystore and truststores must contain the appropriate keys and certificates for use with ZooKeeper (i.e., the keys and certificates need to align with the ZooKeeper configuration either way).
NiFi&#8217;s TLS Toolkit can be used to help generate the keystore and truststore used for ZooKeeper client/server access.</p>
</div>
<div class="paragraph">
<p>After updating the above properties and starting NiFi, network communication with ZooKeeper will be secure and ZooKeeper will now use the NiFi node&#8217;s certificate principal
when authenticating access. This will be reflected in log messages like the following on the ZooKeeper server:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>2020-02-24 23:37:52,671 [myid:2] - INFO [nioEventLoopGroup-4-1:X509AuthenticationProvider@172] - Authenticated Id 'CN=nifi-node1,OU=NIFI' for Scheme 'x509'</code></pre>
</div>
</div>
<div class="paragraph">
<p>ZooKeeper uses Netty to support network encryption and certificate-based authentication. When TLS is enabled, both the ZooKeeper server and its clients must be configured to use Netty-based
connections instead of the default NIO implementations. This is configured automatically for NiFi when <code>nifi.zookeeper.client.secure</code> is set to
<em>true</em>. Once Netty is enabled, you should see log messages like the following in <em>$NIFI_HOME/logs/nifi-app.log</em>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>2020-02-24 23:37:54,082 INFO [nioEventLoopGroup-3-1] o.apache.zookeeper.ClientCnxnSocketNetty SSL handler added for channel: [id: 0xa831f9c3]
2020-02-24 23:37:54,104 INFO [nioEventLoopGroup-3-1] o.apache.zookeeper.ClientCnxnSocketNetty channel is connected: [id: 0xa831f9c3, L:/172.17.0.4:56510 - R:8e38869cd1d1/172.17.0.3:2281]</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="embedded-zookeeper-with-tls"><a class="anchor" href="administration-guide.html#embedded-zookeeper-with-tls"></a>Embedded ZooKeeper with TLS</h3>
<div class="paragraph">
<p>A NiFi cluster can be deployed using a ZooKeeper instance(s) embedded in NiFi itself which all nodes can communicate with. As of NiFi 1.13.0, communication between nodes and this embedded ZooKeeper can now be secured with TLS. Versions of NiFi prior to 1.13 did not use secure client access with embedded ZooKeeper(s). The configuration for the client side of the connection will operate in the same way as an external ZooKeeper. That is, it will use the <code>nifi.security.*</code> properties from the nifi.properties file by default, unless you specifiy explicit ZooKeeper keystore/truststore properties with <code>nifi.zookeeper.security.*</code> as described above.</p>
</div>
<div class="paragraph">
<p>The server configuration will operate in the same way as an insecure embedded server, but with the <code>secureClientPort</code> set (typically port <code>2281</code>).</p>
</div>
<div class="paragraph">
<p>Example <em>$NIFI_HOME/conf/zookeeper.properties</em> file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>secureClientPort=2281
initLimit=10
autopurge.purgeInterval=24
syncLimit=5
tickTime=2000
dataDir=./state/zookeeper
autopurge.snapRetainCount=30
server.1=nifi1.example.com:2888:3888
server.2=nifi2.example.com:2888:3888
server.3=nifi3.example.com:2888:3888</code></pre>
</div>
</div>
<div class="paragraph">
<p>When used with a three node NiFi cluster, the above configuration file would establish a three node ZooKeeper quorum with each node listening on secure port 2281 for client connections with NiFi, 2888 for quorum communication and 3888 for leader election.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
When using a secure server, the secure embedded ZooKeeper server ignores any clientPort or clientPortAddress specified in <em>$NIFI_HOME/conf/zookeeper.properties</em>. I.e., if the NiFi-embedded ZooKeeper exposes a secureClientPort it will not expose an insecure clientPort regardless of configuration. This is a behavioral difference between the embedded server and an external ZooKeeper server and ensures the embedded ZooKeeper will either run securely, or insecurely, but not a mixture of both.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The following is an example of the relevant properties to set in <em>$NIFI_HOME/conf/nifi.properties</em> to run and connect to this quorum:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>nifi.security.keystore=./conf/keystore.jks
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=password
nifi.security.keyPasswd=password
nifi.security.truststore=./conf/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=password
nifi.security.user.authorizer=managed-authorizer
nifi.zookeeper.connect.string=nifi1.example.com:2281,nifi2.example.com:2281,nifi3.example.com:2281
nifi.zookeeper.connect.timeout=10 secs
nifi.zookeeper.session.timeout=10 secs
nifi.zookeeper.root.node=/nifi
nifi.zookeeper.client.secure=true
nifi.state.management.embedded.zookeeper.start=true
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
nifi.state.management.configuration.file=./conf/state-management.xml
nifi.state.management.provider.cluster=zk-provider</code></pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="zookeeper_migrator"><a class="anchor" href="administration-guide.html#zookeeper_migrator"></a>ZooKeeper Migrator</h3>
<div class="paragraph">
<p>You can use the <code>zk-migrator</code> tool to perform the following tasks:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Moving ZooKeeper information from one ZooKeeper cluster to another</p>
</li>
<li>
<p>Migrating ZooKeeper node ownership</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For example, you may want to use the ZooKeeper Migrator when you are:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Upgrading from NiFi 0.x to NiFi 1.x in which embedded ZooKeepers are used</p>
</li>
<li>
<p>Migrating from an embedded ZooKeeper in NiFi 0.x or 1.x to an external ZooKeeper</p>
</li>
<li>
<p>Upgrading from NiFi 0.x with an external ZooKeeper to NiFi 1.x with the same external ZooKeeper</p>
</li>
<li>
<p>Migrating from an external ZooKeeper to an embedded ZooKeeper in NiFi 1.x</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For more information, see the <a href="toolkit-guide.html#zookeeper_migrator">ZooKeeper Migrator</a> section in the <a href="toolkit-guide.html">NiFi Toolkit Guide</a>.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="bootstrap_properties"><a class="anchor" href="administration-guide.html#bootstrap_properties"></a>Bootstrap Properties</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The <em>bootstrap.conf</em> file in the <code>conf</code> directory allows users to configure settings for how NiFi should be started.
This includes parameters, such as the size of the Java Heap, what Java command to run, and Java System Properties.</p>
</div>
<div class="paragraph">
<p>Here, we will address the different properties that are made available in the file. Any changes to this file will
take effect only after NiFi has been stopped and restarted.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>java</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the fully qualified java command to run. By default, it is simply <code>java</code> but could be changed to an absolute path or a reference an environment variable, such as <code>$JAVA_HOME/bin/java</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>run.as</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The username to run NiFi as. For instance, if NiFi should be run as the <code>nifi</code> user, setting this value to <code>nifi</code> will cause the NiFi Process to be run as the <code>nifi</code> user.
This property is ignored on Windows. For Linux, the specified user may require sudo permissions.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>preserve.environment</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether or not to preserve shell environment while using <code>run.as</code> (see "sudo -E" man page). By default, this is set to false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>lib.dir</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The <em>lib</em> directory to use for NiFi. By default, this is set to <code>./lib</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>conf.dir</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The <em>conf</em> directory to use for NiFi. By default, this is set to <code>./conf</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>graceful.shutdown.seconds</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When NiFi is instructed to shutdown, the Bootstrap will wait this number of seconds for the process to shutdown cleanly. At this amount of time,
if the service is still running, the Bootstrap will <code>kill</code> the process, or terminate it abruptly.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>java.arg.N</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Any number of JVM arguments can be passed to the NiFi JVM when the process is started. These arguments are defined by adding properties to <em>bootstrap.conf</em> that
begin with <code>java.arg.</code>. The rest of the property name is not relevant, other than to differentiate property names, and will be ignored. The default includes
properties for minimum and maximum Java Heap size, the garbage collector to use, Java IO temporary directory, etc.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.bootstrap.sensitive.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The root key (in hexadecimal format) for encrypted sensitive configuration values. When NiFi is started, this root key is used to decrypt sensitive values from the <em>nifi.properties</em> file into memory for later use.</p>
<p class="tableblock">The Encrypt-Config Tool can be used to specify the root key, encrypt sensitive values in <em>nifi.properties</em> and update <em>bootstrap.conf</em>. See the <a href="toolkit-guide.html#encrypt_config_tool">NiFi Toolkit Guide</a> for an example.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>notification.services.file</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When NiFi is started, or stopped, or when the Bootstrap detects that NiFi has died, the Bootstrap is able to send notifications of these events
to interested parties. This is configured by specifying an XML file that defines which notification services can be used. More about this
file can be found in the <a href="administration-guide.html#notification_services">Notification Services</a> section.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>notification.max.attempts</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If a notification service is configured but is unable to perform its function, it will try again up to a maximum number of attempts. This property
configures what that maximum number of attempts is. The default value is <code>5</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.start.notification.services</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property is a comma-separated list of Notification Service identifiers that correspond to the Notification Services
defined in the <code>notification.services.file</code> property. The services with the specified identifiers will be used to notify their
configured recipients whenever NiFi is started.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.stop.notification.services</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property is a comma-separated list of Notification Service identifiers that correspond to the Notification Services
defined in the <code>notification.services.file</code> property. The services with the specified identifiers will be used to notify their
configured recipients whenever NiFi is stopped.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.died.notification.services</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property is a comma-separated list of Notification Service identifiers that correspond to the Notification Services
defined in the <code>notification.services.file</code> property. The services with the specified identifiers will be used to notify their
configured recipients if the bootstrap determines that NiFi has unexpectedly died.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.diagnostics.on.shutdown.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">(true or false) This property decides whether to run NiFi diagnostics before shutting down. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.diagnostics.on.shutdown.verbose</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">(true or false) This property decides whether to run NiFi diagnostics in verbose mode. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.diagnostics.on.shutdown.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property specifies the location of the NiFi diagnostics directory. The default value is <code>./diagnostics</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.diagnostics.on.shutdown.max.filecount</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property specifies the maximum permitted number of diagnostic files. If the limit is exceeded, the oldest files are deleted. The default value is <code>10</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.diagnostics.on.shutdown.max.directory.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property specifies the maximum permitted size of the diagnostics directory. If the limit is exceeded, the oldest files are deleted. The default value is <code>10 MB</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.bootstrap.listen.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property defines the port used to listen for communications from NiFi. If this property is missing, empty, or <code>0</code>, a random ephemeral port is used.</p></td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="sect1">
<h2 id="notification_services"><a class="anchor" href="administration-guide.html#notification_services"></a>Notification Services</h2>
<div class="sectionbody">
<div class="paragraph">
<p>When the NiFi bootstrap starts or stops NiFi, or detects that it has died unexpectedly, it is able to notify configured recipients. Currently,
the only mechanisms supplied are to send an e-mail or HTTP POST notification. The notification services configuration file
is an XML file where the notification capabilities are configured.</p>
</div>
<div class="paragraph">
<p>The default location of the XML file is <em>conf/bootstrap-notification-services.xml</em>, but this value can be changed in the <em>conf/bootstrap.conf</em> file.</p>
</div>
<div class="paragraph">
<p>The syntax of the XML file is as follows:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>&lt;services&gt;
&lt;!-- any number of service elements can be defined. --&gt;
&lt;service&gt;
&lt;id&gt;some-identifier&lt;/id&gt;
&lt;!-- The fully-qualified class name of the Notification Service. --&gt;
&lt;class&gt;org.apache.nifi.bootstrap.notification.email.EmailNotificationService&lt;/class&gt;
&lt;!-- Any number of properties can be set using this syntax.
The properties available depend on the Notification Service. --&gt;
&lt;property name="Property Name 1"&gt;Property Value&lt;/property&gt;
&lt;property name="Another Property Name"&gt;Property Value 2&lt;/property&gt;
&lt;/service&gt;
&lt;/services&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>Once the desired services have been configured, they can then be referenced in the <em>bootstrap.conf</em> file.</p>
</div>
<div class="sect2">
<h3 id="email-notification-service"><a class="anchor" href="administration-guide.html#email-notification-service"></a>Email Notification Service<br></h3>
<div class="paragraph">
<p>The first Notifier is to send emails and the implementation is <code>org.apache.nifi.bootstrap.notification.email.EmailNotificationService</code>.
It has the following properties available:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Required</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP Hostname</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The hostname of the SMTP Server that is used to send Email Notifications</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP Port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Port used for SMTP communications</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP Username</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Username for the SMTP account</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP Password</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Password for the SMTP account</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP Auth</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Flag indicating whether authentication should be used</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP TLS</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Flag indicating whether TLS should be enabled</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP Socket Factory</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>javax.net.ssl.SSLSocketFactory</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SMTP X-Mailer Header</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">X-Mailer used in the header of the outgoing email</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Content Type</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Mime Type used to interpret the contents of the email, such as <code>text/plain</code> or <code>text/html</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>From</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the Email address to use as the sender. Otherwise, a "friendly name" can be used as the From address, but the value
must be enclosed in double-quotes.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>To</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The recipients to include in the To-Line of the email</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>CC</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The recipients to include in the CC-Line of the email</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>BCC</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The recipients to include in the BCC-Line of the email</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>In addition to the properties above that are marked as required, at least one of the <code>To</code>, <code>CC</code>, or <code>BCC</code> properties
must be set.</p>
</div>
<div class="paragraph">
<p>A complete example of configuring the Email service would look like the following:</p>
</div>
<div class="literalblock">
<div class="content">
<pre> &lt;service&gt;
&lt;id&gt;email-notification&lt;/id&gt;
&lt;class&gt;org.apache.nifi.bootstrap.notification.email.EmailNotificationService&lt;/class&gt;
&lt;property name="SMTP Hostname"&gt;smtp.gmail.com&lt;/property&gt;
&lt;property name="SMTP Port"&gt;587&lt;/property&gt;
&lt;property name="SMTP Username"&gt;username@gmail.com&lt;/property&gt;
&lt;property name="SMTP Password"&gt;super-secret-password&lt;/property&gt;
&lt;property name="SMTP TLS"&gt;true&lt;/property&gt;
&lt;property name="From"&gt;"NiFi Service Notifier"&lt;/property&gt;
&lt;property name="To"&gt;username@gmail.com&lt;/property&gt;
&lt;/service&gt;</pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="http-notification-service"><a class="anchor" href="administration-guide.html#http-notification-service"></a>HTTP Notification Service<br></h3>
<div class="paragraph">
<p>The second Notifier is to send HTTP POST requests and the implementation is <code>org.apache.nifi.bootstrap.notification.http.HttpNotificationService</code>.
It has the following properties available:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Required</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>URL</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL to send the notification to. Expression language is supported.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Connection timeout</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Max wait time for connection to remote service. Expression language is supported. This defaults to <code>10s</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Write timeout</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Max wait time for remote service to read the request sent. Expression language is supported. This defaults to <code>10s</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Truststore Filename</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The fully-qualified filename of the Truststore</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Truststore Type</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Type of the Truststore. Either <code>JKS</code> or <code>PKCS12</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Truststore Password</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Truststore</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Keystore Filename</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The fully-qualified filename of the Keystore</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Keystore Type</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Type of the Keystore. Either <code>JKS</code> or <code>PKCS12</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Keystore Password</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Keystore</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Key Password</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the key. If this is not specified, but the Keystore Filename, Password, and Type are specified, then the Key Password will be assumed to be the same as the Keystore Password.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>SSL Protocol</code></p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The algorithm to use for this SSL context. This can either be <code>SSL</code> or <code>TLS</code>.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>In addition to the properties above, dynamic properties can be added. They will be added as headers to the HTTP request. Expression language is supported.</p>
</div>
<div class="paragraph">
<p>The notification message is in the body of the POST request. The type of notification is in the header "notification.type" and the subject uses the header "notification.subject".</p>
</div>
<div class="paragraph">
<p>A complete example of configuring the HTTP service could look like the following:</p>
</div>
<div class="literalblock">
<div class="content">
<pre> &lt;service&gt;
&lt;id&gt;http-notification&lt;/id&gt;
&lt;class&gt;org.apache.nifi.bootstrap.notification.http.HttpNotificationService&lt;/class&gt;
&lt;property name="URL"&gt;https://testServer.com:8080/&lt;/property&gt;
&lt;property name="Truststore Filename"&gt;localhost-ts.jks&lt;/property&gt;
&lt;property name="Truststore Type"&gt;JKS&lt;/property&gt;
&lt;property name="Truststore Password"&gt;localtest&lt;property&gt;
&lt;property name="Keystore Filename"&gt;localhost-ts.jks&lt;/property&gt;
&lt;property name="Keystore Type"&gt;JKS&lt;/property&gt;
&lt;property name="Keystore Password"&gt;localtest&lt;/property&gt;
&lt;property name="notification.timestamp"&gt;${now()}&lt;/property&gt;
&lt;/service&gt;</pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="proxy_configuration"><a class="anchor" href="administration-guide.html#proxy_configuration"></a>Proxy Configuration</h2>
<div class="sectionbody">
<div class="paragraph">
<p>When running Apache NiFi behind a proxy there are a couple of key items to be aware of during deployment.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>NiFi is comprised of a number of web applications (web UI, web API, documentation, custom UIs, data viewers, etc), so the mapping needs to be configured for the <strong>root path</strong>. That way all context
paths are passed through accordingly. For instance, if only the <code>/nifi</code> context path was mapped, the custom UI for UpdateAttribute will not work, since it is available at <code>/update-attribute-ui-&lt;version&gt;</code>.</p>
</li>
<li>
<p>NiFi&#8217;s REST API will generate URIs for each component on the graph. Since requests are coming through a proxy, certain elements of the URIs being generated need to be overridden. Without
overriding, the users will be able to view the dataflow on the canvas but will be unable to modify existing components. Requests will be attempting to call back directly to NiFi, not through the
proxy. The elements of the URI can be overridden by adding the following HTTP headers when the proxy generates the HTTP request to the NiFi instance:</p>
</li>
</ul>
</div>
<div class="literalblock">
<div class="content">
<pre>X-ProxyScheme - the scheme to use to connect to the proxy
X-ProxyHost - the host of the proxy
X-ProxyPort - the port the proxy is listening on
X-ProxyContextPath - the path configured to map to the NiFi instance</pre>
</div>
</div>
<div class="ulist">
<ul>
<li>
<p>If NiFi is running securely, any proxy needs to be authorized to proxy user requests. These can be configured in the NiFi UI through the Global Menu. Once these permissions are in place, proxies
can begin proxying user requests. The end user identity must be relayed in a HTTP header. For example, if the end user sent a request to the proxy, the proxy must authenticate the user. Following
this the proxy can send the request to NiFi. In this request an HTTP header should be added as follows.</p>
</li>
</ul>
</div>
<div class="literalblock">
<div class="content">
<pre>X-ProxiedEntitiesChain: &lt;end-user-identity&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>If the proxy is configured to send to another proxy, the request to NiFi from the second proxy should contain a header as follows.</p>
</div>
<div class="literalblock">
<div class="content">
<pre>X-ProxiedEntitiesChain: &lt;end-user-identity&gt;&lt;proxy-1-identity&gt;</pre>
</div>
</div>
<div class="paragraph">
<p>An example Apache proxy configuration that sets the required properties may look like the following. Complete proxy configuration is outside of the scope of this document. Please refer the
documentation of the proxy for guidance for your deployment environment and use case.</p>
</div>
<div class="literalblock">
<div class="content">
<pre>...
&lt;Location "/my-nifi"&gt;
...
SSLEngine On
SSLCertificateFile /path/to/proxy/certificate.crt
SSLCertificateKeyFile /path/to/proxy/key.key
SSLCACertificateFile /path/to/ca/certificate.crt
SSLVerifyClient require
RequestHeader add X-ProxyScheme "https"
RequestHeader add X-ProxyHost "proxy-host"
RequestHeader add X-ProxyPort "443"
RequestHeader add X-ProxyContextPath "/my-nifi"
RequestHeader add X-ProxiedEntitiesChain "&lt;%{SSL_CLIENT_S_DN}&gt;"
ProxyPass https://nifi-host:8443
ProxyPassReverse https://nifi-host:8443
...
&lt;/Location&gt;
...</pre>
</div>
</div>
<div class="ulist">
<ul>
<li>
<p>Additional NiFi proxy configuration must be updated to allow expected Host and context paths HTTP headers.</p>
<div class="ulist">
<ul>
<li>
<p>By default, if NiFi is running securely it will only accept HTTP requests with a Host header matching the host[:port] that it is bound to. If NiFi is to accept requests directed to a different
host[:port] the expected values need to be configured. This may be required when running behind a proxy or in a containerized environment. This is configured in a comma
separated list in <em>nifi.properties</em> using the <code>nifi.web.proxy.host</code> property (e.g. <code>localhost:18443, proxyhost:443</code>). IPv6 addresses are accepted. Please refer to
RFC 5952 Sections <a href="https://tools.ietf.org/html/rfc5952#section-4">4</a> and <a href="https://tools.ietf.org/html/rfc5952#section-6">6</a> for additional details.</p>
</li>
<li>
<p>NiFi will only accept HTTP requests with a X-ProxyContextPath, X-Forwarded-Context, or X-Forwarded-Prefix header if the value is allowed in the <code>nifi.web.proxy.context.path</code> property in
<em>nifi.properties</em>. This property accepts a comma separated list of expected values. In the event an incoming request has an X-ProxyContextPath, X-Forwarded-Context, or X-Forwarded-Prefix header value that is not
present in the allow list, the "An unexpected error has occurred" page will be shown and an error will be written to the <em>nifi-app.log</em>.</p>
</li>
</ul>
</div>
</li>
<li>
<p>Additional configurations at both proxy server and NiFi cluster are required to make NiFi Site-to-Site work behind reverse proxies. See <a href="administration-guide.html#site_to_site_reverse_proxy_properties">Site to Site Routing Properties for Reverse Proxies</a> for details.</p>
<div class="ulist">
<ul>
<li>
<p>In order to transfer data via Site-to-Site protocol through reverse proxies, both proxy and Site-to-Site client NiFi users need to have following policies, 'retrieve site-to-site details', 'receive data via site-to-site' for input ports, and 'send data via site-to-site' for output ports.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="session_affinity"><a class="anchor" href="administration-guide.html#session_affinity"></a>Session Affinity</h3>
<div class="paragraph">
<p>All HTTP requests from a single client must be routed to the same Apache NiFi node for the duration of an authenticated
session. This applies to both browser-based users and programmatic clients accessing the REST API. This is not a concern
for standalone deployments or direct network access to Apache NiFi, but accessing clustered nodes through a proxy server
or load balancer requires enabling session affinity, also known as sticky sessions. Session affinity is required for
mediated access to traditional cluster deployments as well as containerized deployments using platforms such as
Kubernetes.</p>
</div>
<div class="paragraph">
<p>Access to clustered deployments through a gateway requires session affinity for the following reasons:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Each node uses a local key for signing and verifying JSON Web Tokens</p>
</li>
<li>
<p>Each node uses a local cache for tracking configuration change transactions</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Attempting to access a clustered node through a gateway without session affinity will result in intermittent failures of
various types. When authenticating to Apache NiFi with username and password credentials, the lack of session affinity
often results in HTTP 401 Unauthorized responses, indicating that the node did not accept the JSON Web Token. These
failures can occur at different times based on the load balancing strategy. Accessing Apache NiFi using an X.509
certificate avoids the verification issues associated with JSON Web Tokens, but is still subject to problems related to
configuration change transaction handling across cluster nodes.</p>
</div>
<div class="sect3">
<h4 id="session-affinity-configuration"><a class="anchor" href="administration-guide.html#session-affinity-configuration"></a>Session Affinity Configuration</h4>
<div class="paragraph">
<p>Enabling session affinity requires different settings depending on the product or service providing access. It is
essential that the session affinity configuration has a timeout that is greater than the session expiration when
authenticating with username and password credentials.</p>
</div>
<div class="sect4">
<h5 id="apache-http-server-configuration"><a class="anchor" href="administration-guide.html#apache-http-server-configuration"></a>Apache HTTP Server Configuration</h5>
<div class="paragraph">
<p><a href="https://httpd.apache.org">Apache HTTP Server</a> supports session affinity in the
<a href="https://httpd.apache.org/docs/2.4/mod/mod_proxy.html">mod_proxy</a> module using the
<a href="https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass">ProxyPass</a> directive with the
<a href="https://httpd.apache.org/docs/2.4/mod/mod_proxy_balancer.html#stickyness_implementation">stickysession</a> parameter to
configure a cookie name for request routing.</p>
</div>
</div>
<div class="sect4">
<h5 id="kubernetes-nginx-ingress-controller"><a class="anchor" href="administration-guide.html#kubernetes-nginx-ingress-controller"></a>Kubernetes Nginx Ingress Controller</h5>
<div class="paragraph">
<p>The <a href="https://kubernetes.github.io/ingress-nginx/">Kubernetes Nginx Ingress Controller</a>
supports session affinity using deployment annotations to configure
<a href="https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/">sticky sessions</a> with cookies. The deployment
annotations provide the ability to configure cookie attributes, including expiration.</p>
</div>
</div>
<div class="sect4">
<h5 id="nginx-configuration"><a class="anchor" href="administration-guide.html#nginx-configuration"></a>Nginx Configuration</h5>
<div class="paragraph">
<p><a href="https://nginx.org/">Nginx</a> supports session affinity in the upstream module using the
<a href="https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky">sticky</a> directive. The <strong>sticky</strong> directive
supports different strategies, including <strong>cookie</strong> and <strong>route</strong> options.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="kerberos_service"><a class="anchor" href="administration-guide.html#kerberos_service"></a>Kerberos Service</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi can be configured to use Kerberos SPNEGO (or "Kerberos Service") for authentication. In this scenario, users will hit the REST endpoint <code>/access/kerberos</code> and the server will respond with a <code>401</code> status code and the challenge response header <code>WWW-Authenticate: Negotiate</code>. This communicates to the browser to use the GSS-API and load the user&#8217;s Kerberos ticket and provide it as a Base64-encoded header value in the subsequent request. It will be of the form <code>Authorization: Negotiate YII&#8230;&#8203;</code>. NiFi will attempt to validate this ticket with the KDC. If it is successful, the user&#8217;s <em>principal</em> will be returned as the identity, and the flow will follow login/credential authentication, in that a JWT will be issued in the response to prevent the unnecessary overhead of Kerberos authentication on every subsequent request. If the ticket cannot be validated, it will return with the appropriate error response code. The user will then be able to provide their Kerberos credentials to the login form if the <code>KerberosLoginIdentityProvider</code> has been configured. See <a href="administration-guide.html#kerberos_login_identity_provider">Kerberos</a> login identity provider for more details.</p>
</div>
<div class="paragraph">
<p>NiFi will only respond to Kerberos SPNEGO negotiation over an HTTPS connection, as unsecured requests are never authenticated.</p>
</div>
<div class="paragraph">
<p>The following properties must be set in <em>nifi.properties</em> to enable Kerberos service authentication.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Required</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Service Principal</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The service principal used by NiFi to communicate with the KDC</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>Keytab Location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">true</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The file path to the keytab containing the service principal</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>See <a href="administration-guide.html#kerberos_properties">Kerberos Properties</a> for complete documentation.</p>
</div>
<div class="sect2">
<h3 id="kerberos_service_notes"><a class="anchor" href="administration-guide.html#kerberos_service_notes"></a>Notes</h3>
<div class="ulist">
<ul>
<li>
<p>Kerberos is case-sensitive in many places and the error messages (or lack thereof) may not be sufficiently explanatory. Check the case sensitivity of the service principal in your configuration files. Convention is <code>HTTP/fully.qualified.domain@REALM</code>.</p>
</li>
<li>
<p>Browsers have varying levels of restriction when dealing with SPNEGO negotiations. Some will provide the local Kerberos ticket to any domain that requests it, while others explicitly specify the trusted domains in advance via an allow list. See <a href="http://docs.spring.io/autorepo/docs/spring-security-kerberos/1.0.2.BUILD-SNAPSHOT/reference/htmlsingle/#browserspnegoconfig" target="_blank" rel="noopener">Spring Security Kerberos - Reference Documentation: Appendix E. Configure browsers for SPNEGO Negotiation</a> for common browsers.</p>
</li>
<li>
<p>Some browsers (legacy IE) do not support recent encryption algorithms such as AES, and are restricted to legacy algorithms (DES). This should be noted when generating keytabs.</p>
</li>
<li>
<p>The KDC must be configured and a service principal defined for NiFi and a keytab exported. Comprehensive instructions for Kerberos server configuration and administration are beyond the scope of this document (see <a href="http://web.mit.edu/kerberos/krb5-current/doc/admin/index.html" target="_blank" rel="noopener">MIT Kerberos Admin Guide</a>), but an example is below:</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Adding a service principal for a server at <code>nifi.nifi.apache.org</code> and exporting the keytab from the KDC:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>root@kdc:/etc/krb5kdc# kadmin.local
Authenticating as principal admin/admin@NIFI.APACHE.ORG with password.
kadmin.local: listprincs
K/M@NIFI.APACHE.ORG
admin/admin@NIFI.APACHE.ORG
...
kadmin.local: addprinc -randkey HTTP/nifi.nifi.apache.org
WARNING: no policy specified for HTTP/nifi.nifi.apache.org@NIFI.APACHE.ORG; defaulting to no policy
Principal "HTTP/nifi.nifi.apache.org@NIFI.APACHE.ORG" created.
kadmin.local: ktadd -k /http-nifi.keytab HTTP/nifi.nifi.apache.org
Entry for principal HTTP/nifi.nifi.apache.org with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:/http-nifi.keytab.
Entry for principal HTTP/nifi.nifi.apache.org with kvno 2, encryption type des-cbc-crc added to keytab WRFILE:/http-nifi.keytab.
kadmin.local: listprincs
HTTP/nifi.nifi.apache.org@NIFI.APACHE.ORG
K/M@NIFI.APACHE.ORG
admin/admin@NIFI.APACHE.ORG
...
kadmin.local: q
root@kdc:~# ll /http*
-rw------- 1 root root 162 Mar 14 21:43 /http-nifi.keytab
root@kdc:~#</pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="analytics_framework"><a class="anchor" href="administration-guide.html#analytics_framework"></a>Analytics Framework</h2>
<div class="sectionbody">
<div class="paragraph">
<p>NiFi has an internal analytics framework which can be enabled to predict back pressure occurrence, given the configured settings for threshold on a queue. The model used by default for prediction is an ordinary least squares (OLS) linear regression. It uses recent observations from a queue (either number of objects or content size over time) and calculates a regression line for that data. The line&#8217;s equation is then used to determine the next value that will be reached within a given time interval (e.g. number of objects in queue in the next 5 minutes). Below is an example graph of the linear regression model for Queue/Object Count over time which is used for predictions:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="images/back_pressure_prediction_model_example.png" alt="Back pressure prediction based on Queue/Object Count"></span></p>
</div>
<div class="paragraph">
<p>In order to generate predictions, local status snapshot history is queried to obtain enough data to generate a model. By default, component status snapshots are captured every minute. Internal models need at least 2 or more observations to generate a prediction, therefore it may take up to 2 or more minutes for predictions to be available by default. If predictions are needed sooner than what is provided by default, the timing of snapshots can be adjusted using the <code>nifi.components.status.snapshot.frequency</code> value in <em>nifi.properties</em>.</p>
</div>
<div class="paragraph">
<p>NiFi evaluates the model&#8217;s effectiveness before sending prediction information by using the model&#8217;s R-Squared score by default. One important note: R-Square is a measure of how close the regression line fits the observation data vs. how accurate the prediction will be; therefore there may be some measure of error. If the R-Squared score for the calculated model meets the configured threshold (as defined by <code>nifi.analytics.connection.model.score.threshold</code>) then the model will be used for prediction. Otherwise the model will not be used and predictions will not be available until a model is generated with a score that exceeds the threshold. Default R-Squared threshold value is <code>.90</code> however this can be tuned based on prediction requirements.</p>
</div>
<div class="paragraph">
<p>The prediction interval <code>nifi.analytics.predict.interval</code> can be configured to project out further when back pressure will occur. The prediction query interval <code>nifi.analytics.query.interval</code> can also be configured to determine how far back in time past observations should be queried in order to generate the model. Adjustments to these settings may require tuning of the model&#8217;s scoring threshold value to select a score that can offer reasonable predictions.</p>
</div>
<div class="paragraph">
<p>See <a href="administration-guide.html#analytics_properties">Analytics Properties</a> for complete information on configuring analytic properties.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="system_properties"><a class="anchor" href="administration-guide.html#system_properties"></a>System Properties</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The <em>nifi.properties</em> file in the <code>conf</code> directory is the main configuration file for controlling how NiFi runs. This section provides an overview of the properties in this file and their setting options.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Values for periods of time and data sizes must include the unit of measure, for example "10 secs" or "10 MB", not simply "10".
</td>
</tr>
</table>
</div>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
After making changes to <em>nifi.properties</em>, restart NiFi in order for the changes to take effect.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="upgrade-recommendations"><a class="anchor" href="administration-guide.html#upgrade-recommendations"></a>Upgrade Recommendations</h3>
<div class="paragraph">
<p>The contents of the <em>nifi.properties</em> file are relatively stable but can change from version to version. It is always a good idea to review this file when upgrading and pay attention to any changes.</p>
</div>
<div class="paragraph">
<p>Consider configuring items below marked with an asterisk (<code>*</code>) in such a way that upgrading will be easier. For example, change the default directory configurations to locations outside the main root installation. In this way, these items can remain in their configured location through an upgrade, allowing NiFi to find all the repositories and configuration files and pick up where it left off as soon as the old version is stopped and the new version is started. Furthermore, the administrator may reuse this <em>nifi.properties</em> file and any other configuration files without having to re-configure them each time an upgrade takes place. See <a href="administration-guide.html#upgrading_nifi">Upgrading NiFi</a> for more details.</p>
</div>
</div>
<div class="sect2">
<h3 id="core-properties"><a class="anchor" href="administration-guide.html#core-properties"></a>Core Properties<br></h3>
<div class="paragraph">
<p>The first section of the <em>nifi.properties</em> file is for the Core Properties. These properties apply to the core framework as a whole.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.file</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the XML-based flow configuration file. The default value is <code>./conf/flow.xml.gz</code>. This is a legacy property. Older versions of NiFi used an
XML-formatted file to store the flow configuration. However, newer versions use a JSON representation. In order to maintain backward compatibility of flows and still load flows developed using
older versions of NiFi, upon startup, NiFi will use the <code>nifi.flow.configuration.json.file</code> first. If the file exists, it will be used. However, if it does not exist, NiFi will fall back to this
property to determine the XML version of the file and use it.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.json.file</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the flow configuration file (i.e., the file that contains what is currently displayed on the NiFi graph). The default value is <code>./conf/flow.json.gz</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.archive.enabled</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether NiFi creates a backup copy of the flow automatically when the flow is updated. The default value is <code>true</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.archive.dir</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the archive directory where backup copies of the <em>flow.json</em> are saved. The default value is <code>./conf/archive</code>. NiFi removes old archive files to limit disk usage based on archived file lifespan, total size, and number of files, as specified with <code>nifi.flow.configuration.archive.max.time</code>, <code>max.storage</code> and <code>max.count</code> properties respectively. If none of these limitation for archiving is specified, NiFi uses default conditions, that is <code>30 days</code> for <code>max.time</code> and <code>500 MB</code> for <code>max.storage</code>.<br>
This cleanup mechanism takes into account only automatically created archived <em>flow.json</em> files. If there are other files or directories in this archive directory, NiFi will ignore them. Automatically created archives have filename with ISO 8601 format timestamp prefix followed by <code>&lt;original-filename&gt;</code>. That is <code>&lt;year&gt;&lt;month&gt;&lt;day&gt;T&lt;hour&gt;&lt;minute&gt;&lt;second&gt;+&lt;timezone offset&gt;_&lt;original filename&gt;</code>. For example, <code>20160706T160719+0900_flow.json.gz</code>. NiFi checks filenames when it cleans archive directory. If you would like to keep a particular archive in this directory without worrying about NiFi deleting it, you can do so by copying it with a different filename pattern.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.archive.max.time</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The lifespan of archived <em>flow.json</em> files. NiFi will delete expired archive files when it updates <em>flow.json</em> if this property is specified. Expiration is determined based on current system time and the last modified timestamp of an archived <em>flow.json</em>. If no archive limitation is specified in <em>nifi.properties</em>, NiFi removes archives older than <code>30 days</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.archive.max.storage</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The total data size allowed for the archived <em>flow.json</em> files. NiFi will delete the oldest archive files until the total archived file size becomes less than this configuration value, if this property is specified. If no archive limitation is specified in <em>nifi.properties</em>, NiFi uses <code>500 MB</code> for this.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.archive.max.count</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of archive files allowed. NiFi will delete the oldest archive files so that only N latest archives can be kept, if this property is specified.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowcontroller.autoResumeState</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates whether -upon restart- the components on the NiFi graph should return to their last state. The default value is <code>true</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowcontroller.graceful.shutdown.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates the shutdown period. The default value is <code>10 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowservice.writedelay.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When many changes are made to the <em>flow.json</em>, this property specifies how long to wait before writing out the changes, so as to batch the changes into a single write. The default value is <code>500 ms</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.administrative.yield.duration</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If a component allows an unexpected exception to escape, it is considered a bug. As a result, the framework will pause (or administratively yield) the component for this amount of time. This is done so that the component does not use up massive amounts of system resources, since it is known to have problems in the existing state. The default value is <code>30 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.bored.yield.duration</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When a component has no work to do (i.e., is "bored"), this is the amount of time it will wait before checking to see if it has new data to work on. This way, it does not use up CPU resources by checking for new work too often. When setting this property, be aware that it could add extra latency for components that do not constantly have work to do, as once they go into this "bored" state, they will wait this amount of time before checking for more work. The default value is <code>10 ms</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.queue.backpressure.count</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When drawing a new connection between two components, this is the default value for that connection&#8217;s back pressure object threshold. The default is <code>10000</code> and the value must be an integer.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.queue.backpressure.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When drawing a new connection between two components, this is the default value for that connection&#8217;s back pressure data size threshold. The default is <code>1 GB</code> and the value must be a data size including the unit of measure.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.authorizer.configuration.file</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the location of the file that specifies how authorizers are defined. The default value is <code>./conf/authorizers.xml</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.login.identity.provider.configuration.file</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the location of the file that specifies how username/password authentication is performed. This file is
only considered if <code>nifi.security.user.login.identity.provider</code> is configured with a provider identifier. The default value is <code>./conf/login-identity-providers.xml</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.templates.directory</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the location of the directory where flow templates are saved (for backward compatibility only). Templates are stored in the <em>flow.json.gz</em> starting with NiFi 1.0. The template directory can be used to (bulk) import templates into the <em>flow.json.gz</em> automatically on NiFi startup. The default value is <code>./conf/templates</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.ui.banner.text</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is banner text that may be configured to display at the top of the User Interface. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.ui.autorefresh.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The interval at which the User Interface auto-refreshes. The default value is <code>30 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the nar library. The default value is <code>./lib</code> and probably should be left as is.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.restore.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location that certain providers (e.g. UserGroupProviders) will look for previous configurations to restore from. There is no default value.
<br>
<strong>NOTE</strong>: Additional library directories can be specified by using the <code>nifi.nar.library.directory.</code> prefix with unique suffixes and separate paths as values.<br>
<br>
For example, to provide two additional library locations, a user could also specify additional properties with keys of:<br>
<br>
<code>nifi.nar.library.directory.lib1=/nars/lib1</code><br>
<code>nifi.nar.library.directory.lib2=/nars/lib2</code><br>
<br>
Providing three total locations, including <code>nifi.nar.library.directory</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.working.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the nar working directory. The default value is <code>./work/nar</code> and probably should be left as is.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.documentation.working.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The documentation working directory. The default value is <code>./work/docs/components</code> and probably should be left as is.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.unpack.uber.jar</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to <code>true</code>, when a nar file is unpacked, the inner jar files will be unpacked into a single jar file instead of individual jar files. This can result in NiFi taking
longer to startup for the first time (about 1-2 minutes, typically) but can result in far fewer open file handles, which can be helpful in certain environments. The default value is <code>false</code>. This
feature is considered experimental. Changing the value of this property may not take effect unless the working directory is also deleted.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.processor.scheduling.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Time to wait for a Processor&#8217;s life-cycle operation (<code>@OnScheduled</code> and <code>@OnUnscheduled</code>) to finish before other life-cycle operation (e.g., <strong>stop</strong>) could be invoked. The default value is <code>1 min</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="state-management"><a class="anchor" href="administration-guide.html#state-management"></a>State Management<br></h3>
<div class="paragraph">
<p>The State Management section of the Properties file provides a mechanism for configuring local and cluster-wide mechanisms
for components to persist state. See the <a href="administration-guide.html#state_management">State Management</a> section for more information on how this is used.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.configuration.file</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The XML file that contains configuration for the local and cluster-wide State Providers. The default value is <code>./conf/state-management.xml</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.provider.local</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The ID of the Local State Provider to use. This value must match the value of the <code>id</code> element of one of the <code>local-provider</code> elements in the <em>state-management.xml</em> file.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.provider.cluster</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The ID of the Cluster State Provider to use. This value must match the value of the <code>id</code> element of one of the <code>cluster-provider</code> elements in the <em>state-management.xml</em> file. This value is ignored if not clustered but is required for nodes in a cluster.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.embedded.zookeeper.start</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether or not this instance of NiFi should start an embedded ZooKeeper Server. This is used in conjunction with the ZooKeeperStateProvider. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.state.management.embedded.zookeeper.properties</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies a properties file that contains the configuration for the embedded ZooKeeper Server that is started (if the <code>nifi.state.management.embedded.zookeeper.start</code> property is set to <code>true</code>). The default value is <code>./conf/zookeeper.properties</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="h2-settings"><a class="anchor" href="administration-guide.html#h2-settings"></a>H2 Settings</h3>
<div class="paragraph">
<p>The H2 Settings section defines the settings for the H2 database, which keeps track of user access and flow controller history.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.database.directory</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the H2 database directory. The default value is <code>./database_repository</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.h2.url.append</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property specifies additional arguments to add to the connection string for the H2 database. The default value should be used and should not be changed. It is: <code>;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="repository-encryption-properties"><a class="anchor" href="administration-guide.html#repository-encryption-properties"></a>Repository Encryption</h3>
<div class="paragraph">
<p>Repository encryption provides a layer of security for information persisted to the filesystem during processing.
Enabling encryption and configuring a Key Provider using these properties applies to all repositories.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.repository.encryption.protocol.version</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The encryption protocol version applied to all repository implementations. Absence of this property value disables repository encryption. Configuring a supported protocol enables encryption for all repositories. Supported protocol versions include: <code>1</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.repository.encryption.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The key identifier that repository implementations will use for new encryption operations. The key identifier must match the <code>alias</code> value for a Key Entry when using the <code>KEYSTORE</code> provider.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.repository.encryption.key.provider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Key Provider implementation that repository implementations will use for retrieving keys necessary for encryption and decryption. Supported providers include: <code>KEYSTORE</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.repository.encryption.key.provider.keystore.location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Path to the KeyStore resource required for the <code>KEYSTORE</code> provider to read available keys. The KeyStore must contain one or more Secret Key entries. File paths must end with a known extension. Supported KeyStore types include: <code>PKCS12</code> and <code>BCFKS</code>. Supported extensions include: <code>.p12</code> and <code>.bcfks</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.repository.encryption.key.provider.keystore.password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Password for the configured KeyStore resource required for the <code>KEYSTORE</code> provider to decrypt available keys. The configured KeyStore must use the same password for both the KeyStore and individual Key Entries.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Configuring repository encryption properties overrides the following repository implementation class properties, as well
as associated Key Provider properties:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>nifi.content.repository.implementation</code></p>
</li>
<li>
<p><code>nifi.flowfile.repository.wal.implementation</code></p>
</li>
<li>
<p><code>nifi.provenance.repository.implementation</code></p>
</li>
<li>
<p><code>nifi.swap.manager.implementation</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The following provides an example set of configuration properties using a PKCS12 KeyStore as the Key Provider:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.repository.encryption.protocol.version=1
nifi.repository.encryption.key.id=primary-key
nifi.repository.encryption.key.provider=KEYSTORE
nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
nifi.repository.encryption.key.provider.keystore.password=2fRKmwDyMYmT7P5L</pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="flowfile-repository"><a class="anchor" href="administration-guide.html#flowfile-repository"></a>FlowFile Repository</h3>
<div class="paragraph">
<p>The FlowFile repository keeps track of the attributes and current state of each FlowFile in the system. By default,
this repository is installed in the same root installation directory as all the other repositories; however, it is advisable
to configure it on a separate drive if available.</p>
</div>
<div class="paragraph">
<p>There are currently three implementations of the FlowFile Repository, which are detailed below.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The FlowFile Repository implementation. The default value is <code>org.apache.nifi.controller.repository.WriteAheadFlowFileRepository</code>. The other current options are <code>org.apache.nifi.controller.repository.VolatileFlowFileRepository</code> and <code>org.apache.nifi.controller.repository.RocksDBFlowFileRepository</code>.</p></td>
</tr>
</tbody>
</table>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Switching repository implementations should only be done on an instance with zero queued FlowFiles, and should only be done with caution.
</td>
</tr>
</table>
</div>
</div>
<div class="sect2">
<h3 id="write-ahead-flowfile-repository"><a class="anchor" href="administration-guide.html#write-ahead-flowfile-repository"></a>Write Ahead FlowFile Repository</h3>
<div class="paragraph">
<p><code>WriteAheadFlowFileRepository</code> is the default implementation. It persists FlowFiles to disk, and can optionally be configured to synchronize all changes to disk. This is very expensive and can significantly reduce NiFi performance. However, if it is <code>false</code>, there could be the potential for data loss if either there is a sudden power loss or the operating system crashes. The default value is <code>false</code>.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.wal.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If the repository implementation is configured to use the <code>WriteAheadFlowFileRepository</code>, this property can be used to specify which implementation of the
Write-Ahead Log should be used. The default value is <code>org.apache.nifi.wali.SequentialAccessWriteAheadLog</code>. This version of the write-ahead log was added in version 1.6.0 of Apache NiFi and was developed
in order to address an issue that exists in the older implementation. In the event of power loss or an operating system crash, the old implementation was susceptible to recovering FlowFiles
incorrectly. This could potentially lead to the wrong attributes or content being assigned to a FlowFile upon restart, following the power loss or OS crash. However, one can still choose to opt into
using the previous implementation and accept that risk, if desired (for example, if the new implementation were to exhibit some unexpected error).
To do so, set the value of this property to <code>org.wali.MinimalLockingWriteAheadLog</code>. Another available implementation is <code>org.apache.nifi.wali.EncryptedSequentialAccessWriteAheadLog</code>.
If the value of this property is changed, upon restart, NiFi will still recover the records written using the previously configured repository and delete the files written by the previously configured
implementation.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.directory</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the FlowFile Repository. The default value is <code>./flowfile_repository</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.checkpoint.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The FlowFile Repository checkpoint interval. The default value is <code>20 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.always.sync</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to <code>true</code>, any change to the repository will be synchronized to the disk, meaning that NiFi will ask the operating system not to cache the information. This is very expensive and can significantly reduce NiFi performance. However, if it is <code>false</code>, there could be the potential for data loss if either there is a sudden power loss or the operating system crashes. The default value is <code>false</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="encrypted-write-ahead-flowfile-repository-properties"><a class="anchor" href="administration-guide.html#encrypted-write-ahead-flowfile-repository-properties"></a>Encrypted Write Ahead FlowFile Repository Properties</h3>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
The following properties are deprecated in favor of <a href="administration-guide.html#repository-encryption-properties">Repository Encryption</a> properties.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>All of the properties defined above (see <a href="administration-guide.html#write-ahead-flowfile-repository">Write Ahead FlowFile Repository</a>) still apply. Only encryption-specific properties are listed here. See <a href="user-guide.html#encrypted-flowfile">Encrypted FlowFile Repository in the User Guide</a> for more information.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Unlike the encrypted content and provenance repositories, the repository implementation does not change here, only the <em>underlying write-ahead log implementation</em>. This allows for cleaner separation and more flexibility in implementation selection. The property that should be changed to enable encryption is <code>nifi.flowfile.repository.wal.implementation</code>.
</td>
</tr>
</table>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.encryption.key.provider.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the fully-qualified class name of the <strong>key provider</strong>. A key provider is the datastore interface for accessing the encryption key to protect the content claims. There are currently three implementations: <code>StaticKeyProvider</code> which reads a key directly from <em>nifi.properties</em>, <code>FileBasedKeyProvider</code> which reads keys from an encrypted file, and <code>KeyStoreKeyProvider</code> which reads keys from a standard <code>java.security.KeyStore</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.encryption.key.provider.location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The path to the key definition resource (empty for <code>StaticKeyProvider</code>, <code>./keys.nkp</code> or similar path for <code>FileBasedKeyProvider</code>). For future providers like an HSM, this may be a connection string or URL.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.encryption.key.provider.password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password used for decrypting the key definition resource, such as the keystore for <code>KeyStoreKeyProvider</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.encryption.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The active key ID to use for encryption (e.g. <code>Key1</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.encryption.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The key to use for <code>StaticKeyProvider</code>. The key format is hex-encoded (<code>0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210</code>) but can also be encrypted using the <code>./encrypt-config.sh</code> tool in NiFi Toolkit (see the <a href="toolkit-guide.html#encrypt_config_tool">Encrypt-Config Tool</a> section in the <a href="toolkit-guide.html">NiFi Toolkit Guide</a> for more information).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.encryption.key.id.</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows for additional keys to be specified for the <code>StaticKeyProvider</code>. For example, the line <code>nifi.flowfile.repository.encryption.key.id.Key2=012&#8230;&#8203;210</code> would provide an available key <code>Key2</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="volatile-flowfile-repository"><a class="anchor" href="administration-guide.html#volatile-flowfile-repository"></a>Volatile FlowFile Repository</h3>
<div class="paragraph">
<p>This implementation stores FlowFiles in memory instead of on disk. It <strong>will</strong> result in data loss in the event of power/machine failure or a restart of NiFi. To use this implementation, set <code>nifi.flowfile.repository.implementation</code> to <code>org.apache.nifi.controller.repository.VolatileFlowFileRepository</code>.</p>
</div>
</div>
<div class="sect2">
<h3 id="rocksdb-flowfile-repository"><a class="anchor" href="administration-guide.html#rocksdb-flowfile-repository"></a>RocksDB FlowFile Repository</h3>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
The <code>RocksDBFlowFileRepository</code> is deprecated in favor of the <code>WriteAheadFlowFileRepository</code>. It will be removed in future major versions of Apache NiFi. To use it in NiFi 1.x, you must download nifi-rocksdb-nar from the Apache Nexus Repository and place it in your NiFi lib directory.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>This implementation makes use of the RocksDB key-value store. It uses periodic synchronization to ensure that no created or received data is lost (as long as <code>nifi.flowfile.repository.rocksdb.accept.data.loss</code> is set <code>false</code>). In the event of a failure (e.g. power loss), <em>work</em> done on FlowFiles through the system (i.e. routing and transformation) may still be lost. Specifically, the record of these actions may be lost, reverting the affected FlowFiles to a previous, valid state. From there, they will resume their path through the flow as normal. This guarantee comes at the expense of a delay on operations that add new data to the system. This delay is configurable (as <code>nifi.flowfile.repository.rocksdb.sync.period</code>), and can be tuned to the individual system.</p>
</div>
<div class="paragraph">
<p>The configuration parameters for this repository fall in to two categories, "NiFi-centric" and "RocksDB-centric". The NiFi-centric settings have to do with the operations of the FlowFile Repository and its interaction with NiFi. The RocksDB-centric settings directly correlate to settings on the underlying RocksDB repo. More information on these settings can be found in the RocksDB documentation: <a href="https://github.com/facebook/rocksdb/wiki/RocksJava-Basics" class="bare">https://github.com/facebook/rocksdb/wiki/RocksJava-Basics</a>.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Windows users will need to ensure "Microsoft Visual C++ 2015 Redistributable" is installed for this repository to work. See the following link for more details: <a href="https://github.com/facebook/rocksdb/wiki/RocksJava-Basics#maven-windows" class="bare">https://github.com/facebook/rocksdb/wiki/RocksJava-Basics#maven-windows</a>.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>To use this implementation, set <code>nifi.flowfile.repository.implementation</code> to <code>org.apache.nifi.controller.repository.RocksDBFlowFileRepository</code>.</p>
</div>
<div class="paragraph">
<p><strong>NiFi-centric Configuration Properties</strong>:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the FlowFile Repository.
The default value is`./flowfile_repository`.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.sync.warning.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">How often to log warnings if unable to sync.
The default value is 30 seconds.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.claim.cleanup.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">How often to mark content claims destructible (so they can be removed from the content repo).
The default value is 30 seconds.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.deserialization.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">How many threads to use on startup restoring the FlowFile state.
The default value is 16.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.deserialization.buffer.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Size of the buffer to use on startup restoring the FlowFile state.
The default value is 1000.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.sync.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Frequency at which to force a sync to disk. This is the maximum period a data creation operation may block if <code>nifi.flowfile.repository.rocksdb.accept.data.loss</code> is <code>false</code>.
The default value is 10 milliseconds.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.accept.data.loss</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to accept the loss of received / created data. Setting this <code>true</code> increases throughput if loss of data is acceptable.
The default value is false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.enable.stall.stop</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to enable the stall / stop of writes to the repository based on configured limits. Enabling this feature allows the system to protect itself by restricting (delaying or denying) operations that increase the total FlowFile count on the node to prevent the system from being overwhelmed.
The default value is false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.stall.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The period of time to stall when the specified criteria are encountered.
The default value is 100 milliseconds.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.stall.flowfile.count</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The FlowFile count at which to begin stalling writes to the repo.
The default value is 800000.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.stall.heap.usage.percent</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The heap usage at which to begin stalling writes to the repo.
The default value is 95%.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.stop.flowfile.count</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The FlowFile count at which to begin stopping the creation of new FlowFiles.
The default value is 1100000.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.stop.heap.usage.percent</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The heap usage at which to begin stopping the creation of new FlowFiles.
The default value is 99.9%.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.remove.orphaned.flowfiles.on.startup</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to allow the repository to remove FlowFiles it cannot identify on startup. As this is often the result of a configuration or synchronization error, it is disabled by default. This should only be enabled if you are absolutely certain you want to lose the data in question.
The default value is false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.enable.recovery.mode</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to enable "recovery mode". This limits the number of FlowFiles loaded into the graph at a time, while not actually removing any FlowFiles (or content) from the system. This allows for the recovery of a system that is encountering OutOfMemory errors or similar on startup. This should not be enabled unless necessary to recover a system, and should be disabled as soon as that has been accomplished.</p>
<p class="tableblock"><strong>WARNING:</strong> While in recovery mode, <strong>do not</strong> make modifications to the graph. Changes to the graph may result in the inability to restore further FlowFiles from the repository.
The default value is false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.recovery.mode.flowfile.count</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of FlowFiles to load into the graph when in "recovery mode". As FlowFiles leave the system, additional FlowFiles will be loaded up to this limit. This setting does not prevent FlowFiles from coming into the system via normal means.
The default value is 5000.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p><strong>RocksDB-centric Configuration Properties:</strong></p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.parallel.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of threads to use for flush and compaction. A good value is the number of cores. See RockDB <code>DBOptions.setIncreaseParallelism()</code> for more information.
The default value is 8.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.max.write.buffer.number</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of write buffers that are built up in memory. See RockDB <code>ColumnFamilyOptions.setMaxWriteBufferNumber()</code> / <code>max_write_buffer_number</code> for more information.
The default value is 4.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.write.buffer.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The amount of data to build up in memory before converting to a sorted on disk file. Larger values increase performance, especially during bulk loads. Up to <code>max_write_buffer_number</code> write buffers may be held in memory at the same time, so you may wish to adjust this parameter to control memory usage. See RockDB <code>ColumnFamilyOptions.setWriteBufferSize()</code> / <code>write_buffer_size</code> for more information.
The default value is 256 MB.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.level.0.slowdown.writes.trigger</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A soft limit on number of level-0 files. Writes are slowed at this point. A values less than 0 means no write slow down will be triggered by the number of files in level-0. See RocksDB <code>ColumnFamilyOptions.setLevel0SlowdownWritesTrigger()</code> / <code>level0_slowdown_writes_trigger</code> for more information.
The default value is 20.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.level.0.stop.writes.trigger</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of level-0 files. Writes will be stopped at this point. See RocksDB <code>ColumnFamilyOptions.setLevel0StopWritesTrigger()</code> / <code>level0_stop_writes_trigger</code> for more information.
The default value is 40.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.delayed.write.bytes.per.second</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The limited write rate to the DB if slowdown is triggered. RocksDB may decide to slow down more if the compaction gets behind further. See RocksDB <code>DBOptions.setDelayedWriteRate()</code> for more information.
The default value is 16 MB.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.max.background.flushes</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the maximum number of concurrent background flush jobs. See RocksDB <code>DBOptions.setMaxBackgroundFlushes()</code> / <code>max_background_flushes</code> for more information.
The default value is 1.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.max.background.compactions</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the maximum number of concurrent background compaction jobs. See RocksDB <code>DBOptions.setMaxBackgroundCompactions()</code> / <code>max_background_compactions</code> for more information.
The default value is 1.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.min.write.buffer.number.to.merge</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The minimum number of write buffers to merge together before writing to storage. See RocksDB <code>ColumnFamilyOptions.setMinWriteBufferNumberToMerge()</code> / <code>min_write_buffer_number_to_merge</code> for more information.
The default value is 1.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.rocksdb.stat.dump.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The period at which to dump rocksdb.stats to the log. See RocksDB <code>DBOptions.setStatsDumpPeriodSec()</code> / <code>stats_dump_period_sec</code> for more information.
The default value is 600 sec.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="swap-management"><a class="anchor" href="administration-guide.html#swap-management"></a>Swap Management</h3>
<div class="paragraph">
<p>NiFi keeps FlowFile information in memory (the JVM)
but during surges of incoming data, the FlowFile information can start to take up so much of the JVM that system performance
suffers. To counteract this effect, NiFi "swaps" the FlowFile information to disk temporarily until more JVM space becomes
available again. These properties govern how that process occurs.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.swap.manager.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Swap Manager implementation. The default value is <code>org.apache.nifi.controller.FileSystemSwapManager</code>.
There is an alternate implementation, <code>EncryptedFileSystemSwapManager</code>, that encrypts the swap file content on
disk. The encryption key configured for the FlowFile repository is used to perform the encryption, using the AES-GCM algorithm.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.queue.swap.threshold</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The queue threshold at which NiFi starts to swap FlowFile information to disk. The default value is <code>20000</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="content-repository"><a class="anchor" href="administration-guide.html#content-repository"></a>Content Repository</h3>
<div class="paragraph">
<p>The Content Repository holds the content for all the FlowFiles in the system. By default, it is installed in the same root
installation directory as all the other repositories; however, administrators will likely want to configure it on a separate
drive if available. If nothing else, it is best if the Content Repository is not on the same drive as the FlowFile Repository.
In dataflows that handle a large amount of data, the Content Repository could fill up a disk and the
FlowFile Repository, if also on that disk, could become corrupt. To avoid this situation, configure these repositories on different drives.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Content Repository implementation. The default value is <code>org.apache.nifi.controller.repository.FileSystemRepository</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="file-system-content-repository-properties"><a class="anchor" href="administration-guide.html#file-system-content-repository-properties"></a>File System Content Repository Properties</h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Content Repository implementation. The default value is <code>org.apache.nifi.controller.repository.FileSystemRepository</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.claim.max.appendable.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When NiFi processes many small FlowFiles, the contents of those FlowFiles are stored in the content repository, but we do not store the content of each
individual FlowFile as a separate file in the content repository. Doing so would be very detrimental to performance, if each 120 byte FlowFile, for instance, was written to its own file. Instead,
we continue writing to the same file until it reaches some threshold. This property configures that threshold. Setting the value too small can result in poor performance due to reading from and
writing to too many files. However, a file can only be deleted from the content repository once there are no longer any FlowFiles pointing to it. Therefore, setting the value too large can result
in data remaining in the content repository for much longer, potentially leading to the content repository running out of disk space. The default value is <code>50 KB</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.directory.default</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the Content Repository. The default value is <code>./content_repository</code>.<br>
+
<strong>NOTE</strong>: Multiple content repositories can be specified by using the <code>nifi.content.repository.directory.</code> prefix with unique suffixes and separate paths as values.<br>
+
For example, to provide two additional locations to act as part of the content repository, a user could also specify additional properties with keys of:<br>
+
<code>nifi.content.repository.directory.content1=/repos/content1</code><br>
<code>nifi.content.repository.directory.content2=/repos/content2</code><br>
+
Providing three total locations, including <code>nifi.content.repository.directory.default</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.archive.max.retention.period</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If archiving is enabled (see <code>nifi.content.repository.archive.enabled</code> below), then
this property specifies the maximum amount of time to keep the archived data. The default value is <code>7 days</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.archive.max.usage.percentage</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If archiving is enabled (see <code>nifi.content.repository.archive.enabled</code> below), then this property must have a value that indicates the content repository disk usage percentage at which archived data begins to be removed. If the archive is empty and content repository disk usage is above this percentage, then archiving is temporarily disabled. Archiving will resume when disk usage is below this percentage. The default value is <code>50%</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.archive.backpressure.percentage</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property is used to control the content repository disk usage percentage at which backpressure is applied to the processes writing to the content repository. Once this percentage is reached, the content repository will refuse any additional writes. Writes will be refused until the archive delete process has brought the content repository disk usage percentage below <code>nifi.content.repository.archive.max.usage.percentage</code>.<br>
<br>
The value must be a valid percentage e.g. <code>60%</code><br>
<br>
For example, if <code>nifi.content.repository.archive.max.usage.percentage</code> is <code>50%</code> and <code>nifi.content.repository.archive.backpressure.percentage</code> is <code>60%</code>, then if the content repository reaches 60% utilisation of storage capacity, all further writes are blocked until utilisation is brought back down to 50%. <br>
<br>
When not set, the default value is derived as 2% greater than <code>nifi.content.repository.archive.max.usage.percentage</code>.<br>
<br>
For example, if <code>nifi.content.repository.archive.max.usage.percentage</code> is <code>50%</code> and <code>nifi.content.repository.archive.backpressure.percentage</code> is not set, the effective value of <code>nifi.content.repository.archive.backpressure.percentage</code> will be <code>52%</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.archive.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">To enable content archiving, set this to <code>true</code> and specify a value for the <code>nifi.content.repository.archive.max.usage.percentage</code> property above. Content archiving enables the provenance UI to view or replay content that is no longer in a dataflow queue. By default, archiving is enabled.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.always.sync</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to <code>true</code>, any change to the repository will be synchronized to the disk, meaning that NiFi will ask the operating system not to cache the information. This is very expensive and can significantly reduce NiFi performance. However, if it is <code>false</code>, there could be the potential for data loss if either there is a sudden power loss or the operating system crashes. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.viewer.url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL for a web-based content viewer if one is available. The default is <code>../nifi-content-viewer/</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.archive.cleanup.frequency</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The frequency with which to schedule the content archive clean up task. The default value is <code>1 Second</code>. A value lower than <code>1 Second</code> is not allowed.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="encrypted-file-system-content-repository-properties"><a class="anchor" href="administration-guide.html#encrypted-file-system-content-repository-properties"></a>Encrypted File System Content Repository Properties</h3>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
The following properties are deprecated in favor of <a href="administration-guide.html#repository-encryption-properties">Repository Encryption</a> properties.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>All of the properties defined above (see <a href="administration-guide.html#file-system-content-repository-properties">File System Content Repository Properties</a>) still apply. Only encryption-specific properties are listed here. See <a href="user-guide.html#encrypted-content">Encrypted Content Repository in the User Guide</a> for more information.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.encryption.key.provider.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the fully-qualified class name of the <strong>key provider</strong>. A key provider is the datastore interface for accessing the encryption key to protect the content claims. There are currently three implementations: <code>StaticKeyProvider</code> which reads a key directly from <em>nifi.properties</em>, <code>FileBasedKeyProvider</code> which reads keys from an encrypted file, and <code>KeyStoreKeyProvider</code> which reads keys from a standard <code>java.security.KeyStore</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.encryption.key.provider.location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The path to the key definition resource (empty for <code>StaticKeyProvider</code>, <code>./keys.nkp</code> or similar path for <code>FileBasedKeyProvider</code>). For future providers like an HSM, this may be a connection string or URL.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.encryption.key.provider.password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password used for decrypting the key definition resource, such as the keystore for <code>KeyStoreKeyProvider</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.encryption.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The active key ID to use for encryption (e.g. <code>Key1</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.encryption.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The key to use for <code>StaticKeyProvider</code>. The key format is hex-encoded (<code>0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210</code>) but can also be encrypted using the <code>./encrypt-config.sh</code> tool in NiFi Toolkit (see the <a href="toolkit-guide.html#encrypt_config_tool">Encrypt-Config Tool</a> section in the <a href="toolkit-guide.html">NiFi Toolkit Guide</a> for more information).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.encryption.key.id.</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows for additional keys to be specified for the <code>StaticKeyProvider</code>. For example, the line <code>nifi.content.repository.encryption.key.id.Key2=012&#8230;&#8203;210</code> would provide an available key <code>Key2</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="provenance-repository"><a class="anchor" href="administration-guide.html#provenance-repository"></a>Provenance Repository</h3>
<div class="paragraph">
<p>The Provenance Repository contains the information related to Data Provenance. The next four sections are for Provenance Repository properties.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Provenance Repository implementation. The default value is <code>org.apache.nifi.provenance.WriteAheadProvenanceRepository</code>.
Three additional repositories are available as well.
To store provenance events in memory instead of on disk (in which case all events will be lost on restart, and events will be evicted in a first-in-first-out order),
set this property to <code>org.apache.nifi.provenance.VolatileProvenanceRepository</code>. This leaves a configurable number of Provenance Events in the Java heap, so the number
of events that can be retained is very limited.</p>
<p class="tableblock">A third and fourth option are available: <code>org.apache.nifi.provenance.PersistentProvenanceRepository</code> and <code>org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository</code>.
The <code>PersistentProvenanceRepository</code> was originally written with the simple goal of persisting
Provenance Events as they are generated and providing the ability to iterate over those events sequentially. Later, it was desired to be able to compress the data so that
more data could be stored. After that, the ability to index and query the data was added. As requirements evolved over time, the repository kept changing without any major
redesigns. When used in a NiFi instance that is responsible for processing large volumes of small FlowFiles, the <code>PersistentProvenanceRepository</code> can quickly become a bottleneck.
The <code>WriteAheadProvenanceRepository</code> was then written to provide the same capabilities as the <code>PersistentProvenanceRepository</code> while providing far better performance.
The <code>WriteAheadProvenanceRepository</code> was added in version 1.2.0 of NiFi. Since then, it has proven to be very stable and robust and as such was made the default implementation.
The <code>PersistentProvenanceRepository</code> is now considered deprecated and should no longer be used. If administering an instance of NiFi that is currently using the
<code>PersistentProvenanceRepository</code>, it is highly recommended to upgrade to the <code>WriteAheadProvenanceRepository</code>. Doing so is as simple as changing the implementation property value
from <code>org.apache.nifi.provenance.PersistentProvenanceRepository</code> to <code>org.apache.nifi.provenance.WriteAheadProvenanceRepository</code>. Because the Provenance Repository is backward
compatible, there will be no loss of data or functionality.</p>
<p class="tableblock">The <code>EncryptedWriteAheadProvenanceRepository</code> builds upon the <code>WriteAheadProvenanceRepository</code> and ensures that data is encrypted at rest.</p>
<p class="tableblock"><strong>NOTE:</strong> The <code>WriteAheadProvenanceRepository</code> will make use of the Provenance data stored by the <code>PersistentProvenanceRepository</code>. However, the
<code>PersistentProvenanceRepository</code> may not be able to read the data written by the <code>WriteAheadProvenanceRepository</code>. Therefore, once the Provenance Repository is changed to use
the <code>WriteAheadProvenanceRepository</code>, it cannot be changed back to the <code>PersistentProvenanceRepository</code> without deleting the data in the Provenance Repository.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.rollover.events</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of events that should be written to a single event file before the file is rolled over. The default value is <code>Integer.MAX_VALUE</code></p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="write-ahead-provenance-repository-properties"><a class="anchor" href="administration-guide.html#write-ahead-provenance-repository-properties"></a>Write Ahead Provenance Repository Properties</h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.directory.default</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the Provenance Repository. The default value is <code>./provenance_repository</code>.<br>
+
<strong>NOTE</strong>: Multiple provenance repositories can be specified by using the <code>nifi.provenance.repository.directory.</code> prefix with unique suffixes and separate paths as values.<br>
+
For example, to provide two additional locations to act as part of the provenance repository, a user could also specify additional properties with keys of:<br>
+
<code>nifi.provenance.repository.directory.provenance1=/repos/provenance1</code><br>
<code>nifi.provenance.repository.directory.provenance2=/repos/provenance2</code><br>
+
Providing three total locations, including <code>nifi.provenance.repository.directory.default</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.max.storage.time</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum amount of time to keep data provenance information. The default value is <code>30 days</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.max.storage.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum amount of data provenance information to store at a time.
The default value is <code>10 GB</code>. The Data Provenance capability can consume a great deal of storage space because so much data is kept.
For production environments, values of 1-2 TB or more is not uncommon. The repository will write to a single "event file" (or set of
"event files" if multiple storage locations are defined, as described above) until the event file reaches the size defined in the <code>nifi.provenance.repository.rollover.size</code> property. It will then "roll over" and begin writing new events to a new file. Data is always aged off one file at a time, so it is not advisable to write a tremendous amount of data to a single "event file," as it will prevent old data from aging off as smoothly.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.rollover.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The amount of data to write to a single "event file." The default value is <code>100 MB</code>. For production
environments where a very large amount of Data Provenance is generated, a value of <code>1 GB</code> is also very reasonable.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.query.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of threads to use for Provenance Repository queries. The default value is <code>2</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.index.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of threads to use for indexing Provenance events so that they are searchable. The default value is <code>2</code>.
For flows that operate on a very high number of FlowFiles, the indexing of Provenance events could become a bottleneck. If this happens, increasing the
value of this property may increase the rate at which the Provenance Repository is able to process these records, resulting in better overall throughput.
It is advisable to use at least 1 thread per storage location (i.e., if there are 3 storage locations, at least 3 threads should be used). For high
throughput environments, where more CPU and disk I/O is available, it may make sense to increase this value significantly. Typically going beyond
2-4 threads per storage location is not valuable. However, this can be tuned depending on the CPU resources available compared to the I/O resources.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.compress.on.rollover</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates whether to compress the provenance information when an "event file" is rolled over. The default value is <code>true</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.always.sync</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to <code>true</code>, any change to the repository will be synchronized to the disk, meaning that NiFi will ask the operating system
not to cache the information. This is very expensive and can significantly reduce NiFi performance. However, if it is <code>false</code>, there could be the potential for data
loss if either there is a sudden power loss or the operating system crashes. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.indexed.fields</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is a comma-separated list of the fields that should be indexed and made searchable.
Fields that are not indexed will not be searchable. Valid fields are: <code>EventType</code>, <code>FlowFileUUID</code>, <code>Filename</code>, <code>TransitURI</code>, <code>ProcessorID</code>,
<code>AlternateIdentifierURI</code>, <code>Relationship</code>, <code>Details</code>. The default value is: <code>EventType, FlowFileUUID, Filename, ProcessorID</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.indexed.attributes</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is a comma-separated list of FlowFile Attributes that should be indexed and made searchable. It is blank by default.
But some good examples to consider are <code>filename</code> and <code>mime.type</code> as well as any custom attributes you might use which are valuable for your use case.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.index.shard.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The repository uses Apache Lucene to performing indexing and searching capabilities. This value indicates how large a Lucene Index should
become before the Repository starts writing to a new Index. Large values for the shard size will result in more Java heap usage when searching the Provenance Repository but should
provide better performance. The default value is <code>500 MB</code>. However, this is due to the fact that defaults are tuned for very small environments where most users begin to use NiFi.
For production environments, it is advisable to change this value to <code>4</code> to <code>8 GB</code>. Once all Provenance Events in the index have been aged off from the "event files," the index
will be destroyed as well.</p>
<p class="tableblock"> <strong>NOTE:</strong> This value should be smaller than (no more than half of) the <code>nifi.provenance.repository.max.storage.size</code> property.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.max.attribute.length</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from the repository.
If the length of any attribute exceeds this value, it will be truncated when the event is retrieved. The default value is <code>65536</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.concurrent.merge.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Apache Lucene creates several "segments" in an Index. These segments are periodically merged together in order to provide faster
querying. This property specifies the maximum number of threads that are allowed to be used for <strong>each</strong> of the storage directories. The default value is <code>2</code>. For high throughput
environments, it is advisable to set the number of index threads larger than the number of merge threads * the number of storage locations. For example, if there are 2 storage
locations and the number of index threads is set to <code>8</code>, then the number of merge threads should likely be less than <code>4</code>. While it is not critical that this be done, setting the
number of merge threads larger than this can result in all index threads being used to merge, which would cause the NiFi flow to periodically pause while indexing is happening,
resulting in some data being processed with much higher latency than other data.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.warm.cache.frequency</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Each time that a Provenance query is run, the query must first search the Apache Lucene indices (at least, in most cases - there are
some queries that are run often and the results are cached to avoid searching the Lucene indices). When a Lucene index is opened for the first time, it can be very expensive and take
several seconds. This is compounded by having many different indices, and can result in a Provenance query taking much longer. After the index has been opened, the Operating System&#8217;s
disk cache will typically hold onto enough data to make re-opening the index much faster - at least for a period of time, until the disk cache evicts this data. If this value is set,
NiFi will periodically open each Lucene index and then close it, in order to "warm" the cache. This will result in far faster queries when the Provenance Repository is large. As with
all great things, though, it comes with a cost. Warming the cache does take some CPU resources, but more importantly it will evict other data from the Operating System disk cache and
will result in reading (potentially a great deal of) data from the disk. This can result in lower NiFi performance. However, if NiFi is running in an environment where CPU and disk
are not fully utilized, this feature can result in far faster Provenance queries. The default value for this property is blank (i.e. disabled).</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="encrypted-write-ahead-provenance-repository-properties"><a class="anchor" href="administration-guide.html#encrypted-write-ahead-provenance-repository-properties"></a>Encrypted Write Ahead Provenance Repository Properties</h3>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
The following properties are deprecated in favor of <a href="administration-guide.html#repository-encryption-properties">Repository Encryption</a> properties.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>All of the properties defined above (see <a href="administration-guide.html#write-ahead-provenance-repository-properties">Write Ahead Repository Properties</a>) still apply. Only encryption-specific properties are listed here. See <a href="user-guide.html#encrypted-provenance">Encrypted Provenance Repository in the User Guide</a> for more information.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.encryption.key.provider.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the fully-qualified class name of the <strong>key provider</strong>. A key provider is the datastore interface for accessing the encryption key to protect the provenance events. There are currently three implementations: <code>StaticKeyProvider</code> which reads a key directly from <em>nifi.properties</em>, <code>FileBasedKeyProvider</code> which reads keys from an encrypted file, and <code>KeyStoreKeyProvider</code> which reads keys from a standard <code>java.security.KeyStore</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.encryption.key.provider.location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The path to the key definition resource (empty for <code>StaticKeyProvider</code>, <code>./keys.nkp</code> or similar path for <code>FileBasedKeyProvider</code>). For future providers like an HSM, this may be a connection string or URL.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.encryption.key.provider.password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password used for decrypting the key definition resource, such as the keystore for <code>KeyStoreKeyProvider</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.encryption.key.id</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The active key ID to use for encryption (e.g. <code>Key1</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.encryption.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The key to use for <code>StaticKeyProvider</code>. The key format is hex-encoded (<code>0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210</code>) but can also be encrypted using the <code>./encrypt-config.sh</code> tool in NiFi Toolkit (see the <a href="toolkit-guide.html#encrypt_config_tool">Encrypt-Config Tool</a> section in the <a href="toolkit-guide.html">NiFi Toolkit Guide</a> for more information).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.encryption.key.id.</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Allows for additional keys to be specified for the <code>StaticKeyProvider</code>. For example, the line <code>nifi.provenance.repository.encryption.key.id.Key2=012&#8230;&#8203;210</code> would provide an available key <code>Key2</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="persistent-provenance-repository-properties"><a class="anchor" href="administration-guide.html#persistent-provenance-repository-properties"></a>Persistent Provenance Repository Properties</h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.directory.default</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the Provenance Repository. The default value is <code>./provenance_repository</code>.<br>
+
<strong>NOTE</strong>: Multiple provenance repositories can be specified by using the <code>nifi.provenance.repository.directory.</code> prefix with unique suffixes and separate paths as values.<br>
+
For example, to provide two additional locations to act as part of the provenance repository, a user could also specify additional properties with keys of:<br>
+
<code>nifi.provenance.repository.directory.provenance1=/repos/provenance1</code><br>
<code>nifi.provenance.repository.directory.provenance2=/repos/provenance2</code><br>
+
Providing three total locations, including <code>nifi.provenance.repository.directory.default</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.max.storage.time</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum amount of time to keep data provenance information. The default value is <code>30 days</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.max.storage.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum amount of data provenance information to store at a time. The default value is <code>10 GB</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.rollover.time</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The amount of time to wait before rolling over the latest data provenance information so that it is available in the User Interface. The default value is <code>10 mins</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.rollover.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The amount of information to roll over at a time. The default value is <code>100 MB</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.query.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of threads to use for Provenance Repository queries. The default value is <code>2</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.index.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of threads to use for indexing Provenance events so that they are searchable. The default value is <code>2</code>.
For flows that operate on a very high number of FlowFiles, the indexing of Provenance events could become a bottleneck. If this is the case, a bulletin will appear, indicating that
"The rate of the dataflow is exceeding the provenance recording rate. Slowing down flow to accommodate." If this happens, increasing the value of this property
may increase the rate at which the Provenance Repository is able to process these records, resulting in better overall throughput.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.compress.on.rollover</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates whether to compress the provenance information when rolling it over. The default value is <code>true</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.always.sync</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to <code>true</code>, any change to the repository will be synchronized to the disk, meaning that NiFi will ask the operating system not to cache the information. This is very expensive and can significantly reduce NiFi performance. However, if it is <code>false</code>, there could be the potential for data loss if either there is a sudden power loss or the operating system crashes. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.journal.count</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of journal files that should be used to serialize Provenance Event data. Increasing this value will allow more tasks to simultaneously update the repository but will result in more expensive merging of the journal files later. This value should ideally be equal to the number of threads that are expected to update the repository simultaneously, but 16 tends to work well in must environments. The default value is <code>16</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.indexed.fields</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is a comma-separated list of the fields that should be indexed and made searchable. Fields that are not indexed will not be searchable. Valid fields are: <code>EventType</code>, <code>FlowFileUUID</code>, <code>Filename</code>, <code>TransitURI</code>, <code>ProcessorID</code>, <code>AlternateIdentifierURI</code>, <code>Relationship</code>, <code>Details</code>. The default value is: <code>EventType, FlowFileUUID, Filename, ProcessorID</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.indexed.attributes</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is a comma-separated list of FlowFile Attributes that should be indexed and made searchable. It is blank by default. But some good examples to consider are <code>filename</code>, <code>uuid</code>, and <code>mime.type</code> as well as any custom attritubes you might use which are valuable for your use case.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.index.shard.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Large values for the shard size will result in more Java heap usage when searching the Provenance Repository but should provide better performance. The default value is <code>500 MB</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.max.attribute.length</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved. The default value is <code>65536</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="volatile-provenance-repository-properties"><a class="anchor" href="administration-guide.html#volatile-provenance-repository-properties"></a>Volatile Provenance Repository Properties</h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.buffer.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Provenance Repository buffer size. The default value is <code>100000</code> provenance events.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="status-history-repository"><a class="anchor" href="administration-guide.html#status-history-repository"></a>Status History Repository</h3>
<div class="paragraph">
<p>The Status History Repository contains the information for the Component Status History and the Node Status History tools in
the User Interface. The following properties govern how these tools work.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.components.status.repository.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Status History Repository implementation. The default value is <code>org.apache.nifi.controller.status.history.VolatileComponentStatusRepository</code>,
which stores status history in memory. <code>org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepository</code> is also supported and stores status history information on disk so that it is
available across restarts and can be stored for much longer periods of time.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.components.status.snapshot.frequency</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This value indicates how often to capture a snapshot of the components' status history. The default value is <code>1 min</code>.</p></td>
</tr>
</tbody>
</table>
<div class="sect3">
<h4 id="in-memory-repository"><a class="anchor" href="administration-guide.html#in-memory-repository"></a>In memory repository</h4>
<div class="paragraph">
<p>If the value of the property <code>nifi.components.status.repository.implementation</code> is <code>VolatileComponentStatusRepository</code>, the
status history data will be stored in memory. If the application stops, all gathered information will be lost.</p>
</div>
<div class="paragraph">
<p>The <code>buffer.size</code> and <code>snapshot.frequency</code> work together to determine the amount of historical data to retain. As an example, to
configure two days' worth of historical data with a data point snapshot occurring every 5 minutes you would configure
<code>snapshot.frequency</code> to be "5 mins" and the buffer.size to be "576". To further explain this example, for every 60 minutes there
are 12 (60 / 5) snapshot windows for that time period. To keep that data for 48 hours (12 * 48) you end up with a buffer size
of 576.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.components.status.repository.buffer.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the buffer size for the Status History Repository. The default value is <code>1440</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect3">
<h4 id="persistent-repository"><a class="anchor" href="administration-guide.html#persistent-repository"></a>Persistent repository</h4>
<div class="paragraph">
<p>If the value of the property <code>nifi.components.status.repository.implementation</code> is <code>EmbeddedQuestDbStatusHistoryRepository</code>, the
status history data will be stored to the disk in a persistent manner. Data will be kept between restarts.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.status.repository.questdb.persist.node.days</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of days the node status data (such as Repository disk space free, garbage collection information, etc.) will be kept. The default values
is <code>14</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.status.repository.questdb.persist.component.days</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of days the component status data (i.e., stats for each Processor, Connection, etc.) will be kept. The default value is <code>3</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.status.repository.questdb.persist.location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the persistent Status History Repository. The default value is <code>./status_repository</code>.</p></td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="sect2">
<h3 id="site_to_site_properties"><a class="anchor" href="administration-guide.html#site_to_site_properties"></a>Site to Site Properties</h3>
<div class="paragraph">
<p>These properties govern how this instance of NiFi communicates with remote instances of NiFi when Remote Process Groups are configured in the dataflow.
Remote Process Groups can choose transport protocol from RAW and HTTP. Properties named with <code>nifi.remote.input.socket.*</code> are RAW transport protocol specific. Similarly, <code>nifi.remote.input.http.*</code> are HTTP transport protocol specific properties.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.input.host</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The host name that will be given out to clients to connect to this NiFi instance for Site-to-Site communication. By default, it is the value from <code>InetAddress.getLocalHost().getHostName()</code>. On UNIX-like operating systems, this is typically the output from the <code>hostname</code> command.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.input.secure</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This indicates whether communication between this instance of NiFi and remote NiFi instances should be secure (i.e., secure site-to-site). By default, it is set to <code>true</code>. Many other <a href="administration-guide.html#security_properties">Security Properties</a> must also be configured.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.input.socket.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The remote input socket port for Site-to-Site communication. By default, it is blank, but it must have a value in order to use RAW socket as transport protocol for Site-to-Site.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.input.http.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether HTTP Site-to-Site should be enabled on this host. By default, it is set to <code>true</code>.<br>
Whether a Site-to-Site client uses HTTP or HTTPS is determined by <code>nifi.remote.input.secure</code>. If it is set to <code>true</code>, then requests are sent as HTTPS to <code>nifi.web.https.port</code>. If set to <code>false</code>, HTTP requests are sent to <code>nifi.web.http.port</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.input.http.transaction.ttl</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies how long a transaction can stay alive on the server. By default, it is set to <code>30 secs</code>.<br>
If a Site-to-Site client hasn’t proceeded to the next action after this period of time, the transaction is discarded from the remote NiFi instance. For example, when a client creates a transaction but doesn’t send or receive flow files, or when a client sends or receives flow files but doesn’t confirm that transaction.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.contents.cache.expiration</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies how long NiFi should cache information about a remote NiFi instance when communicating via Site-to-Site. By default, NiFi will cache the<br>
responses from the remote system for <code>30 secs</code>. This allows NiFi to avoid constantly making HTTP requests to the remote system, which is particularly important when this instance of NiFi<br>
has many instances of Remote Process Groups.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="site_to_site_reverse_proxy_properties"><a class="anchor" href="administration-guide.html#site_to_site_reverse_proxy_properties"></a>Site to Site Routing Properties for Reverse Proxies</h3>
<div class="paragraph">
<p>Site-to-Site requires peer-to-peer communication between a client and a remote NiFi node. E.g. if a remote NiFi cluster has 3 nodes (<code>nifi0</code>, <code>nifi1</code> and <code>nifi2</code>) then client requests have to be reachable to each of those remote nodes.</p>
</div>
<div class="paragraph">
<p>If a NiFi cluster is planned to receive/transfer data from/to Site-to-Site clients over the internet or a company firewall, a reverse proxy server can be deployed in front of the NiFi cluster nodes as a gateway to route client requests to upstream NiFi nodes, to reduce number of servers and ports those have to be exposed.</p>
</div>
<div class="paragraph">
<p>In such environment, the same NiFi cluster would also be expected to be accessed by Site-to-Site clients within the same network. Sending FlowFiles to itself for load distribution among NiFi cluster nodes can be a typical example. In this case, client requests should be routed directly to a node without going through the reverse proxy.</p>
</div>
<div class="paragraph">
<p>In order to support such deployments, remote NiFi clusters need to expose its Site-to-Site endpoints dynamically based on client request contexts. Following properties configure how peers should be exposed to clients. A routing definition consists of 4 properties, <code>when</code>, <code>hostname</code>, <code>port</code>, and <code>secure</code>, grouped by <code>protocol</code> and <code>name</code>. Multiple routing definitions can be configured. <code>protocol</code> represents Site-to-Site transport protocol, i.e. <code>RAW</code> or <code>HTTP</code>.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.route.{protocol}.{name}.when</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Boolean value, <code>true</code> or <code>false</code>. Controls whether the routing definition for this name should be used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.route.{protocol}.{name}.hostname</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specify hostname that will be introduced to Site-to-Site clients for further communications.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.route.{protocol}.{name}.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specify port number that will be introduced to Site-to-Site clients for further communications.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.remote.route.{protocol}.{name}.secure</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Boolean value, <code>true</code> or <code>false</code>. Specify whether the remote peer should be accessed via secure protocol. Defaults to <code>false</code>.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>All of above routing properties can use NiFi Expression Language to compute target peer description from request context. Available variables are:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Variable name</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>s2s.{source|target}.hostname</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Hostname of the source where the request came from, and the original target.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>s2s.{source|target}.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Same as above, for ports. Source port may not be useful as it is just a client side TCP port.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>s2s.{source|target}.secure</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Same as above, for secure or not.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>s2s.protocol</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of Site-to-Site protocol being used, <code>RAW</code> or <code>HTTP</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>s2s.request</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of current request type, <code>SiteToSiteDetail</code> or <code>Peers</code>. See Site-to-Site protocol sequence below for detail.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>HTTP request headers</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">HTTP request header values can be referred by its name.</p></td>
</tr>
</tbody>
</table>
<div class="sect3">
<h4 id="site-to-site-protocol-sequence"><a class="anchor" href="administration-guide.html#site-to-site-protocol-sequence"></a>Site to Site protocol sequence</h4>
<div class="paragraph">
<p>Configuring these properties correctly would require some understandings on Site-to-Site protocol sequence.</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>A client initiates Site-to-Site protocol by sending a HTTP(S) request to the specified remote URL to get remote cluster Site-to-Site information. Specifically, to '<em>/nifi-api/site-to-site</em>'. This request is called <code>SiteToSiteDetail</code>.</p>
</li>
<li>
<p>A remote NiFi node responds with its input and output ports, and TCP port numbers for RAW and TCP transport protocols.</p>
</li>
<li>
<p>The client sends another request to get remote peers using the TCP port number returned at #2. From this request, raw socket communication is used for RAW transport protocol, while HTTP keeps using HTTP(S). This request is called <code>Peers</code>.</p>
</li>
<li>
<p>A remote NiFi node responds with list of available remote peers containing hostname, port, secure and workload such as the number of queued FlowFiles. From this point, further communication is done between the client and the remote NiFi node.</p>
</li>
<li>
<p>The client decides which peer to transfer data from/to, based on workload information.</p>
</li>
<li>
<p>The client sends a request to create a transaction to a remote NiFi node.</p>
</li>
<li>
<p>The remote NiFi node accepts the transaction.</p>
</li>
<li>
<p>Data is sent to the target peer. Multiple Data packets can be sent in batch manner.</p>
</li>
<li>
<p>When there is no more data to send, or reached to batch limit, the transaction is confirmed on both end by calculating CRC32 hash of sent data.</p>
</li>
<li>
<p>The transaction is committed on both end.</p>
</li>
</ol>
</div>
</div>
<div class="sect3">
<h4 id="reverse-proxy-configurations"><a class="anchor" href="administration-guide.html#reverse-proxy-configurations"></a>Reverse Proxy Configurations</h4>
<div class="paragraph">
<p>Most reverse proxy software implement HTTP and TCP proxy mode. For NiFi RAW Site-to-Site protocol, both HTTP and TCP proxy configurations are required, and at least 2 ports needed to be opened. NiFi HTTP Site-to-Site protocol can minimize the required number of open ports at the reverse proxy to 1.</p>
</div>
<div class="paragraph">
<p>Setting correct HTTP headers at reverse proxies are crucial for NiFi to work correctly, not only routing requests but also authorize client requests. See also <a href="administration-guide.html#proxy_configuration">Proxy Configuration</a> for details.</p>
</div>
<div class="paragraph">
<p>There are two types of requests-to-NiFi-node mapping techniques those can be applied at reverse proxy servers. One is 'Server name to Node' and the other is 'Port number to Node'.</p>
</div>
<div class="paragraph">
<p>With 'Server name to Node', the same port can be used to route requests to different upstream NiFi nodes based on the requested server name (e.g. <code>nifi0.example.com</code>, <code>nifi1.example.com</code>). Host name resolution should be configured to map different host names to the same reverse proxy address, that can be done by adding /etc/hosts file or DNS server entries. Also, if clients to reverse proxy uses HTTPS, reverse proxy server certificate should have wildcard common name or SAN to be accessed by different host names.</p>
</div>
<div class="paragraph">
<p>Some reverse proxy technologies do not support server name routing rules, in such case, use 'Port number to Node' technique. 'Port number to Node' mapping requires N open port at a reverse proxy for a NiFi cluster consists of N nodes.</p>
</div>
<div class="paragraph">
<p>Refer to the following examples for actual configurations.</p>
</div>
</div>
<div class="sect3">
<h4 id="site-to-site-and-reverse-proxy-examples"><a class="anchor" href="administration-guide.html#site-to-site-and-reverse-proxy-examples"></a>Site to Site and Reverse Proxy Examples</h4>
<div class="paragraph">
<p>Here are some example reverse proxy and NiFi setups to illustrate what configuration files look like.</p>
</div>
<div class="paragraph">
<p>Client1 in the following diagrams represents a client that does not have direct access to NiFi nodes, and it accesses through the reverse proxy, while Client2 has direct access.</p>
</div>
<div class="paragraph">
<p>In this example, Nginx is used as a reverse proxy.</p>
</div>
<div class="sect4">
<h5 id="example-1-raw-server-name-to-node-mapping"><a class="anchor" href="administration-guide.html#example-1-raw-server-name-to-node-mapping"></a>Example 1: RAW - Server name to Node mapping</h5>
<div class="paragraph">
<p><span class="image"><img src="images/s2s-rproxy-servername.svg" alt="Server name to Node mapping"></span></p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Client1 initiates Site-to-Site protocol, the request is routed to one of upstream NiFi nodes. The NiFi node computes Site-to-Site port for RAW. By the routing rule <strong>example1</strong> in <em>nifi.properties</em> shown below, port 10443 is returned.</p>
</li>
<li>
<p>Client1 asks peers to <code>nifi.example.com:10443</code>, the request is routed to <code>nifi0:8081</code>. The NiFi node computes available peers, by <strong>example1</strong> routing rule, <code>nifi0:8081</code> is converted to <code>nifi0.example.com:10443</code>, so are <code>nifi1</code> and <code>nifi2</code>. As a result, <code>nifi0.example.com:10443</code>, <code>nifi1.example.com:10443</code> and <code>nifi2.example.com:10443</code> are returned.</p>
</li>
<li>
<p>Client1 decides to use <code>nifi2.example.com:10443</code> for further communication.</p>
</li>
<li>
<p>On the other hand, Client2 has two URIs for Site-to-Site bootstrap URIs, and initiates the protocol using one of them. The <strong>example1</strong> routing does not match this for this request, and port 8081 is returned.</p>
</li>
<li>
<p>Client2 asks peers from <code>nifi1:8081</code>. The <strong>example1</strong> does not match, so the original <code>nifi0:8081</code>, <code>nifi1:8081</code> and <code>nifi2:8081</code> are returned as they are.</p>
</li>
<li>
<p>Client2 decides to use <code>nifi2:8081</code> for further communication.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Routing rule <strong>example1</strong> defined in <em>nifi.properties</em> (all nodes have the same routing configuration):</p>
</div>
<div class="literalblock">
<div class="content">
<pre># S2S Routing for RAW, using server name to node
nifi.remote.route.raw.example1.when=\
${X-ProxyHost:equals('nifi.example.com'):or(\
${s2s.source.hostname:equals('nifi.example.com'):or(\
${s2s.source.hostname:equals('192.168.99.100')})})}
nifi.remote.route.raw.example1.hostname=${s2s.target.hostname}.example.com
nifi.remote.route.raw.example1.port=10443
nifi.remote.route.raw.example1.secure=true</pre>
</div>
</div>
<div class="paragraph">
<p><em>nginx.conf</em> :</p>
</div>
<div class="literalblock">
<div class="content">
<pre>http {
upstream nifi {
server nifi0:8443;
server nifi1:8443;
server nifi2:8443;
}
# Use dnsmasq so that hostnames such as 'nifi0' can be resolved by /etc/hosts
resolver 127.0.0.1;
server {
listen 443 ssl;
server_name nifi.example.com;
ssl_certificate /etc/nginx/nginx.crt;
ssl_certificate_key /etc/nginx/nginx.key;
proxy_ssl_certificate /etc/nginx/nginx.crt;
proxy_ssl_certificate_key /etc/nginx/nginx.key;
proxy_ssl_trusted_certificate /etc/nginx/nifi-cert.pem;
location / {
proxy_pass https://nifi;
proxy_set_header X-ProxyScheme https;
proxy_set_header X-ProxyHost nginx.example.com;
proxy_set_header X-ProxyPort 17590;
proxy_set_header X-ProxyContextPath /;
proxy_set_header X-ProxiedEntitiesChain &lt;$ssl_client_s_dn&gt;;
}
}
}
stream {
map $ssl_preread_server_name $nifi {
nifi0.example.com nifi0;
nifi1.example.com nifi1;
nifi2.example.com nifi2;
default nifi0;
}
resolver 127.0.0.1;
server {
listen 10443;
proxy_pass $nifi:8081;
}
}</pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="example-2-raw-port-number-to-node-mapping"><a class="anchor" href="administration-guide.html#example-2-raw-port-number-to-node-mapping"></a>Example 2: RAW - Port number to Node mapping</h5>
<div class="paragraph">
<p><span class="image"><img src="images/s2s-rproxy-portnumber.svg" alt="Port number to Node mapping"></span></p>
</div>
<div class="paragraph">
<p>The <strong>example2</strong> routing maps original host names (<code>nifi0</code>, <code>nifi1</code> and <code>nifi2</code>) to different proxy ports (<code>10443</code>, <code>10444</code> and <code>10445</code>) using <code>equals</code> and <code>ifElse</code> expressions.</p>
</div>
<div class="paragraph">
<p>Routing rule <strong>example2</strong> defined in <em>nifi.properties</em> (all nodes have the same routing configuration):</p>
</div>
<div class="literalblock">
<div class="content">
<pre># S2S Routing for RAW, using port number to node
nifi.remote.route.raw.example2.when=\
${X-ProxyHost:equals('nifi.example.com'):or(\
${s2s.source.hostname:equals('nifi.example.com'):or(\
${s2s.source.hostname:equals('192.168.99.100')})})}
nifi.remote.route.raw.example2.hostname=nifi.example.com
nifi.remote.route.raw.example2.port=\
${s2s.target.hostname:equals('nifi0'):ifElse('10443',\
${s2s.target.hostname:equals('nifi1'):ifElse('10444',\
${s2s.target.hostname:equals('nifi2'):ifElse('10445',\
'undefined')})})}
nifi.remote.route.raw.example2.secure=true</pre>
</div>
</div>
<div class="paragraph">
<p><em>nginx.conf</em> :</p>
</div>
<div class="literalblock">
<div class="content">
<pre>http {
# Same as example 1.
}
stream {
map $ssl_preread_server_name $nifi {
nifi0.example.com nifi0;
nifi1.example.com nifi1;
nifi2.example.com nifi2;
default nifi0;
}
resolver 127.0.0.1;
server {
listen 10443;
proxy_pass nifi0:8081;
}
server {
listen 10444;
proxy_pass nifi1:8081;
}
server {
listen 10445;
proxy_pass nifi2:8081;
}
}</pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="example-3-http-server-name-to-node-mapping"><a class="anchor" href="administration-guide.html#example-3-http-server-name-to-node-mapping"></a>Example 3: HTTP - Server name to Node mapping</h5>
<div class="paragraph">
<p><span class="image"><img src="images/s2s-rproxy-http.svg" alt="Server name to Node mapping"></span></p>
</div>
<div class="paragraph">
<p>Routing rule <strong>example3</strong> defined in <em>nifi.properties</em> (all nodes have the same routing configuration):</p>
</div>
<div class="literalblock">
<div class="content">
<pre># S2S Routing for HTTP
nifi.remote.route.http.example3.when=${X-ProxyHost:contains('.example.com')}
nifi.remote.route.http.example3.hostname=${s2s.target.hostname}.example.com
nifi.remote.route.http.example3.port=443
nifi.remote.route.http.example3.secure=true</pre>
</div>
</div>
<div class="paragraph">
<p><em>nginx.conf</em> :</p>
</div>
<div class="literalblock">
<div class="content">
<pre>http {
upstream nifi_cluster {
server nifi0:8443;
server nifi1:8443;
server nifi2:8443;
}
# If target node is not specified, use one from cluster.
map $http_host $nifi {
nifi0.example.com:443 "nifi0:8443";
nifi1.example.com:443 "nifi1:8443";
nifi2.example.com:443 "nifi2:8443";
default "nifi_cluster";
}
resolver 127.0.0.1;
server {
listen 443 ssl;
server_name ~^(.+\.example\.com)$;
ssl_certificate /etc/nginx/nginx.crt;
ssl_certificate_key /etc/nginx/nginx.key;
proxy_ssl_certificate /etc/nginx/nginx.crt;
proxy_ssl_certificate_key /etc/nginx/nginx.key;
proxy_ssl_trusted_certificate /etc/nginx/nifi-cert.pem;
location / {
proxy_pass https://$nifi;
proxy_set_header X-ProxyScheme https;
proxy_set_header X-ProxyHost $1;
proxy_set_header X-ProxyPort 443;
proxy_set_header X-ProxyContextPath /;
proxy_set_header X-ProxiedEntitiesChain &lt;$ssl_client_s_dn&gt;;
}
}
}</pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="web-properties"><a class="anchor" href="administration-guide.html#web-properties"></a>Web Properties</h3>
<div class="paragraph">
<p>These properties pertain to the web-based User Interface.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.http.host</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The HTTP host. The default value is blank.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.http.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The HTTP port. The default value is blank.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.http.port.forwarding</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The port which forwards incoming HTTP requests to <code>nifi.web.http.host</code>. This property is designed to be used with 'port forwarding', when NiFi has to be started by a non-root user for better security, yet it needs to be accessed via low port to go through a firewall. For example, to expose NiFi via HTTP protocol on port 80, but actually listening on port 8080, you need to configure OS level port forwarding such as <code>iptables</code> (Linux/Unix) or <code>pfctl</code> (macOS) that redirects requests from 80 to 8080. Then set <code>nifi.web.http.port</code> as 8080, and <code>nifi.web.http.port.forwarding</code> as 80. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.http.network.interface</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the network interface to which NiFi should bind for HTTP requests. It is blank by default.<br>
+
<strong>NOTE</strong>: Multiple network interfaces can be specified by using the <code>nifi.web.http.network.interface.</code> prefix with unique suffixes and separate network interface names as values.<br>
+
For example, to provide two additional network interfaces, a user could also specify additional properties with keys of:<br>
+
<code>nifi.web.http.network.interface.eth0=eth0</code><br>
<code>nifi.web.http.network.interface.eth1=eth1</code><br>
+
Providing three total network interfaces, including <code>nifi.web.http.network.interface.default</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.host</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The HTTPS host. The default value is <code>127.0.0.1</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The HTTPS port. The default value is <code>8443</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.port.forwarding</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Same as <code>nifi.web.http.port.forwarding</code>, but with HTTPS for secure communication. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.ciphersuites.include</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Cipher suites used to initialize the SSLContext of the Jetty HTTPS port. If unspecified, the runtime SSLContext defaults are used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.ciphersuites.exclude</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Cipher suites that may not be used by an SSL client to establish a connection to Jetty. If unspecified, the runtime SSLContext defaults are used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.should.send.server.version</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether the Server header should be included in HTTP responses. The default value is <code>true</code><br></p>
<p class="tableblock">In Chrome, the SSL cipher negotiated with Jetty may be examined in the 'Developer Tools' plugin, in the 'Security' tab.
In Firefox, the SSL cipher negotiated with Jetty may be examined in the 'Secure Connection' widget found to the left of the URL in the browser address bar.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.network.interface</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the network interface to which NiFi should bind for HTTPS requests. It is blank by default.<br>
+
<strong>NOTE</strong>: Multiple network interfaces can be specified by using the <code>nifi.web.https.network.interface.</code> prefix with unique suffixes and separate network interface names as values.<br>
+
For example, to provide two additional network interfaces, a user could also specify additional properties with keys of:<br>
+
<code>nifi.web.https.network.interface.eth0=eth0</code><br>
<code>nifi.web.https.network.interface.eth1=eth1</code><br>
+
Providing three total network interfaces, including <code>nifi.web.https.network.interface.default</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.https.application.protocols</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The space-separated list of application protocols supported when running with HTTPS enabled.</p>
<p class="tableblock">The default value is <code>http/1.1</code>.</p>
<p class="tableblock">The value can be set to <code>h2 http/1.1</code> to support Application Layer Protocol Negotiation (ALPN) for HTTP/2 or HTTP/1.1 based on client capabilities.</p>
<p class="tableblock">The value can be set to <code>h2</code> to require HTTP/2 and disable HTTP/1.1.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.jetty.working.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the Jetty working directory. The default value is <code>./work/jetty</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.jetty.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of Jetty threads. The default value is <code>200</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.max.header.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum size allowed for request and response headers. The default value is <code>16 KB</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.proxy.host</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated list of allowed HTTP Host header values to consider when NiFi is running securely and will be receiving requests to a different host[:port] than it is bound to.
For example, when running in a Docker container or behind a proxy (e.g. localhost:18443, proxyhost:443). By default, this value is blank meaning NiFi should only allow requests sent to the
host[:port] that NiFi is bound to.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.proxy.context.path</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated list of allowed HTTP X-ProxyContextPath, X-Forwarded-Context, or X-Forwarded-Prefix header values to consider. By default, this value is
blank meaning all requests containing a proxy context path are rejected. Configuring this property would allow requests where the proxy path is contained in this listing.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.max.content.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum size (HTTP <code>Content-Length</code>) for PUT and POST requests. No default value is set for backward compatibility. Providing a value for this property enables the <code>Content-Length</code> filter on all incoming API requests (except Site-to-Site and cluster communications). A suggested value is <code>20 MB</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.max.requests.per.second</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of requests from a connection per second. Requests in excess of this are first delayed, then throttled. The default value is <code>30000</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.max.access.token.requests.per.second</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of requests for login Access Tokens from a connection per second. Requests in excess of this are rejected with HTTP 429. The default value is <code>25</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.request.ip.whitelist</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated list of IP addresses. Used to specify the IP addresses of clients which can exceed the maximum requests per second (<code>nifi.web.max.requests.per.second</code>). Does not apply to web request timeout.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.request.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The request timeout for web requests. Requests running longer than this time will be forced to end with a HTTP 503 Service Unavailable response. Default value is <code>60 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.web.request.log.format</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The parameterized format for HTTP request log messages.
The format property supports the modifiers and codes described in the Jetty
<a href="https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/server/CustomRequestLog.html" target="_blank" rel="noopener">CustomRequestLog</a>.</p>
<p class="tableblock">The default value uses the Combined Log Format, which follows the
<a href="https://en.wikipedia.org/wiki/Common_Log_Format" target="_blank" rel="noopener">Common Log Format</a> with the addition of <code>Referer</code> and <code>User-Agent</code>
request headers. The default value is:</p>
<p class="tableblock"><code>%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"</code></p>
<p class="tableblock">The CustomRequestLog writes formatted messages using the following SLF4J logger:</p>
<p class="tableblock"><code>org.apache.nifi.web.server.RequestLog</code></p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="security_properties"><a class="anchor" href="administration-guide.html#security_properties"></a>Security Properties</h3>
<div class="paragraph">
<p>These properties pertain to various security features in NiFi. Many of these properties are covered in more detail in the
<a href="administration-guide.html#security_configuration">Security Configuration</a> section of this Administrator&#8217;s Guide.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.sensitive.props.key</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the password used to encrypt any sensitive property values that are configured in processors. By default, it is blank, but the system administrator should provide a value for it. It can be a string of any length, although the recommended minimum length is 10 characters. Be aware that once this password is set and one or more sensitive processor properties have been configured, this password should not be changed.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.sensitive.props.algorithm</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The algorithm used to encrypt sensitive properties. The default value is <code>NIFI_PBKDF2_AES_GCM_256</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.sensitive.props.additional.keys</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The comma separated list of properties in <em>nifi.properties</em> to encrypt in addition to the default sensitive properties (see <a href="administration-guide.html#encrypt-config_tool">Encrypted Passwords in Configuration Files</a>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.autoreload.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies whether the SSL context factory should be automatically reloaded if updates to the keystore and truststore are detected. By default, it is set to <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.autoreload.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the interval at which the keystore and truststore are checked for updates. Only applies if <code>nifi.security.autoreload.enabled</code> is set to <code>true</code>. The default value is <code>10 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keystore</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The full path and name of the keystore. The default value is <code>./conf/keystore.p12</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keystoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The keystore type. The default value is <code>PKCS12</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keystorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The keystore password. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.keyPasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The key password. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.truststore</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The full path and name of the truststore. The default value is <code>./conf/truststore.p12</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.truststoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The truststore type. The default value is <code>PKCS12</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.truststorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The truststore password. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.authorizer</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies which of the configured Authorizers in the <em>authorizers.xml</em> file to use. By default, it is set to <code>single-user-authorizer</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.allow.anonymous.authentication</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether anonymous authentication is allowed when running over HTTPS. If set to true, client certificates are not required to connect via TLS. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.login.identity.provider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This indicates what type of login identity provider to use. It can be set to the identifier from a provider in the file specified in <code>nifi.login.identity.provider.configuration.file</code>. Setting this property will trigger NiFi to support username/password authentication. The default value is <code>single-user-provider</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.ocsp.responder.url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the URL for the Online Certificate Status Protocol (OCSP) responder if one is being used. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.ocsp.responder.certificate</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is the location of the OCSP responder certificate if one is being used. It is blank by default.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="bootstrap-properties"><a class="anchor" href="administration-guide.html#bootstrap-properties"></a>Bootstrap Properties</h3>
<div class="paragraph">
<p>These properties pertain to the connection NiFi uses to receive communications from NiFi Bootstrap.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.listener.bootstrap.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This property defines the port used to listen for communications from NiFi Bootstrap. If this property is missing, empty, or <code>0</code>, a random ephemeral port is used.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="identity-mapping-properties"><a class="anchor" href="administration-guide.html#identity-mapping-properties"></a>Identity Mapping Properties</h3>
<div class="paragraph">
<p>These properties can be utilized to normalize user identities. When implemented, identities authenticated by different identity providers (certificates, LDAP, Kerberos) are treated the same internally in NiFi. As a result, duplicate users are avoided and user-specific configurations such as authorizations only need to be setup once per user.</p>
</div>
<div class="paragraph">
<p>The following examples demonstrate normalizing DNs from certificates and principals from Kerberos:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
nifi.security.identity.mapping.value.dn=$1@$2
nifi.security.identity.mapping.transform.dn=NONE
nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
nifi.security.identity.mapping.value.kerb=$1@$2
nifi.security.identity.mapping.transform.kerb=NONE</pre>
</div>
</div>
<div class="paragraph">
<p>The last segment of each property is an identifier used to associate the pattern with the replacement value. When a user makes a request to NiFi, their identity is checked to see if it matches each of those patterns in lexicographical order. For the first one that matches, the replacement specified in the <code>nifi.security.identity.mapping.value.xxxx</code> property is used. So a login with <code>CN=localhost, OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, C=US</code> matches the DN mapping pattern above and the DN mapping value <code>$1@$2</code> is applied. The user is normalized to <code>localhost@Apache NiFi</code>.</p>
</div>
<div class="paragraph">
<p>In addition to mapping, a transform may be applied. The supported versions are <code>NONE</code> (no transform applied), <code>LOWER</code> (identity lowercased), and <code>UPPER</code> (identity uppercased). If not specified, the default value is <code>NONE</code>.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
These mappings are also applied to the "Initial Admin Identity", "Cluster Node Identity", and any legacy users in the <em>authorizers.xml</em> file as well as users imported from LDAP (See <a href="administration-guide.html#authorizers-setup">Authorizers.xml Setup</a>).
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Group names can also be mapped. The following example will accept the existing group name but will lowercase it. This may be helpful when used in conjunction with an external authorizer.</p>
</div>
<div class="listingblock">
<div class="content">
<pre>nifi.security.group.mapping.pattern.anygroup=^(.*)$
nifi.security.group.mapping.value.anygroup=$1
nifi.security.group.mapping.transform.anygroup=LOWER</pre>
</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
These mappings are applied to any legacy groups referenced in the <em>authorizers.xml</em> as well as groups imported from LDAP.
</td>
</tr>
</table>
</div>
</div>
<div class="sect2">
<h3 id="cluster_common_properties"><a class="anchor" href="administration-guide.html#cluster_common_properties"></a>Cluster Common Properties</h3>
<div class="paragraph">
<p>When setting up a NiFi cluster, these properties should be configured the same way on all nodes.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.protocol.heartbeat.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The interval at which nodes should emit heartbeats to the Cluster Coordinator. The default value is <code>5 sec</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.protocol.heartbeat.missable.max</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Maximum number of heartbeats a Cluster Coordinator can miss for a node in the cluster before the Cluster Coordinator updates the node status to Disconnected. The default value is <code>8</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.protocol.is.secure</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This indicates whether cluster communications are secure. The default value is <code>false</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="cluster_node_properties"><a class="anchor" href="administration-guide.html#cluster_node_properties"></a>Cluster Node Properties</h3>
<div class="paragraph">
<p>Configure these properties for cluster nodes.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.is.node</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Set this to <code>true</code> if the instance is a node in a cluster. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.address</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The fully qualified address of the node. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.protocol.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The node&#8217;s protocol port. It is blank by default.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.protocol.max.threads</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of threads that should be used to communicate with other nodes in the cluster. This property defaults to <code>50</code>. When a request is made to one node, it must be forwarded to the coordinator. The coordinator then replicates it to all nodes. There could be up to <code>n+2</code> threads for a given request, where <code>n</code> = number of nodes in your cluster. As an example, if 4 requests are made, a 5 node cluster will use <code>4 * 7 = 28</code> threads.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.event.history.size</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When the state of a node in the cluster is changed, an event is generated
and can be viewed in the Cluster page. This value indicates how many events to keep in memory for each node. The default value is <code>25</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.connection.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When connecting to another node in the cluster, specifies how long this node should wait before considering
the connection a failure. The default value is <code>5 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.read.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When communicating with another node in the cluster, specifies how long this node should wait to receive information
from the remote node before considering the communication with the node a failure. The default value is <code>5 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.node.max.concurrent.requests</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of outstanding web requests that can be replicated to nodes in the cluster. If this number of requests is exceeded, the embedded Jetty server will return a "409: Conflict" response. This property defaults to <code>100</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.firewall.file</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the node firewall file. This is a file that may be used to list all the nodes that are allowed to connect
to the cluster. It provides an additional layer of security. This value is blank by default, meaning that no firewall file is to be used. See <a href="administration-guide.html#cluster_firewall_configuration">Cluster Firewall Configuration</a> for file format details.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.flow.election.max.wait.time</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the amount of time to wait before electing a Flow as the "correct" Flow. If the number of Nodes that have voted is equal to the number specified
by the <code>nifi.cluster.flow.election.max.candidates</code> property, the cluster will not wait this long. The default value is <code>5 mins</code>. Note that the time starts as soon as the first vote is cast.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.flow.election.max.candidates</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the number of Nodes required in the cluster to cause early election of Flows. This allows the Nodes in the cluster to avoid having to wait a
long time before starting processing if we reach at least this number of nodes in the cluster.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.load.balance.port</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the port to listen on for incoming connections for load balancing data across the cluster. The default value is <code>6342</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.load.balance.host</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies the hostname to listen on for incoming connections for load balancing data across the cluster. If not specified, will default to the value used by the
<code>nifi.cluster.node.address</code> property. The value set here does not have to be a hostname/IP address that is addressable outside of the cluster. However, all nodes within the cluster must be able to
connect to the node using this hostname/IP address.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.load.balance.connections.per.node</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of connections to create between this node and each other node in the cluster. For example, if there are 5 nodes in the cluster and this value is set to 4, there will be up to 20 socket connections established for load-balancing purposes (5 x 4 = 20). The default value is <code>1</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.load.balance.max.thread.count</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of threads to use for transferring data from this node to other nodes in the cluster. While a given thread can only write to a single socket at a time, a single thread is capable of servicing multiple connections simultaneously because a given connection may not be available for reading/writing at any given time. The default value is <code>8</code>—i.e., up to 8 threads will be responsible for transferring data to other nodes, regardless of how many nodes are in the cluster.</p>
<p class="tableblock"><strong>NOTE:</strong> Increasing this value will allow additional threads to be used for communicating with other nodes in the cluster and writing the data to the Content and FlowFile Repositories. However, if this property is set to a value greater than the number of nodes in the cluster multiplied by the number of connections per node (<code>nifi.cluster.load.balance.connections.per.node</code>), then no further benefit will be gained and resources will be wasted.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.cluster.load.balance.comms.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">When communicating with another node, if this amount of time elapses without making any progress when reading from or writing to a socket, then a TimeoutException will be thrown. This will then result in the data either being retried or sent to another node in the cluster, depending on the configured Load Balancing Strategy. The default value is <code>30 sec</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="zookeeper-properties"><a class="anchor" href="administration-guide.html#zookeeper-properties"></a>ZooKeeper Properties</h3>
<div class="paragraph">
<p>NiFi depends on Apache ZooKeeper for determining which node in the cluster should play the role of Primary Node
and which node should play the role of Cluster Coordinator. These properties must be configured in order for NiFi
to join a cluster.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.connect.string</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The Connect String that is needed to connect to Apache ZooKeeper. This is a comma-separated list
of hostname:port pairs. For example, <code>localhost:2181,localhost:2182,localhost:2183</code>. This should contain a list of all ZooKeeper
instances in the ZooKeeper quorum. This property must be specified to join a cluster and has no default value.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.connect.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">How long to wait when connecting to ZooKeeper before considering the connection a failure. The default value is <code>10 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.session.timeout</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">How long to wait after losing a connection to ZooKeeper before the session is expired. The default value is <code>10 secs</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.root.node</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The root ZNode that should be used in ZooKeeper. ZooKeeper provides a directory-like structure
for storing data. Each 'directory' in this structure is referred to as a ZNode. This denotes the root ZNode, or 'directory',
that should be used for storing data. The default value is <code>/nifi</code>. This is important to set correctly, as which cluster
the NiFi instance attempts to join is determined by which ZooKeeper instance it connects to and the ZooKeeper Root Node
that is specified.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.client.secure</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to acccess ZooKeeper using client TLS. The default value is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.keystore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filename of the Keystore containing the private key to use when communicating with ZooKeeper.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.keystoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. The type of the Keystore. Must be <code>PKCS12</code>, <code>JKS</code>, or <code>PEM</code>. If not specified the type will be determined from the file extension (<code>.p12</code>, <code>.jks</code>, <code>.pem</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.keystorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Keystore.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.truststore</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Filename of the Truststore that will be used to verify the ZooKeeper server(s).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.truststoreType</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. The type of the Truststore. Must be <code>PKCS12</code>, <code>JKS</code>, or <code>PEM</code>. If not specified the type will be determined from the file extension (<code>.p12</code>, <code>.jks</code>, <code>.pem</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.security.truststorePasswd</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The password for the Truststore.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.zookeeper.jute.maxbuffer</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Maximum buffer size in bytes for packets sent to and received from ZooKeeper.
Defaults to <code>1048575</code> bytes (<code>0xfffff</code> in hexadecimal) following ZooKeeper default <code>jute.maxbuffer</code> property.</p>
<p class="tableblock">The <a href="http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Unsafe+Options">ZooKeeper Administrator&#8217;s Guide</a> categorizes this property as an <strong>unsafe option</strong>.
Changing this property <strong>requires</strong> setting <code>jute.maxbuffer</code> on ZooKeeper servers.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="kerberos_properties"><a class="anchor" href="administration-guide.html#kerberos_properties"></a>Kerberos Properties</h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.kerberos.krb5.file</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the krb5 file, if used. It is blank by default. At this time, only a single krb5 file is allowed to
be specified per NiFi instance, so this property is configured here to support SPNEGO and service principals rather than in individual Processors.
If necessary the krb5 file can support multiple realms.
Example: <code>/etc/krb5.conf</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.kerberos.service.principal</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the NiFi Kerberos service principal, if used. It is blank by default. Note that this property is for NiFi to authenticate as a client other systems.
Example: <code>nifi/nifi.example.com</code> or <code>nifi/nifi.example.com@EXAMPLE.COM</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.kerberos.service.keytab.location</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The file path of the NiFi Kerberos keytab, if used. It is blank by default. Note that this property is for NiFi to authenticate as a client other systems.
Example: <code>/etc/nifi.keytab</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.kerberos.spnego.principal</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the NiFi Kerberos service principal, if used. It is blank by default. Note that this property is used to authenticate NiFi users.
Example: <code>HTTP/nifi.example.com</code> or <code>HTTP/nifi.example.com@EXAMPLE.COM</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.kerberos.spnego.keytab.location</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The file path of the NiFi Kerberos keytab, if used. It is blank by default. Note that this property is used to authenticate NiFi users.
Example: <code>/etc/http-nifi.keytab</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.kerberos.spengo.authentication.expiration</code>*</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The expiration duration of a successful Kerberos user authentication, if used. The default value is <code>12 hours</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="analytics_properties"><a class="anchor" href="administration-guide.html#analytics_properties"></a>Analytics Properties</h3>
<div class="paragraph">
<p>These properties determine the behavior of the internal NiFi predictive analytics capability, such as backpressure prediction, and should be configured the same way on all nodes.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.analytics.predict.enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This indicates whether prediction should be enabled for the cluster. The default is <code>false</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.analytics.predict.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The time interval for which analytical predictions (e.g. queue saturation) should be made. The default value is <code>3 mins</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.analytics.query.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The time interval to query for past observations (e.g. the last 3 minutes of snapshots). The default value is <code>5 mins</code>. NOTE: This value should be at least 3 times greater than <code>nifi.components.status.snapshot.frequency</code> to ensure enough observations are retrieved for predictions.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.analytics.connection.model.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The implementation class for the status analytics model used to make connection predictions. The default value is <code>org.apache.nifi.controller.status.analytics.models.OrdinaryLeastSquares</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.analytics.connection.model.score.name</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the scoring type that should be used to evaluate the model. The default value is <code>rSquared</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.analytics.connection.model.score.threshold</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The threshold for the scoring value (where model score should be above given threshold). The default value is <code>.90</code>.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="runtime_monitoring_properties"><a class="anchor" href="administration-guide.html#runtime_monitoring_properties"></a>Runtime Monitoring Properties</h3>
<div class="paragraph">
<p>Long-Running Task Monitor periodically checks the NiFi processor executor threads and produces warning logs and bulletin messages for those that have been running for a longer period of time.
It can be used to detect possibly stuck / hanging processor tasks.
Please note the performance impact of the task monitor: it creates a thread dump for every run that may affect the normal flow execution.
The Long-Running Task Monitor can be disabled via defining no values for its properties, and it is disabled by default.
To enable it, both <code>nifi.monitor.long.running.task.schedule</code> and <code>nifi.monitor.long.running.task.threshold</code> properties need to be configured with valid time periods.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.monitor.long.running.task.schedule</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The time period between successive executions of the Long-Running Task Monitor (e.g. <code>1 min</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.monitor.long.running.task.threshold</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The time period beyond which a task is considered long-running, i.e. stuck / hanging (e.g. <code>5 mins</code>).</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="performance_tracking_properties"><a class="anchor" href="administration-guide.html#performance_tracking_properties"></a>Performance Tracking Properties</h3>
<div class="paragraph">
<p>NiFi exposes a very significant number of metrics by default through the User Interface. However, there are sometimes additional metrics that may add in diagnosing bottlenecks
and improving the performance of the NiFi dataflow.</p>
</div>
<div class="paragraph">
<p>The <code>nifi.performance.tracking.percentage</code> property can be used to enable the tracking of additional metrics. Gathering these metrics, however, require system calls, which can be
expensive on some systems. As a result, this property defaults to a value of <code>0</code>, indicating that the metrics should be captured 0% of the time. I.e., the feature is disabled by
default. To enable this feature, set the value of this property to an integer value in the range of 0 to 100, inclusive. This represents what percentage of the time NiFi should
gather these metrics.</p>
</div>
<div class="paragraph">
<p>For example, if the value is set to 20, then NiFi will gather these metrics for each processor approximately 20% of the times that the Processor is run. The remainder of the time,
it will use the values that it has already captured in order to extrapolate the metrics to additional runs.</p>
</div>
<div class="paragraph">
<p>The metrics that are gathered include what percentage of the time the processor is utilizing the CPU (versus waiting for I/O to complete or blocking due to monitor/lock contention),
what percentage of time the Processor spends reading from the Content Repository, writing to the Content Repository, blocked due to Garbage Collection, etc.</p>
</div>
<div class="paragraph">
<p>So, continuing our example, if we set the value of the <code>nifi.performance.tracking.percentage</code> and a processor is triggered to run 1,000 times, then NiFi will measure how much CPU
time was consumed over the 200 iterations during which it was measured (i.e., 20% of 1,000). Let&#8217;s say that this amounts to 500 milliseconds of CPU time. Additionally, let&#8217;s consider
that the Processor took 5,000 milliseconds to complete those 200 invocations because most of the time was spent blocking on Socket I/O. From this, NiFi will calculate that the CPU
is used approximately 10% of the time (500 / 5,000 * 100%). Now, let&#8217;s consider that in order to complete all 1,000 invocations the Processor took 35 seconds. NiFi will calculate,
then, that the Processor has used approximately 3.5 seconds (or 3500 milliseconds) of CPU time.</p>
</div>
<div class="paragraph">
<p>As a result, if we set the value of this property higher, up to a value of <code>100</code>, we will get more accurate results. However, it may be more expensive to monitor.</p>
</div>
<div class="paragraph">
<p>In order to view these metrics, we can gather diagnostics by running the command <code>nifi.sh diagnostics &lt;filename&gt;</code> and inspecting the generated file. See <a href="administration-guide.html#nifi_diagnostics">NiFi diagnostics</a> for more information.</p>
</div>
</div>
<div class="sect2">
<h3 id="custom_properties"><a class="anchor" href="administration-guide.html#custom_properties"></a>Custom Properties</h3>
<div class="paragraph">
<p>To configure custom properties for use with NiFi’s Expression Language:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Create the custom property. Ensure that:</p>
<div class="ulist">
<ul>
<li>
<p>Each custom property contains a distinct property value, so that it is not overridden by existing environment properties, system properties, or FlowFile attributes.</p>
</li>
<li>
<p>Each node in a clustered environment is configured with the same custom properties.</p>
</li>
</ul>
</div>
</li>
<li>
<p>Update <code>nifi.variable.registry.properties</code> with the location of the custom property file(s):</p>
</li>
</ul>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.variable.registry.properties</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This is a comma-separated list of file location paths for one or more custom property files.</p></td>
</tr>
</tbody>
</table>
<div class="ulist">
<ul>
<li>
<p>Restart your NiFi instance(s) for the updates to be picked up.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Custom properties can also be configured in the NiFi UI. See the <a href="user-guide.html#Variables_Window">Variables Window</a> section in the User Guide for more information.</p>
</div>
</div>
<div class="sect2">
<h3 id="nar_provider_properties"><a class="anchor" href="administration-guide.html#nar_provider_properties"></a>NAR Provider Properties</h3>
<div class="paragraph">
<p>These properties are used for all the configured providers.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.poll.interval</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The interval between polls. The default value is <code>5 min</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.conflict.resolution</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the conflict resolution strategy to use. The default is <code>IGNORE</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.restrain.startup</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If true, the provider restrains NiFi from startup until the first successful resource fetch.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>The following properties allow configuring one or more NAR providers. A NAR provider retrieves NARs from an external source and copies them to the directory specified by <code>nifi.nar.library.autoload.directory</code>.</p>
</div>
<div class="paragraph">
<p>Each NAR provider property follows the format <code>nifi.nar.library.provider.&lt;identifier&gt;.&lt;property-name&gt;</code> and each provider must have at least one property named <code>implementation</code>.</p>
</div>
<div class="sect3">
<h4 id="hdfs-nar-provider"><a class="anchor" href="administration-guide.html#hdfs-nar-provider"></a>HDFS NAR Provider</h4>
<div class="paragraph">
<p>The HDFS NAR provider retrieves NARs using the Hadoop FileSystem API. This can be used with a traditional HDFS instance or with cloud storage, such as <code>s3a</code> or <code>abfs</code>. In order to use cloud storage, the Hadoop Libraries NAR must be re-built with the cloud storage profiles enabled.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.hdfs.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The fully qualified class name of the implementation class which is <code>org.apache.nifi.flow.resource.hadoop.HDFSExternalResourceProvider</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.hdfs.resources</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The comma separated list of configuration resources, such as <code>core-site.xml</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.hdfs.storage.location</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The optional storage location, such as <code>hdfs://hdfs-location</code>. If not specified, the <code>defaultFs</code> from <code>core-site.xml</code> will be used.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.hdfs.source.directory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The directory within the storage location where NARs are located.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.hdfs.kerberos.principal</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">An optional Kerberos principal for authentication. If specified, one of keytab or password must also be specified.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.hdfs.kerberos.keytab</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">An optional Kerberos keytab for authentication.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.hdfs.kerberos.password</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">An optional Kerberos password for authentication.</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect3">
<h4 id="nifi-registry-nar-provider"><a class="anchor" href="administration-guide.html#nifi-registry-nar-provider"></a>NiFi Registry NAR Provider</h4>
<div class="paragraph">
<p>The NiFi Registry NAR provider retrieves NARs from a NiFi Registry instance. In a secure installation, this provider will retrieve NARs from all buckets that the NiFi server is authorized to read from.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.nifi-registry.implementation</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The fully qualified class name of the implementation class which is <code>org.apache.nifi.registry.extension.NiFiRegistryNarProvider</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.nar.library.provider.nifi-registry.url</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL of the NiFi Registry instance, such as <code><a href="http://localhost:18080" class="bare">http://localhost:18080</a></code>. If the URL begins with <code>https</code>, then the NiFi keystore and truststore will be used to make the TLS connection.</p></td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="upgrading_nifi"><a class="anchor" href="administration-guide.html#upgrading_nifi"></a>Upgrading NiFi</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The instructions below are general steps to follow when upgrading from a 1.x.0 release to another.</p>
</div>
<div class="paragraph">
<p>Prior to upgrade you should review the <a href="https://cwiki.apache.org/confluence/display/NIFI/Release+Notes" target="_blank" rel="noopener">Release Notes</a> carefully to ensure that you understand the changes made in the new version and the impact they may have on your existing dataflows and/or environment. Additionally, check the <a href="https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance" target="_blank" rel="noopener">Migration Guidance</a> page for items that you should be aware of when moving between specific NiFi versions.</p>
</div>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
All nodes in a cluster must be upgraded to the same NiFi version as nodes with different NiFi versions are not supported in the same cluster.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="preserve-custom-processors"><a class="anchor" href="administration-guide.html#preserve-custom-processors"></a>Preserve Custom Processors</h3>
<div class="paragraph">
<p>If you have any custom NARs, preserve them during upgrade by storing them in a centralized location as follows:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Create a second library directory called <code>custom_lib</code>.</p>
</li>
<li>
<p>Move your custom NARs to this new lib directory.</p>
</li>
<li>
<p>Add a new line to the <em>nifi.properties</em> file to specify this new lib directory:</p>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.directory=./lib
nifi.nar.library.directory.custom=/opt/configuration_resources/custom_lib</pre>
</div>
</div>
</li>
</ol>
</div>
</div>
<div class="sect2">
<h3 id="preserve-modified-nars"><a class="anchor" href="administration-guide.html#preserve-modified-nars"></a>Preserve Modified NARs</h3>
<div class="paragraph">
<p>If you have modified any of the default NAR files, an upgrade will overwrite these changes. Preserve your customizations as follows:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Identify and save the changes you made to the default NAR files.</p>
</li>
<li>
<p>Perform your NiFi upgrade.</p>
</li>
<li>
<p>Implement the same NAR file changes in your new NiFi instance.</p>
</li>
</ol>
</div>
</div>
<div class="sect2">
<h3 id="clear-activity-and-shutdown-existing-nifi"><a class="anchor" href="administration-guide.html#clear-activity-and-shutdown-existing-nifi"></a>Clear Activity and Shutdown Existing NiFi</h3>
<div class="paragraph">
<p>On your existing NiFi installation:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Stop all the source processors to prevent the ingestion of new data.</p>
</li>
<li>
<p>Allow NiFi to run until there is no active data in any of the queues in the dataflow(s).</p>
</li>
<li>
<p>Shutdown your existing NiFi instance(s).</p>
</li>
</ol>
</div>
</div>
<div class="sect2">
<h3 id="install-the-new-nifi-version"><a class="anchor" href="administration-guide.html#install-the-new-nifi-version"></a>Install the new NiFi Version</h3>
<div class="paragraph">
<p>Install the new NiFi into a directory parallel to the existing NiFi installation.</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Download the <a href="https://nifi.apache.org/download.html" target="_blank" rel="noopener">latest version</a> of Apache NiFi.</p>
</li>
<li>
<p>Uncompress the NiFi <em>.tar</em> file (<code>tar -xvzf file-name</code>) into a directory parallel to your existing NiFi directory. For example, if your existing NiFi installation is installed in <code>/opt/nifi/existing-nifi/</code>, install your new NiFi version in <code>/opt/nifi/new-nifi/</code>.</p>
</li>
<li>
<p>If you are upgrading a NiFi cluster, repeat these steps on each node in the cluster.</p>
<div class="literalblock">
<div class="content">
<pre>Host Machine - Node 1
|--&gt; opt/
|--&gt; existing-nifi
|--&gt; new-nifi</pre>
</div>
</div>
<div class="literalblock">
<div class="content">
<pre>Host Machine - Node 2
|--&gt; opt/
|--&gt; existing-nifi
|--&gt; new-nifi</pre>
</div>
</div>
<div class="literalblock">
<div class="content">
<pre>Host Machine - Node 3
|--&gt; opt/
|--&gt; existing-nifi
|--&gt; new-nifi</pre>
</div>
</div>
</li>
</ol>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Make sure that all file and directory ownerships for your new NiFi directories match what you set on the existing directories.
</td>
</tr>
</table>
</div>
</div>
<div class="sect2">
<h3 id="update-the-configuration-files-for-your-new-nifi-installation"><a class="anchor" href="administration-guide.html#update-the-configuration-files-for-your-new-nifi-installation"></a>Update the Configuration Files for Your New NiFi Installation</h3>
<div class="paragraph">
<p>Use the configuration files from your existing NiFi installation to manually update the corresponding properties in your new NiFi deployment.</p>
</div>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
In general, do not copy configuration files from your existing NiFi version to the new NiFi version. The newer configuration files may introduce new properties that would be lost if you copy and paste configuration files.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Use the following table to guide the update of configuration files located in <code>&lt;installation-directory&gt;/conf</code>.</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Configuration file</th>
<th class="tableblock halign-left valign-top">Necessary changes</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>authorizers.xml</em></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Copy the <code>&lt;authorizer&gt;&#8230;&#8203;&lt;/authorizer&gt;</code> configured in the existing <em>authorizers.xml</em> to the new NiFi file.</p>
<p class="tableblock">If you are using the <code>file-provider</code> authorizer, ensure that you copy the <em>users.xml</em> and <em>authorizations.xml</em> files from the existing to the new NiFi.</p>
<p class="tableblock">Configuration best practices recommend creating a separate location outside of the NiFi base directory for storing such configuration files, for example: <code>/opt/nifi/configuration-resources/</code>. If you are storing these files in a separate directory, you do not need to move them. Instead, ensure that the new NiFi is pointing to the same files.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>bootstrap-notification-services.xml</em></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Use the existing NiFi <em>bootstrap-notification-services.xml</em> file to update properties in the new NiFi.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>bootstrap.conf</em></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Use the existing NiFi <em>bootstrap.conf</em> file to update properties in the new NiFi.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><em>flow.json.gz</em></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If you retained the default location for storing flows (<code>&lt;installation-directory&gt;/conf/</code>), copy <em>flow.json.gz</em> from the existing to the new NiFi base install <code>conf</code> directory. If you stored flows to an external location via <em>nifi.properties</em>, update the property <code>nifi.flow.configuration.file</code> to point there.</p>
<p class="tableblock">If you are encrypting sensitive component properties in your dataflow via the sensitive properties key in <em>nifi.properties</em>, make sure the same key is used when copying over your <em>flow.json.gz</em>. If you need to change the key, see the <a href="administration-guide.html#sensitive_flow_migration">Migrating a Flow with Sensitive Properties</a> section below.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="8"><p class="tableblock"><em>nifi.properties</em></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Use the existing <em>nifi.properties</em> to populate the same properties in the new NiFi file.</p>
<p class="tableblock"><strong>Note:</strong> This file contains the majority of NiFi configuration settings, so ensure that you have copied the values correctly.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">If you followed NiFi best practices, the following properties should be pointing to external directories outside of the base NiFi installation path.</p>
<p class="tableblock">If the below properties point to directories inside the NiFi base installation path, you must copy the target directories to the new NiFi. Stop your existing NiFi installation before you do this.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.json.file=</code></p>
<p class="tableblock">If you have retained the default value (<code>./conf/flow.json.gz</code>), copy <em>flow.json.gz</em> from the existing to the new NiFi base install conf directory.</p>
<p class="tableblock">If you stored flows to an external location, update the property value to point there.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flow.configuration.archive.dir=</code></p>
<p class="tableblock">Same applies as above if you want to retain archived copies of the <em>flow.json.gz</em>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.database.directory=</code></p>
<p class="tableblock">Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external database repository location.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.flowfile.repository.directory=</code></p>
<p class="tableblock">Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external flowfile repository location.</p>
<p class="tableblock"><strong>Warning:</strong> You may experience data loss if flowfile repositories are not accessible to the new NiFi.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.content.repository.directory.default=</code></p>
<p class="tableblock">Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external content repository location.</p>
<p class="tableblock">Your existing NiFi may have multiple content repos defined. Make sure the exact same property names are used and point to the appropriate matching content repo locations. For example:</p>
<p class="tableblock"><code>nifi.content.repository.directory.content1=</code>
<code>nifi.content.repository.directory.content2=</code></p>
<p class="tableblock"><strong>Warning:</strong> You may experience data loss if content repositories are not accessible to the new NiFi.</p>
<p class="tableblock"><strong>Warning:</strong> You may experience data loss if property names are wrong or the property points to the wrong content repository.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.provenance.repository.directory.default=</code></p>
<p class="tableblock">Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external provenance repository location.</p>
<p class="tableblock">Your existing NiFi may have multiple content repos defined. Make sure the exact same property names are used and point to the appropriate matching provenance repo locations. For example:</p>
<p class="tableblock"><code>nifi.provenance.repository.directory.provenance1=</code>
<code>nifi.provenance.repository.directory.provenance2=</code></p>
<p class="tableblock"><strong>Note:</strong> You may not be able to query old events if provenance repos are not moved correctly or properties are not updated correctly.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="4"><p class="tableblock"><em>state-management.xml</em></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">For the <code>local-provider</code> state provider, verify the location of the local directory.</p>
<p class="tableblock">If you have retained the default location (<code>./state/local</code>), copy the complete directory tree to the new NiFi. The existing NiFi should be stopped if you are copying this directory because it may be constantly writing to this directory while running.</p>
<p class="tableblock">Configuration best practices recommend that you move the state to an external directory like <code>/opt/nifi/configuration-resources/</code> to facilitate easier upgrading later.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">For a NiFi cluster, the <code>cluster-provider</code>
ZooKeeper “Connect String" property should be set to the same external ZooKeeper as the existing NiFi installation.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">For a NiFi cluster, make sure the <code>cluster-provider</code> ZooKeeper "Root Node" property matches exactly the value used in the existing NiFi.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">If you are also setting up a new external ZooKeeper, see the <a href="administration-guide.html#zookeeper_migrator">ZooKeeper Migrator</a> section for instructions on how to move ZooKeeper information from one cluster to another and migrate ZooKeeper node ownership.</p></td>
</tr>
</tbody>
</table>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
Double check all configured properties for typos.
</td>
</tr>
</table>
</div>
<div class="sect3">
<h4 id="sensitive_flow_migration"><a class="anchor" href="administration-guide.html#sensitive_flow_migration"></a>Migrating a Flow with Sensitive Properties</h4>
<div class="paragraph">
<p>When a value is set for <code>nifi.sensitive.props.key</code> in <em>nifi.properties</em>, the specified key is used to encrypt sensitive properties in the flow (e.g. password fields in components). If the key needs to change, the Encrypt-Config tool in the NiFi Toolkit can migrate the sensitive properties key and update the <em>flow.json.gz</em>. Specifically, Encrypt-Config:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Reads the existing <em>flow.json.gz</em> and decrypts the sensitive values using the current key.</p>
</li>
<li>
<p>Encrypts all the sensitive values with a specified new key.</p>
</li>
<li>
<p>Updates the <em>nifi.properties</em> and <em>flow.json.gz</em> files or creates new versions of them.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>As an example, assume version 1.9.2 is the existing NiFi instance and the sensitive properties key is set to <code>password</code>. The goal is to move the 1.9.2 <em>flow.xml.gz</em> to a 1.10.0 instance with a new sensitive properties key: <code>new_password</code>. Running the following Encrypt-Config command would read in the <em>flow.xml.gz</em> and <em>nifi.properties</em> files from 1.9.2 using the original sensitive properties key and write out new versions in 1.10.0 with the sensitive properties encrypted with the new password:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>$ ./nifi-toolkit-1.10.0/bin/encrypt-config.sh -f /path/to/nifi/nifi-1.9.2/conf/flow.json.gz -g /path/to/nifi/nifi-1.10.0/conf/flow.json.gz -s new_password -n /path/to/nifi/nifi-1.9.2/conf/nifi
.properties -o /path/to/nifi/nifi-1.10.0/conf/nifi.properties -x</code></pre>
</div>
</div>
<div class="paragraph">
<p>where:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>-f</code> specifies the source <em>flow.json.gz</em> (nifi-1.9.2)</p>
</li>
<li>
<p><code>-g</code> specifies the destination <em>flow.json.gz</em> (nifi-1.10.0)</p>
</li>
<li>
<p><code>-s</code> specifies the new sensitive properties key (<code>new_password</code>)</p>
</li>
<li>
<p><code>-n</code> specifies the source <em>nifi.properties</em> (nifi-1.9.2)</p>
</li>
<li>
<p><code>-o</code> specifies the destination <em>nifi.properties</em> (nifi-1.10.0)</p>
</li>
<li>
<p><code>-x</code> tells Encrypt-Config to only process the sensitive properties</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For more information see the <a href="toolkit-guide.html#encrypt_config_tool">Encrypt-Config Tool</a> section in the NiFi Toolkit Guide.</p>
</div>
</div>
<div class="sect3">
<h4 id="updating-the-sensitive-properties-algorithm"><a class="anchor" href="administration-guide.html#updating-the-sensitive-properties-algorithm"></a>Updating the Sensitive Properties Algorithm</h4>
<div class="paragraph">
<p>The following command can be used to read an existing flow configuration and set a new sensitive properties algorithm in <em>nifi.properties</em>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>$ ./bin/nifi.sh set-sensitive-properties-algorithm &lt;algorithm&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>The command reads the following flow configuration file properties from <em>nifi.properties</em>:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>nifi.flow.configuration.file</code></p>
</li>
<li>
<p><code>nifi.flow.configuration.json.file</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The command checks for the existence of each file and updates the sensitive property values found.</p>
</div>
<div class="paragraph">
<p>See <a href="administration-guide.html#property-encryption-algorithms">Property Encryption Algorithms</a> for supported values.</p>
</div>
</div>
<div class="sect3">
<h4 id="updating-the-sensitive-properties-key"><a class="anchor" href="administration-guide.html#updating-the-sensitive-properties-key"></a>Updating the Sensitive Properties Key</h4>
<div class="paragraph">
<p>Starting with version 1.14.0, NiFi requires a value for <code>nifi.sensitive.props.key</code> in <em>nifi.properties</em>.</p>
</div>
<div class="paragraph">
<p>The following command can be used to read an existing flow configuration and set a new sensitive properties key in <em>nifi.properties</em>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>$ ./bin/nifi.sh set-sensitive-properties-key &lt;sensitivePropertiesKey&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>The command reads the following flow configuration file properties from <em>nifi.properties</em>:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>nifi.flow.configuration.file</code></p>
</li>
<li>
<p><code>nifi.flow.configuration.json.file</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The command checks for the existence of each file and updates the sensitive property values found.</p>
</div>
<div class="paragraph">
<p>The minimum required length for a new sensitive properties key is 12 characters.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="start-new-nifi"><a class="anchor" href="administration-guide.html#start-new-nifi"></a>Start New NiFi</h3>
<div class="paragraph">
<p>In your new NiFi installation:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Start each of your new NiFi instances.</p>
</li>
<li>
<p>Verify that:</p>
<div class="ulist">
<ul>
<li>
<p>All your dataflows have returned to a running state. Some processors may have new properties that need to be configured, in which case they will be stopped and marked Invalid (<span class="image"><img src="images/iconAlert.png" alt="Invalid"></span>).</p>
</li>
<li>
<p>All your expected controller services and reporting tasks are running again. Address any controller services or reporting tasks that are marked Invalid (<span class="image"><img src="images/iconAlert.png" alt="Invalid"></span>).</p>
</li>
</ul>
</div>
</li>
<li>
<p>After confirming your new NiFi instances are stable and working as expected, the old installation can be removed.</p>
</li>
</ol>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
If the original NiFi was setup to run as a service, update any symlinks or service scripts to point to the new NiFi version executables.
</td>
</tr>
</table>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="processor-locations"><a class="anchor" href="administration-guide.html#processor-locations"></a>Processor Locations</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="processor-location-options"><a class="anchor" href="administration-guide.html#processor-location-options"></a>Available Configuration Options</h3>
<div class="paragraph">
<p>NiFi provides 3 configuration options for processor locations. Namely:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.directory
nifi.nar.library.directory.&lt;custom&gt;
nifi.nar.library.autoload.directory</pre>
</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Paths set using these options are relative to the NiFi Home Directory. For example, if the NiFi Home Directory is <code>/var/lib/nifi</code>, and the Library Directory is <code>./lib</code>, then the final path is <code>/var/lib/nifi/lib</code>.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>The <code>nifi.nar.library.directory</code> is used for the default location for provided NiFi processors. It is not recommended to use this for custom processors as these could be lost during a NiFi upgrade. For example:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.directory=./lib</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>nifi.nar.library.directory.&lt;custom&gt;</code> allows the admin to provide multiple arbritary paths for NiFi to locate custom processors. A unique property identifier must append the property for each unique path. For example:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.directory.myCustomLibs=./my-custom-nars/lib
nifi.nar.library.directory.otherCustomLibs=./other-custom-nars/lib</pre>
</div>
</div>
<div class="paragraph">
<p>The <code>nifi.nar.library.autoload.directory</code> is used by the autoload feature, where NiFi can automatically load new processors added to the configured path without requiring a restart. For example:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.autoload.directory=./autoload/lib</pre>
</div>
</div>
</div>
<div class="sect2">
<h3 id="installing-custom-processors"><a class="anchor" href="administration-guide.html#installing-custom-processors"></a>Installing Custom Processors</h3>
<div class="paragraph">
<p>This section describes the original process for installing custom processors that requires a restart to NiFi. To use the Autoloading feature, see the below <a href="administration-guide.html#autoloading-processors">Autoloading Custom Processors</a> section.</p>
</div>
<div class="paragraph">
<p>Firstly, we will configure a directory for the custom processors. See <a href="administration-guide.html#processor-location-options">Available Configuration Options</a> for more about these configuration options.</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.directory.myCustomLibs=./my-custom-nars/lib</pre>
</div>
</div>
<div class="paragraph">
<p>Ensure that this directory exists and has appropriate permissions for the nifi user and group.</p>
</div>
<div class="paragraph">
<p>Now, we must place our custom processor nar in the configured directory. The configured directory is relative to the NiFi Home directory; for example, let us say that our NiFi Home Dir is <code>/var/lib/nifi</code>, we would place our custom processor nar in <code>/var/lib/nifi/my-custom-nars/lib</code>.</p>
</div>
<div class="paragraph">
<p>Ensure that the file has appropriate permissions for the nifi user and group.</p>
</div>
<div class="paragraph">
<p>Restart NiFi and the custom processor should now be available when adding a new Processor to your flow.</p>
</div>
</div>
<div class="sect2">
<h3 id="autoloading-processors"><a class="anchor" href="administration-guide.html#autoloading-processors"></a>Autoloading Custom Processors</h3>
<div class="paragraph">
<p>This section describes the process to use the Autoloading feature for custom processors.</p>
</div>
<div class="paragraph">
<p>To use the autoloading feature, the <code>nifi.nar.library.autoload.directory</code> property must be configured to point at the desired directory. By default, this points at <code>./extensions</code>.</p>
</div>
<div class="paragraph">
<p>For example:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.autoload.directory=./extensions</pre>
</div>
</div>
<div class="paragraph">
<p>Ensure that this directory exists and has appropriate permissions for the nifi user and group.</p>
</div>
<div class="paragraph">
<p>Now, we must place our custom processor nar in the configured directory. The configured directory is relative to the NiFi Home directory; for example, let us say that our NiFi Home Dir is <code>/var/lib/nifi</code>, we would place our custom processor nar in <code>/var/lib/nifi/extensions</code>.</p>
</div>
<div class="paragraph">
<p>Ensure that the file has appropriate permissions for the nifi user and group.</p>
</div>
<div class="paragraph">
<p>Refresh the browser page and the custom processor should now be available when adding a new Processor to your flow.</p>
</div>
</div>
<div class="sect2">
<h3 id="nar-providers"><a class="anchor" href="administration-guide.html#nar-providers"></a>NAR Providers</h3>
<div class="paragraph">
<p>NiFi supports fetching NAR files for the autoloading feature from external sources. This can be achieved by using External Resource Providers.</p>
</div>
<div class="paragraph">
<p>An External Resource Provider serves as a connector between an external data source and NiFi.</p>
</div>
<div class="paragraph">
<p>When configured, an External Resource Provider polls the external source for available NAR files and offers them to the framework. The framework then fetches new NAR files and copies them to
the <code>nifi.nar.library.autoload.directory</code> for autoloading.</p>
</div>
<div class="paragraph">
<p>By default, the polling will happen every 5 minutes. It is possible to change this frequency by specifying the property <code>nifi.nar.library.poll.interval</code>.</p>
</div>
<div class="paragraph">
<p>By default NAR files will be downloaded if no file with the same name exists in the folder defined by <code>nifi.nar.library.autoload.directory</code>. By setting the <code>nifi.nar.library.conflict.resolution</code> other conflict resolution strategies might be applied. Currently, the following strategies are supported:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Name</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Description</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">IGNORE</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Will not replace files: if a file exists in the directory with the same name, it will not be downloaded again.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">REPLACE</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Will replace a file in the target directory if there is an available file in the source but with newer modification date.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Until the first External Resource collection succeeds for every provider, the service prevents NiFi from finishing startup. In order to override this behaviour, the <code>nifi.nar.library.restrain.startup</code> needs to be declared.</p>
</div>
<div class="paragraph">
<p>With value <code>true</code> the service prevents NiFi from starting up until the execution succeeds, with <code>false</code> it does not. The default value is <code>true</code> in case of the property is not set.</p>
</div>
<div class="paragraph">
<p>An External Resource Provider can be configured by adding the <code>nifi.nar.library.provider.&lt;providerName&gt;.implementation</code> property with value containing the proper implementation class. Some implementations might need
further properties. These are defined by the implementation and must be prefixed with <code>nifi.nar.library.provider.&lt;providerName&gt;.</code>.</p>
</div>
<div class="paragraph">
<p>The <code>&lt;providerName&gt;</code> is arbitrary and serves to correlate multiple properties together for a single provider. Multiple providers might be set, with different <code>&lt;providerName&gt;</code>. Currently NiFi supports HDFS based providers.</p>
</div>
<div class="sect3">
<h4 id="hdfs-external-resource-provider"><a class="anchor" href="administration-guide.html#hdfs-external-resource-provider"></a>HDFS External Resource Provider</h4>
<div class="paragraph">
<p>This implementation is capable of downloading files from an HDFS file system.</p>
</div>
<div class="paragraph">
<p>The value of the <code>nifi.nar.library.provider.&lt;providerName&gt;.implementation</code> must be <code>org.apache.nifi.flow.resource.hadoop.HDFSExternalResourceProvider</code>.
The following additional properties are defined by the provider:</p>
</div>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 50%;">
<col style="width: 50%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Name</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">resources</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">List of HDFS resources, separated by comma.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">source.directory</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The source directory of NAR files within HDFS. Note: the provider does not check for files recursively.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">storage.location</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. If set the storage location defined in the core-site.xml will be overwritten by this value.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">kerberos.principal</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. Kerberos principal to authenticate as.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">kerberos.keytab</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. Kerberos keytab associated with the principal.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">kerberos.password</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Optional. Kerberos password associated with the principal.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Example configuration:</p>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.provider.hdfs1.implementation=org.apache.nifi.flow.resource.hadoop.HDFSExternalResourceProvider
nifi.nar.library.provider.hdfs1.resources=/etc/hadoop/core-site.xml
nifi.nar.library.provider.hdfs1.source.directory=/customNars</pre>
</div>
</div>
<div class="literalblock">
<div class="content">
<pre>nifi.nar.library.provider.hdfs2.implementation=org.apache.nifi.flow.resource.hadoop.HDFSExternalResourceProvider
nifi.nar.library.provider.hdfs2.resources=/etc/hadoop/core-site.xml
nifi.nar.library.provider.hdfs2.source.directory=/other/dir/for/customNars</pre>
</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
For backward compatibility reasons, <code>org.apache.nifi.nar.hadoop.HDFSNarProvider</code> is still an accepted implementation, but it is suggested to use 'org.apache.nifi.flow.resource.hadoop.HDFSExternalResourceProvider' instead.Features introduced with the External Resource Provider might not work with this implementation!
</td>
</tr>
</table>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="nifi_diagnostics"><a class="anchor" href="administration-guide.html#nifi_diagnostics"></a>NiFi diagnostics</h2>
<div class="sectionbody">
<div class="paragraph">
<p>It is possible to get diagnostics data from a NiFi node by executing the below command:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>$ ./bin/nifi.sh diagnostics --verbose &lt;file&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>If the file argument is not specified, the information would be added to the <code>nifi-bootstrap.log</code> file.</p>
</div>
<div class="paragraph">
<p>During the diagnostics command execution, the NiFi bootstrap process sends a request to the running NiFi instance, which collects information about the JVM, the operating system and hardware, the NARs loaded in NiFi, the flow configuration and the components being used, the long-running processor tasks, the clustering status, garbage collection, memory pool peak usage, NiFi repositories, parts of the NiFi configuration, a thread dump, etc., and writes it to the specified location.</p>
</div>
<div class="paragraph">
<p>The <code>--verbose</code> flag may be provided as an option before the filename, which may result in additional diagnostic information being written.</p>
</div>
<div class="sect2">
<h3 id="automatic-diagnostics-on-restart-and-shutdown"><a class="anchor" href="administration-guide.html#automatic-diagnostics-on-restart-and-shutdown"></a>Automatic diagnostics on restart and shutdown</h3>
<div class="paragraph">
<p>NiFi can be configured to automatically execute the diagnostics command in the event of a shutdown. The feature is disabled by default and can be enabled with the <code>nifi.diagnostics.on.shutdown.enabled</code> property in the <code>nifi.properties</code> configuration file. It is also possible to configure where the files should be stored and how many files should be kept using the below properties:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>nifi.diagnostics.on.shutdown.directory=./diagnostics
nifi.diagnostics.on.shutdown.max.filecount=10
nifi.diagnostics.on.shutdown.max.directory.size=10 MB</code></pre>
</div>
</div>
<div class="paragraph">
<p>In the case of a lengthy diagnostic, NiFi may terminate before the command execution ends. In this case, the <code>graceful.shutdown.seconds</code> property should be set to a higher value in the <code>bootstrap.conf</code> configuration file.</p>
</div>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2022-12-05 18:19:00 -0600
</div>
</div>
</body>
</html>