| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PutKudu</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PutKudu</h1><h2>Description: </h2><p>Reads records from an incoming FlowFile using the provided Record Reader, and writes those records to the specified Kudu's table. The schema for the Kudu table is inferred from the schema of the Record Reader. If any error occurs while reading records from the input, or writing records to Kudu, the FlowFile will be routed to failure</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>put, database, NoSQL, kudu, HDFS, record</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Display Name</th><th>API Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Kudu Masters</strong></td><td>Kudu Masters</td><td></td><td id="allowable-values"></td><td id="description">Comma separated addresses of the Kudu masters to connect to.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Table Name</strong></td><td>Table Name</td><td></td><td id="allowable-values"></td><td id="description">The name of the Kudu Table to put data into<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Failure Strategy</strong></td><td>Failure Strategy</td><td id="default-value">Route to Failure</td><td id="allowable-values"><ul><li>Route to Failure <img src="../../../../../html/images/iconInfo.png" alt="The FlowFile containing the Records that failed to insert will be routed to the 'failure' relationship" title="The FlowFile containing the Records that failed to insert will be routed to the 'failure' relationship"></img></li><li>Rollback Session <img src="../../../../../html/images/iconInfo.png" alt="If any Record cannot be inserted, all FlowFiles in the session will be rolled back to their input queue. This means that if data cannot be pushed, it will block any subsequent data from be pushed to Kudu as well until the issue is resolved. However, this may be advantageous if a strict ordering is required." title="If any Record cannot be inserted, all FlowFiles in the session will be rolled back to their input queue. This means that if data cannot be pushed, it will block any subsequent data from be pushed to Kudu as well until the issue is resolved. However, this may be advantageous if a strict ordering is required."></img></li></ul></td><td id="description">If one or more Records in a batch cannot be transferred to Kudu, specifies how to handle the failure</td></tr><tr><td id="name">Kerberos User Service</td><td>kerberos-user-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>KerberosUserService<br/><strong>Implementations: </strong><a href="../../../nifi-kerberos-user-service-nar/1.19.1/org.apache.nifi.kerberos.KerberosPasswordUserService/index.html">KerberosPasswordUserService</a><br/><a href="../../../nifi-kerberos-user-service-nar/1.19.1/org.apache.nifi.kerberos.KerberosKeytabUserService/index.html">KerberosKeytabUserService</a><br/><a href="../../../nifi-kerberos-user-service-nar/1.19.1/org.apache.nifi.kerberos.KerberosTicketCacheUserService/index.html">KerberosTicketCacheUserService</a></td><td id="description">Specifies the Kerberos User Controller Service that should be used for authenticating with Kerberos</td></tr><tr><td id="name">Kerberos Credentials Service</td><td>kerberos-credentials-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>KerberosCredentialsService<br/><strong>Implementation: </strong><a href="../../../nifi-kerberos-credentials-service-nar/1.19.1/org.apache.nifi.kerberos.KeytabCredentialsService/index.html">KeytabCredentialsService</a></td><td id="description">Specifies the Kerberos Credentials to use for authentication</td></tr><tr><td id="name">Kerberos Principal</td><td>kerberos-principal</td><td></td><td id="allowable-values"></td><td id="description">The principal to use when specifying the principal and password directly in the processor for authenticating via Kerberos.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Kerberos Password</td><td>kerberos-password</td><td></td><td id="allowable-values"></td><td id="description">The password to use when specifying the principal and password directly in the processor for authenticating via Kerberos.<br/><strong>Sensitive Property: true</strong></td></tr><tr><td id="name"><strong>Skip head line</strong></td><td>Skip head line</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Deprecated. Used to ignore header lines, but this should be handled by a RecordReader (e.g. "Treat First Line as Header" property of CSVReader)</td></tr><tr><td id="name"><strong>Lowercase Field Names</strong></td><td>Lowercase Field Names</td><td id="default-value">false</td><td id="allowable-values"></td><td id="description">Convert column names to lowercase when finding index of Kudu table columns<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Handle Schema Drift</strong></td><td>Handle Schema Drift</td><td id="default-value">false</td><td id="allowable-values"></td><td id="description">If set to true, when fields with names that are not in the target Kudu table are encountered, the Kudu table will be altered to include new columns for those fields.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Record Reader</strong></td><td>record-reader</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.xml.XMLReader/index.html">XMLReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.syslog.Syslog5424Reader/index.html">Syslog5424Reader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.cef.CEFReader/index.html">CEFReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.syslog.SyslogReader/index.html">SyslogReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.csv.CSVReader/index.html">CSVReader</a><br/><a href="../../../nifi-scripting-nar/1.19.1/org.apache.nifi.record.script.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-parquet-nar/1.19.1/org.apache.nifi.parquet.ParquetReader/index.html">ParquetReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.lookup.ReaderLookup/index.html">ReaderLookup</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.windowsevent.WindowsEventLogReader/index.html">WindowsEventLogReader</a></td><td id="description">The service for reading records from incoming flow files.</td></tr><tr><td id="name">Data RecordPath</td><td>Data RecordPath</td><td></td><td id="allowable-values"></td><td id="description">If specified, this property denotes a RecordPath that will be evaluated against each incoming Record and the Record that results from evaluating the RecordPath will be sent to Kudu instead of sending the entire incoming Record. If not specified, the entire incoming Record will be published to Kudu.</td></tr><tr><td id="name">Operation RecordPath</td><td>Operation RecordPath</td><td></td><td id="allowable-values"></td><td id="description">If specified, this property denotes a RecordPath that will be evaluated against each incoming Record in order to determine the Kudu Operation Type. When evaluated, the RecordPath must evaluate to one of hte valid Kudu Operation Types, or the incoming FlowFile will be routed to failure. If this property is specified, the <Kudu Operation Type> property will be ignored.</td></tr><tr><td id="name">Kudu Operation Type</td><td>Insert Operation</td><td id="default-value">INSERT</td><td id="allowable-values"></td><td id="description">Specify operationType for this processor. |
| Valid values are: INSERT, INSERT_IGNORE, UPSERT, UPDATE, DELETE, UPDATE_IGNORE, DELETE_IGNORE. This Property will be ignored if the <Operation RecordPath> property is set.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Flush Mode</strong></td><td>Flush Mode</td><td id="default-value">AUTO_FLUSH_BACKGROUND</td><td id="allowable-values"><ul><li>AUTO_FLUSH_SYNC</li><li>AUTO_FLUSH_BACKGROUND</li><li>MANUAL_FLUSH</li></ul></td><td id="description">Set the new flush mode for a kudu session. |
| AUTO_FLUSH_SYNC: the call returns when the operation is persisted, else it throws an exception. |
| AUTO_FLUSH_BACKGROUND: the call returns when the operation has been added to the buffer. This call should normally perform only fast in-memory operations but it may have to wait when the buffer is full and there's another buffer being flushed. |
| MANUAL_FLUSH: the call returns when the operation has been added to the buffer, else it throws a KuduException if the buffer is full.</td></tr><tr><td id="name"><strong>FlowFiles per Batch</strong></td><td>FlowFiles per Batch</td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">The maximum number of FlowFiles to process in a single execution, between 1 - 100000. Depending on your memory size, and data size per row set an appropriate batch size for the number of FlowFiles to process per client connection setup.Gradually increase this number, only if your FlowFiles typically contain a few records.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Max Records per Batch</strong></td><td>Batch Size</td><td id="default-value">100</td><td id="allowable-values"></td><td id="description">The maximum number of Records to process in a single Kudu-client batch, between 1 - 100000. Depending on your memory size, and data size per row set an appropriate batch size. Gradually increase this number to find out the best one for best performances.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Ignore NULL</strong></td><td>Ignore NULL</td><td id="default-value">false</td><td id="allowable-values"></td><td id="description">Ignore NULL on Kudu Put Operation, Update only non-Null columns if set true<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Kudu Operation Timeout</td><td>kudu-operations-timeout-ms</td><td id="default-value">30000ms</td><td id="allowable-values"></td><td id="description">Default timeout used for user operations (using sessions and scanners)<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Kudu Keep Alive Period Timeout</td><td>kudu-keep-alive-period-timeout-ms</td><td id="default-value">15000ms</td><td id="allowable-values"></td><td id="description">Default timeout used for user operations<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Kudu Client Worker Count</strong></td><td>worker-count</td><td id="default-value">12</td><td id="allowable-values"></td><td id="description">The maximum number of worker threads handling Kudu client read and write operations. Defaults to the number of available processors.</td></tr><tr><td id="name">Kudu SASL Protocol Name</td><td>kudu-sasl-protocol-name</td><td id="default-value">kudu</td><td id="allowable-values"></td><td id="description">The SASL protocol name to use for authenticating via Kerberos. Must match the service principal name.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>A FlowFile is routed to this relationship after it has been successfully stored in Kudu</td></tr><tr><td>failure</td><td>A FlowFile is routed to this relationship if it cannot be sent to Kudu</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>record.count</td><td>Number of records written to Kudu</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3><table id="system-resource-considerations"><tr><th>Resource</th><th>Description</th></tr><tr><td>MEMORY</td><td>An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.</td></tr></table></body></html> |
