| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PutElasticsearchRecord</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PutElasticsearchRecord</h1><h2>Description: </h2><p>A record-aware Elasticsearch put processor that uses the official Elastic REST client libraries.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>json, elasticsearch, elasticsearch5, elasticsearch6, elasticsearch7, elasticsearch8, put, index, record</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Display Name</th><th>API Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Index Operation</strong></td><td>put-es-record-index-op</td><td id="default-value">index</td><td id="allowable-values"></td><td id="description">The type of the operation used to index (create, delete, index, update, upsert)<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Index</strong></td><td>el-rest-fetch-index</td><td></td><td id="allowable-values"></td><td id="description">The name of the index to use.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Type</td><td>el-rest-type</td><td></td><td id="allowable-values"></td><td id="description">The type of this document (used by Elasticsearch for indexing and searching).<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">@timestamp Value</td><td>put-es-record-at-timestamp</td><td></td><td id="allowable-values"></td><td id="description">The value to use as the @timestamp field (required for Elasticsearch Data Streams)<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Client Service</strong></td><td>el-rest-client-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>ElasticSearchClientService<br/><strong>Implementation: </strong><a href="../../../nifi-elasticsearch-client-service-nar/1.19.1/org.apache.nifi.elasticsearch.ElasticSearchClientServiceImpl/index.html">ElasticSearchClientServiceImpl</a></td><td id="description">An Elasticsearch client service to use for running queries.</td></tr><tr><td id="name"><strong>Record Reader</strong></td><td>put-es-record-reader</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.xml.XMLReader/index.html">XMLReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.syslog.Syslog5424Reader/index.html">Syslog5424Reader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.cef.CEFReader/index.html">CEFReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.syslog.SyslogReader/index.html">SyslogReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.csv.CSVReader/index.html">CSVReader</a><br/><a href="../../../nifi-scripting-nar/1.19.1/org.apache.nifi.record.script.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-parquet-nar/1.19.1/org.apache.nifi.parquet.ParquetReader/index.html">ParquetReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.lookup.ReaderLookup/index.html">ReaderLookup</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.windowsevent.WindowsEventLogReader/index.html">WindowsEventLogReader</a></td><td id="description">The record reader to use for reading incoming records from flowfiles.</td></tr><tr><td id="name"><strong>Batch Size</strong></td><td>put-es-record-batch-size</td><td id="default-value">100</td><td id="allowable-values"></td><td id="description">The number of records to send over in a single batch.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">ID Record Path</td><td>put-es-record-id-path</td><td></td><td id="allowable-values"></td><td id="description">A record path expression to retrieve the ID field for use with Elasticsearch. If left blank the ID will be automatically generated by Elasticsearch.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Retain ID (Record Path)</td><td>put-es-record-retain-id-field</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Whether to retain the existing field used as the ID Record Path.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong><br/><br/><strong>This Property is only considered if </strong><strong>the [ID Record Path] Property has a value specified.</strong></td></tr><tr><td id="name">Index Operation Record Path</td><td>put-es-record-index-op-path</td><td></td><td id="allowable-values"></td><td id="description">A record path expression to retrieve the Index Operation field for use with Elasticsearch. If left blank the Index Operation will be determined using the main Index Operation property.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Index Record Path</td><td>put-es-record-index-record-path</td><td></td><td id="allowable-values"></td><td id="description">A record path expression to retrieve the index field for use with Elasticsearch. If left blank the index will be determined using the main index property.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Type Record Path</td><td>put-es-record-type-record-path</td><td></td><td id="allowable-values"></td><td id="description">A record path expression to retrieve the type field for use with Elasticsearch. If left blank the type will be determined using the main type property.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">@timestamp Record Path</td><td>put-es-record-at-timestamp-path</td><td></td><td id="allowable-values"></td><td id="description">A RecordPath pointing to a field in the record(s) that contains the @timestamp for the document. If left blank the @timestamp will be determined using the main @timestamp property<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Retain @timestamp (Record Path)</td><td>put-es-record-retain-at-timestamp-field</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Whether to retain the existing field used as the @timestamp Record Path.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong><br/><br/><strong>This Property is only considered if </strong><strong>the [@timestamp Record Path] Property has a value specified.</strong></td></tr><tr><td id="name">Date Format</td><td>put-es-record-at-timestamp-date-format</td><td></td><td id="allowable-values"></td><td id="description">Specifies the format to use when writing Date fields. If not specified, the default format 'yyyy-MM-dd' is used. If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters, as in 01/25/2017).<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Time Format</td><td>put-es-record-at-timestamp-time-format</td><td></td><td id="allowable-values"></td><td id="description">Specifies the format to use when writing Time fields. If not specified, the default format 'HH:mm:ss' is used. If specified, the value must match the Java Simple Date Format (for example, HH:mm:ss for a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 18:04:15).<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Timestamp Format</td><td>put-es-record-at-timestamp-timestamp-format</td><td></td><td id="allowable-values"></td><td id="description">Specifies the format to use when writing Timestamp fields. If not specified, the default format 'yyyy-MM-dd HH:mm:ss' is used. If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy HH:mm:ss for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters; and then followed by a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 01/25/2017 18:04:15).<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Log Error Responses</td><td>put-es-record-log-error-responses</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If this is enabled, errors will be logged to the NiFi logs at the error log level. Otherwise, they will only be logged if debug logging is enabled on NiFi as a whole. The purpose of this option is to give the user the ability to debug failed operations without having to turn on debug logging.</td></tr><tr><td id="name">Result Record Writer</td><td>put-es-record-error-writer</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSetWriterFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.text.FreeFormTextRecordSetWriter/index.html">FreeFormTextRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.avro.AvroRecordSetWriter/index.html">AvroRecordSetWriter</a><br/><a href="../../../nifi-scripting-nar/1.19.1/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html">ScriptedRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.csv.CSVRecordSetWriter/index.html">CSVRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.xml.XMLRecordSetWriter/index.html">XMLRecordSetWriter</a><br/><a href="../../../nifi-parquet-nar/1.19.1/org.apache.nifi.parquet.ParquetRecordSetWriter/index.html">ParquetRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonRecordSetWriter/index.html">JsonRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.lookup.RecordSetWriterLookup/index.html">RecordSetWriterLookup</a></td><td id="description">If this configuration property is set, the response from Elasticsearch will be examined for failed records and the failed records will be written to a record set with this record writer service and sent to the "errors" relationship. Successful records will be written to a record setwith this record writer service and sent to the "successful_records" relationship.</td></tr><tr><td id="name">Treat "Not Found" as Error</td><td>put-es-record-not_found-is-error</td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If true, "not_found" Elasticsearch Document associated Records will be routed to the "successful_records" relationship, otherwise to the "errors" relationship.<br/><br/><strong>This Property is only considered if </strong><strong>the [Result Record Writer] Property has a value specified.</strong></td></tr></table><h3>Dynamic Properties: </h3><p>Supports Sensitive Dynamic Properties: <strong>No</strong></p><p>Dynamic Properties allow the user to specify both the name and value of a property.<table id="dynamic-properties"><tr><th>Name</th><th>Value</th><th>Description</th></tr><tr><td id="name">The name of a URL query parameter to add</td><td id="value">The value of the URL query parameter</td><td>Adds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the _bulk request body<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table></p><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>retry</td><td>All flowfiles that fail due to server/cluster availability go to this relationship.</td></tr><tr><td>success</td><td>All flowfiles that succeed in being transferred into Elasticsearch go here. Documents received by the Elasticsearch _bulk API may still result in errors on the Elasticsearch side. The Elasticsearch response will need to be examined to determine whether any Document(s)/Record(s) resulted in errors.</td></tr><tr><td>successful_records</td><td>If a "Result Record Writer" is set, any Record(s) corresponding to Elasticsearch document(s) that did not result in an "error" (within Elasticsearch) will be routed here.</td></tr><tr><td>failure</td><td>All flowfiles that fail for reasons unrelated to server availability go to this relationship.</td></tr><tr><td>errors</td><td>If a "Result Record Writer" is set, any Record(s) corresponding to Elasticsearch document(s) that resulted in an "error" (within Elasticsearch) will be routed here.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>elasticsearch.put.error</td><td>The error message provided by Elasticsearch if there is an error indexing the documents.</td></tr><tr><td>elasticsearch.put.error.count</td><td>The number of records that generated errors in the Elasticsearch _bulk API.</td></tr><tr><td>elasticsearch.put.success.count</td><td>The number of records that were successfully processed by the Elasticsearch _bulk API.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3><table id="system-resource-considerations"><tr><th>Resource</th><th>Description</th></tr><tr><td>MEMORY</td><td>The Batch of Records will be stored in memory until the bulk operation is performed.</td></tr></table></body></html> |