| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PutElasticsearchHttpRecord</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PutElasticsearchHttpRecord</h1><h2>Deprecation notice: </h2><p>This processor is deprecated and may be removed in future releases.</p><p>Please consider using one the following alternatives: <a href="../../../nifi-elasticsearch-restapi-nar/1.19.1/org.apache.nifi.processors.elasticsearch.PutElasticsearchRecord/index.html">PutElasticsearchRecord</a></p><h2>Description: </h2><p>Writes the records from a FlowFile into to Elasticsearch, using the specified parameters such as the index to insert into and the type of the document, as well as the operation type (index, upsert, delete, etc.). Note: The Bulk API is used to send the records. This means that the entire contents of the incoming flow file are read into memory, and each record is transformed into a JSON document which is added to a single HTTP request body. For very large flow files (files with a large number of records, e.g.), this could cause memory usage issues.</p><h3>Tags: </h3><p>elasticsearch, insert, update, upsert, delete, write, put, http, record</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Display Name</th><th>API Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Elasticsearch URL</strong></td><td>elasticsearch-http-url</td><td></td><td id="allowable-values"></td><td id="description">Elasticsearch URL which will be connected to, including scheme (http, e.g.), host, and port. The default port for the REST API is 9200.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">SSL Context Service</td><td>SSL Context Service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SSLContextService<br/><strong>Implementations: </strong><a href="../../../nifi-ssl-context-service-nar/1.19.1/org.apache.nifi.ssl.StandardRestrictedSSLContextService/index.html">StandardRestrictedSSLContextService</a><br/><a href="../../../nifi-ssl-context-service-nar/1.19.1/org.apache.nifi.ssl.StandardSSLContextService/index.html">StandardSSLContextService</a></td><td id="description">The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.</td></tr><tr><td id="name"><strong>Character Set</strong></td><td>Character Set</td><td id="default-value">UTF-8</td><td id="allowable-values"></td><td id="description">Specifies the character set of the document data.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Username</td><td>Username</td><td></td><td id="allowable-values"></td><td id="description">Username to access the Elasticsearch cluster<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Password</td><td>Password</td><td></td><td id="allowable-values"></td><td id="description">Password to access the Elasticsearch cluster<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Connection Timeout</strong></td><td>elasticsearch-http-connect-timeout</td><td id="default-value">5 secs</td><td id="allowable-values"></td><td id="description">Max wait time for the connection to the Elasticsearch REST API.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Response Timeout</strong></td><td>elasticsearch-http-response-timeout</td><td id="default-value">15 secs</td><td id="allowable-values"></td><td id="description">Max wait time for a response from the Elasticsearch REST API.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Configuration Service</td><td>proxy-configuration-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>ProxyConfigurationService<br/><strong>Implementation: </strong><a href="../../../nifi-proxy-configuration-nar/1.19.1/org.apache.nifi.proxy.StandardProxyConfigurationService/index.html">StandardProxyConfigurationService</a></td><td id="description">Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.</td></tr><tr><td id="name">Proxy Host</td><td>elasticsearch-http-proxy-host</td><td></td><td id="allowable-values"></td><td id="description">The fully qualified hostname or IP address of the proxy server<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Port</td><td>elasticsearch-http-proxy-port</td><td></td><td id="allowable-values"></td><td id="description">The port of the proxy server<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Username</td><td>proxy-username</td><td></td><td id="allowable-values"></td><td id="description">Proxy Username<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Password</td><td>proxy-password</td><td></td><td id="allowable-values"></td><td id="description">Proxy Password<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Record Reader</strong></td><td>put-es-record-record-reader</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.xml.XMLReader/index.html">XMLReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.syslog.Syslog5424Reader/index.html">Syslog5424Reader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.cef.CEFReader/index.html">CEFReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.syslog.SyslogReader/index.html">SyslogReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.csv.CSVReader/index.html">CSVReader</a><br/><a href="../../../nifi-scripting-nar/1.19.1/org.apache.nifi.record.script.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-parquet-nar/1.19.1/org.apache.nifi.parquet.ParquetReader/index.html">ParquetReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.lookup.ReaderLookup/index.html">ReaderLookup</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.windowsevent.WindowsEventLogReader/index.html">WindowsEventLogReader</a></td><td id="description">Specifies the Controller Service to use for parsing incoming data and determining the data's schema.</td></tr><tr><td id="name">Record Writer</td><td>put-es-record-record-writer</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSetWriterFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.text.FreeFormTextRecordSetWriter/index.html">FreeFormTextRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.avro.AvroRecordSetWriter/index.html">AvroRecordSetWriter</a><br/><a href="../../../nifi-scripting-nar/1.19.1/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html">ScriptedRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.csv.CSVRecordSetWriter/index.html">CSVRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.xml.XMLRecordSetWriter/index.html">XMLRecordSetWriter</a><br/><a href="../../../nifi-parquet-nar/1.19.1/org.apache.nifi.parquet.ParquetRecordSetWriter/index.html">ParquetRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.json.JsonRecordSetWriter/index.html">JsonRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.1/org.apache.nifi.lookup.RecordSetWriterLookup/index.html">RecordSetWriterLookup</a></td><td id="description">After sending a batch of records, Elasticsearch will report if individual records failed to insert. As an example, this can happen if the record doesn't match the mappingfor the index it is being inserted into. This property specifies the Controller Service to use for writing out those individual records sent to 'failure'. If this is not set, then the whole FlowFile will be routed to failure (including any records which may have been inserted successfully). Note that this will only be used if Elasticsearch reports that individual records failed and that in the event that the entire FlowFile fails (e.g. in the event ES is down), the FF will be routed to failure without being interpreted by this record writer. If there is an error while attempting to route the failures, the entire FlowFile will be routed to Failure. Also if every record failed individually, the entire FlowFile will be routed to Failure without being parsed by the writer.</td></tr><tr><td id="name">Log all errors in batch</td><td>put-es-record-log-all-errors</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">After sending a batch of records, Elasticsearch will report if individual records failed to insert. As an example, this can happen if the record doesn't match the mapping for the index it is being inserted into. If this is set to true, the processor will log the failure reason for the every failed record. When set to false only the first error in the batch will be logged.</td></tr><tr><td id="name">Identifier Record Path</td><td>put-es-record-id-path</td><td></td><td id="allowable-values"></td><td id="description">A RecordPath pointing to a field in the record(s) that contains the identifier for the document. If the Index Operation is "index" or "create", this property may be left empty or evaluate to an empty value, in which case the document's identifier will be auto-generated by Elasticsearch. For all other Index Operations, the field's value must be non-empty.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">@timestamp Record Path</td><td>put-es-record-at-timestamp-path</td><td></td><td id="allowable-values"></td><td id="description">A RecordPath pointing to a field in the record(s) that contains the @timestamp for the document. If left blank the @timestamp will be determined using the main @timestamp property<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">@timestamp Value</td><td>put-es-record-at-timestamp</td><td></td><td id="allowable-values"></td><td id="description">The value to use as the @timestamp field (required for Elasticsearch Data Streams)<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Index</strong></td><td>put-es-record-index</td><td></td><td id="allowable-values"></td><td id="description">The name of the index to insert into<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Type</td><td>put-es-record-type</td><td></td><td id="allowable-values"></td><td id="description">The type of this document (required by Elasticsearch versions < 7.0 for indexing and searching). This must be unset or '_doc' for Elasticsearch 7.0+.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Index Operation</strong></td><td>put-es-record-index-op</td><td id="default-value">index</td><td id="allowable-values"></td><td id="description">The type of the operation used to index (create, index, update, upsert, delete)<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Suppress Null Values</strong></td><td>suppress-nulls</td><td id="default-value">Never Suppress</td><td id="allowable-values"><ul><li>Never Suppress <img src="../../../../../html/images/iconInfo.png" alt="Fields that are missing (present in the schema but not in the record), or that have a value of null, will be written out as a null value" title="Fields that are missing (present in the schema but not in the record), or that have a value of null, will be written out as a null value"></img></li><li>Always Suppress <img src="../../../../../html/images/iconInfo.png" alt="Fields that are missing (present in the schema but not in the record), or that have a value of null, will not be written out" title="Fields that are missing (present in the schema but not in the record), or that have a value of null, will not be written out"></img></li><li>Suppress Missing Values <img src="../../../../../html/images/iconInfo.png" alt="When a field has a value of null, it will be written out. However, if a field is defined in the schema and not present in the record, the field will not be written out." title="When a field has a value of null, it will be written out. However, if a field is defined in the schema and not present in the record, the field will not be written out."></img></li></ul></td><td id="description">Specifies how the writer should handle a null field</td></tr><tr><td id="name">Date Format</td><td>Date Format</td><td></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Date fields. If not specified, the default format 'yyyy-MM-dd' is used. If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters, as in 01/25/2017).<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Time Format</td><td>Time Format</td><td></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Time fields. If not specified, the default format 'HH:mm:ss' is used. If specified, the value must match the Java Simple Date Format (for example, HH:mm:ss for a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 18:04:15).<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Timestamp Format</td><td>Timestamp Format</td><td></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Timestamp fields. If not specified, the default format 'yyyy-MM-dd HH:mm:ss' is used. If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy HH:mm:ss for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters; and then followed by a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 01/25/2017 18:04:15).<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr></table><h3>Dynamic Properties: </h3><p>Supports Sensitive Dynamic Properties: <strong>No</strong></p><p>Dynamic Properties allow the user to specify both the name and value of a property.<table id="dynamic-properties"><tr><th>Name</th><th>Value</th><th>Description</th></tr><tr><td id="name">A URL query parameter</td><td id="value">The value to set it to</td><td>Adds the specified property name/value as a query parameter in the Elasticsearch URL used for processing<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr></table></p><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>retry</td><td>A FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeed</td></tr><tr><td>success</td><td>All FlowFiles that are written to Elasticsearch are routed to this relationship</td></tr><tr><td>failure</td><td>All FlowFiles that cannot be written to Elasticsearch are routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>record.count</td><td>The number of records in an outgoing FlowFile. This is only populated on the 'success' relationship.</td></tr><tr><td>failure.count</td><td>The number of records found by Elasticsearch to have errors. This is only populated on the 'failure' relationship.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html> |