| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PaginatedJsonQueryElasticsearch</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PaginatedJsonQueryElasticsearch</h1><h2>Description: </h2><p>A processor that allows the user to run a paginated query (with aggregations) written with the Elasticsearch JSON DSL. It will use the flowfile's content for the query unless the QUERY attribute is populated. Search After/Point in Time queries must include a valid "sort" field.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>elasticsearch, elasticsearch5, elasticsearch6, elasticsearch7, elasticsearch8, query, scroll, page, read, json</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Display Name</th><th>API Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Query</td><td>el-rest-query</td><td></td><td id="allowable-values"></td><td id="description">A query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Query Attribute</td><td>el-query-attribute</td><td></td><td id="allowable-values"></td><td id="description">If set, the executed query will be set on each result flowfile in the specified attribute.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Index</strong></td><td>el-rest-fetch-index</td><td></td><td id="allowable-values"></td><td id="description">The name of the index to use.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Type</td><td>el-rest-type</td><td></td><td id="allowable-values"></td><td id="description">The type of this document (used by Elasticsearch for indexing and searching).<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Client Service</strong></td><td>el-rest-client-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>ElasticSearchClientService<br/><strong>Implementation: </strong><a href="../../../nifi-elasticsearch-client-service-nar/1.19.1/org.apache.nifi.elasticsearch.ElasticSearchClientServiceImpl/index.html">ElasticSearchClientServiceImpl</a></td><td id="description">An Elasticsearch client service to use for running queries.</td></tr><tr><td id="name"><strong>Search Results Split</strong></td><td>el-rest-split-up-hits</td><td id="default-value">Per Response</td><td id="allowable-values"><ul><li>Per Response <img src="../../../../../html/images/iconInfo.png" alt="Flowfile per response." title="Flowfile per response."></img></li><li>Per Hit <img src="../../../../../html/images/iconInfo.png" alt="Flowfile per hit." title="Flowfile per hit."></img></li><li>Per Query <img src="../../../../../html/images/iconInfo.png" alt="Combine results from all query responses (one flowfile per entire paginated result set of hits). Note that aggregations cannot be paged, they are generated across the entire result set and returned as part of the first page. Results are output with one JSON object per line (allowing hits to be combined from multiple pages without loading all results into memory)." title="Combine results from all query responses (one flowfile per entire paginated result set of hits). Note that aggregations cannot be paged, they are generated across the entire result set and returned as part of the first page. Results are output with one JSON object per line (allowing hits to be combined from multiple pages without loading all results into memory)."></img></li></ul></td><td id="description">Output a flowfile containing all hits or one flowfile for each individual hit or one flowfile containing all hits from all paged responses.</td></tr><tr><td id="name"><strong>Aggregation Results Split</strong></td><td>el-rest-split-up-aggregations</td><td id="default-value">Per Response</td><td id="allowable-values"><ul><li>Per Response <img src="../../../../../html/images/iconInfo.png" alt="Flowfile per response." title="Flowfile per response."></img></li><li>Per Hit <img src="../../../../../html/images/iconInfo.png" alt="Flowfile per hit." title="Flowfile per hit."></img></li></ul></td><td id="description">Output a flowfile containing all aggregations or one flowfile for each individual aggregation.</td></tr><tr><td id="name"><strong>Pagination Type</strong></td><td>el-rest-pagination-type</td><td id="default-value">Scroll</td><td id="allowable-values"><ul><li>Scroll <img src="../../../../../html/images/iconInfo.png" alt="Use Elasticsearch "scroll" to page results." title="Use Elasticsearch "scroll" to page results."></img></li><li>Search After <img src="../../../../../html/images/iconInfo.png" alt="Use Elasticsearch "search_after" to page sorted results." title="Use Elasticsearch "search_after" to page sorted results."></img></li><li>Point in Time <img src="../../../../../html/images/iconInfo.png" alt="Use Elasticsearch (7.10+ with XPack) "point in time" to page sorted results." title="Use Elasticsearch (7.10+ with XPack) "point in time" to page sorted results."></img></li></ul></td><td id="description">Pagination method to use. Not all types are available for all Elasticsearch versions, check the Elasticsearch docs to confirm which are applicable and recommended for your service.</td></tr><tr><td id="name"><strong>Pagination Keep Alive</strong></td><td>el-rest-pagination-keep-alive</td><td id="default-value">10 mins</td><td id="allowable-values"></td><td id="description">Pagination "keep_alive" period. Period Elasticsearch will keep the scroll/pit cursor alive in between requests (this is not the time expected for all pages to be returned, but the maximum allowed time for requests between page retrievals).</td></tr><tr><td id="name"><strong>Output No Hits</strong></td><td>el-rest-output-no-hits</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Output a "hits" flowfile even if no hits found for query. If true, an empty "hits" flowfile will be output even if "aggregations" are output.</td></tr></table><h3>Dynamic Properties: </h3><p>Supports Sensitive Dynamic Properties: <strong>No</strong></p><p>Dynamic Properties allow the user to specify both the name and value of a property.<table id="dynamic-properties"><tr><th>Name</th><th>Value</th><th>Description</th></tr><tr><td id="name">The name of a URL query parameter to add</td><td id="value">The value of the URL query parameter</td><td>Adds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request body<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table></p><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>aggregations</td><td>Aggregations are routed to this relationship.</td></tr><tr><td>failure</td><td>All flowfiles that fail for reasons unrelated to server availability go to this relationship.</td></tr><tr><td>original</td><td>All original flowfiles that don't cause an error to occur go to this relationship.</td></tr><tr><td>hits</td><td>Search hits are routed to this relationship.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>mime.type</td><td>application/json</td></tr><tr><td>aggregation.name</td><td>The name of the aggregation whose results are in the output flowfile</td></tr><tr><td>aggregation.number</td><td>The number of the aggregation whose results are in the output flowfile</td></tr><tr><td>page.number</td><td>The number of the page (request) in which the results were returned that are in the output flowfile</td></tr><tr><td>hit.count</td><td>The number of hits that are in the output flowfile</td></tr><tr><td>elasticsearch.query.error</td><td>The error message provided by Elasticsearch if there is an error querying the index.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3><table id="system-resource-considerations"><tr><th>Resource</th><th>Description</th></tr><tr><td>MEMORY</td><td>Care should be taken on the size of each page because each response from Elasticsearch will be loaded into memory all at once and converted into the resulting flowfiles.</td></tr></table></body></html> |