| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PutBigQueryStreaming</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PutBigQueryStreaming</h1><h2>Deprecation notice: </h2><p>This processor is deprecated and may be removed in future releases.</p><p>Please consider using one the following alternatives: <a href="../org.apache.nifi.processors.gcp.bigquery.PutBigQuery/index.html">PutBigQuery</a></p><h2>Description: </h2><p>Please be aware this processor is deprecated and may be removed in the near future. Use PutBigQuery instead. Load data into Google BigQuery table using the streaming API. This processor is not intended to load large flow files as it will load the full content into memory. If you need to insert large flow files, consider using PutBigQueryBatch instead.</p><h3>Tags: </h3><p>google, google cloud, bq, gcp, bigquery, record</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Display Name</th><th>API Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Project ID</td><td>gcp-project-id</td><td></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td>GCP Credentials Provider Service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name"><strong>Number of retries</strong></td><td>gcp-retry-count</td><td id="default-value">6</td><td id="allowable-values"></td><td id="description">How many retry attempts should be made before routing to the failure relationship.</td></tr><tr><td id="name">Proxy host</td><td>gcp-proxy-host</td><td></td><td id="allowable-values"></td><td id="description">IP or hostname of the proxy to be used. |
| You might need to set the following properties in bootstrap for https proxy usage: |
| -Djdk.http.auth.tunneling.disabledSchemes= |
| -Djdk.http.auth.proxying.disabledSchemes=<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy port</td><td>gcp-proxy-port</td><td></td><td id="allowable-values"></td><td id="description">Proxy port number<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Username</td><td>gcp-proxy-user-name</td><td></td><td id="allowable-values"></td><td id="description">HTTP Proxy Username<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Password</td><td>gcp-proxy-user-password</td><td></td><td id="allowable-values"></td><td id="description">HTTP Proxy Password<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Configuration Service</td><td>proxy-configuration-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>ProxyConfigurationService<br/><strong>Implementation: </strong><a href="../../../nifi-proxy-configuration-nar/1.19.0/org.apache.nifi.proxy.StandardProxyConfigurationService/index.html">StandardProxyConfigurationService</a></td><td id="description">Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN</td></tr><tr><td id="name"><strong>Dataset</strong></td><td>bq.dataset</td><td id="default-value">${bq.dataset}</td><td id="allowable-values"></td><td id="description">BigQuery dataset name (Note - The dataset must exist in GCP)<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Table Name</strong></td><td>bq.table.name</td><td id="default-value">${bq.table.name}</td><td id="allowable-values"></td><td id="description">BigQuery table name<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Ignore Unknown Values</strong></td><td>bq.load.ignore_unknown</td><td id="default-value">false</td><td id="allowable-values"></td><td id="description">Sets whether BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. By default unknown values are not allowed.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Record Reader</strong></td><td>bq.record.reader</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.cef.CEFReader/index.html">CEFReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.syslog.Syslog5424Reader/index.html">Syslog5424Reader</a><br/><a href="../../../nifi-scripting-nar/1.19.0/org.apache.nifi.record.script.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.syslog.SyslogReader/index.html">SyslogReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.csv.CSVReader/index.html">CSVReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.xml.XMLReader/index.html">XMLReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.lookup.ReaderLookup/index.html">ReaderLookup</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.windowsevent.WindowsEventLogReader/index.html">WindowsEventLogReader</a><br/><a href="../../../nifi-parquet-nar/1.19.0/org.apache.nifi.parquet.ParquetReader/index.html">ParquetReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.19.0/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a></td><td id="description">Specifies the Controller Service to use for parsing incoming data.</td></tr><tr><td id="name"><strong>Skip Invalid Rows</strong></td><td>bq.skip.invalid.rows</td><td id="default-value">false</td><td id="allowable-values"></td><td id="description">Sets whether to insert all valid rows of a request, even if invalid rows exist. If not set the entire insert request will fail if it contains an invalid row.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google BigQuery operation.</td></tr><tr><td>failure</td><td>FlowFiles are routed to this relationship if the Google BigQuery operation fails.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>bq.records.count</td><td>Number of records successfully inserted</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3><table id="system-resource-considerations"><tr><th>Resource</th><th>Description</th></tr><tr><td>MEMORY</td><td>An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.</td></tr></table><h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch/index.html">PutBigQueryBatch</a></p></body></html> |
