| <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>AvroRecordSetWriter</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">AvroRecordSetWriter</h1><h2>Description: </h2><p>Writes the contents of a RecordSet in Binary Avro format.</p><h3>Tags: </h3><p>avro, result, set, writer, serializer, record, recordset, row</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Display Name</th><th>API Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Schema Write Strategy</strong></td><td>Schema Write Strategy</td><td id="default-value">Embed Avro Schema</td><td id="allowable-values"><ul><li>Embed Avro Schema <img src="../../../../../html/images/iconInfo.png" alt="The FlowFile will have the Avro schema embedded into the content, as is typical with Avro" title="The FlowFile will have the Avro schema embedded into the content, as is typical with Avro"></img></li><li>Do Not Write Schema <img src="../../../../../html/images/iconInfo.png" alt="Do not add any schema-related information to the FlowFile." title="Do not add any schema-related information to the FlowFile."></img></li><li>Set 'schema.name' Attribute <img src="../../../../../html/images/iconInfo.png" alt="The FlowFile will be given an attribute named 'schema.name' and this attribute will indicate the name of the schema in the Schema Registry. Note that ifthe schema for a record is not obtained from a Schema Registry, then no attribute will be added." title="The FlowFile will be given an attribute named 'schema.name' and this attribute will indicate the name of the schema in the Schema Registry. Note that ifthe schema for a record is not obtained from a Schema Registry, then no attribute will be added."></img></li><li>Set 'avro.schema' Attribute <img src="../../../../../html/images/iconInfo.png" alt="The FlowFile will be given an attribute named 'avro.schema' and this attribute will contain the Avro Schema that describes the records in the FlowFile. The contents of the FlowFile need not be Avro, but the text of the schema will be used." title="The FlowFile will be given an attribute named 'avro.schema' and this attribute will contain the Avro Schema that describes the records in the FlowFile. The contents of the FlowFile need not be Avro, but the text of the schema will be used."></img></li><li>HWX Schema Reference Attributes <img src="../../../../../html/images/iconInfo.png" alt="The FlowFile will be given a set of 3 attributes to describe the schema: 'schema.identifier', 'schema.version', and 'schema.protocol.version'. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data." title="The FlowFile will be given a set of 3 attributes to describe the schema: 'schema.identifier', 'schema.version', and 'schema.protocol.version'. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data."></img></li><li>HWX Content-Encoded Schema Reference <img src="../../../../../html/images/iconInfo.png" alt="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single byte indicating the 'protocol version', followed by 8 bytes indicating the schema identifier, and finally 4 bytes indicating the schema version, as per the Hortonworks Schema Registry serializers and deserializers, as found at https://github.com/hortonworks/registry. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data." title="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single byte indicating the 'protocol version', followed by 8 bytes indicating the schema identifier, and finally 4 bytes indicating the schema version, as per the Hortonworks Schema Registry serializers and deserializers, as found at https://github.com/hortonworks/registry. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data."></img></li><li>Confluent Schema Registry Reference <img src="../../../../../html/images/iconInfo.png" alt="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single 'Magic Byte' followed by 4 bytes representing the identifier of the schema, as outlined at http://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data. This is based on the encoding used by version 3.2.x of the Confluent Schema Registry." title="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single 'Magic Byte' followed by 4 bytes representing the identifier of the schema, as outlined at http://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data. This is based on the encoding used by version 3.2.x of the Confluent Schema Registry."></img></li></ul></td><td id="description">Specifies how the schema for a Record should be added to the data.</td></tr><tr><td id="name">Schema Cache</td><td>schema-cache</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSchemaCacheService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.schema.inference.VolatileSchemaCache/index.html">VolatileSchemaCache</a></td><td id="description">Specifies a Schema Cache to add the Record Schema to so that Record Readers can quickly lookup the schema.</td></tr><tr><td id="name">Schema Protocol Version</td><td>schema-protocol-version</td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">The protocol version to be used for Schema Write Strategies that require a protocol version, such as Hortonworks Schema Registry strategies. Valid protocol versions for Hortonworks Schema Registry are integer values 1, 2, or 3.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong><br/><br/><strong>This Property is only considered if </strong><strong>the [Schema Write Strategy] Property is set to one of the following values: [HWX Schema Reference Attributes], [HWX Content-Encoded Schema Reference]</strong></td></tr><tr><td id="name"><strong>Schema Access Strategy</strong></td><td>schema-access-strategy</td><td id="default-value">Inherit Record Schema</td><td id="allowable-values"><ul><li>Inherit Record Schema <img src="../../../../../html/images/iconInfo.png" alt="The schema used to write records will be the same schema that was given to the Record when the Record was created." title="The schema used to write records will be the same schema that was given to the Record when the Record was created."></img></li><li>Use 'Schema Name' Property <img src="../../../../../html/images/iconInfo.png" alt="The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service." title="The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service."></img></li><li>Use 'Schema Text' Property <img src="../../../../../html/images/iconInfo.png" alt="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions." title="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions."></img></li></ul></td><td id="description">Specifies how to obtain the schema that is to be used for interpreting the data.</td></tr><tr><td id="name">Schema Registry</td><td>schema-registry</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SchemaRegistry<br/><strong>Implementations: </strong><a href="../../../nifi-confluent-platform-nar/1.19.1/org.apache.nifi.confluent.schemaregistry.ConfluentSchemaRegistry/index.html">ConfluentSchemaRegistry</a><br/><a href="../../../nifi-registry-nar/1.19.1/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html">AvroSchemaRegistry</a><br/><a href="../../../nifi-hwx-schema-registry-nar/1.19.1/org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry/index.html">HortonworksSchemaRegistry</a></td><td id="description">Specifies the Controller Service to use for the Schema Registry<br/><br/><strong>This Property is only considered if </strong><strong>the [Schema Access Strategy] Property is set to one of the following values: [Use 'Schema Name' Property]</strong></td></tr><tr><td id="name">Schema Name</td><td>schema-name</td><td id="default-value">${schema.name}</td><td id="allowable-values"></td><td id="description">Specifies the name of the schema to lookup in the Schema Registry property<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong><br/><br/><strong>This Property is only considered if </strong><strong>the [Schema Access Strategy] Property has a value of "Use 'Schema Name' Property".</strong></td></tr><tr><td id="name">Schema Version</td><td>schema-version</td><td></td><td id="allowable-values"></td><td id="description">Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong><br/><br/><strong>This Property is only considered if </strong><strong>the [Schema Access Strategy] Property has a value of "Use 'Schema Name' Property".</strong></td></tr><tr><td id="name">Schema Branch</td><td>schema-branch</td><td></td><td id="allowable-values"></td><td id="description">Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong><br/><br/><strong>This Property is only considered if </strong><strong>the [Schema Access Strategy] Property has a value of "Use 'Schema Name' Property".</strong></td></tr><tr><td id="name">Schema Text</td><td>schema-text</td><td id="default-value">${avro.schema}</td><td id="allowable-values"></td><td id="description">The text of an Avro-formatted Schema<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong><br/><br/><strong>This Property is only considered if </strong><strong>the [Schema Access Strategy] Property has a value of "Use 'Schema Text' Property".</strong></td></tr><tr><td id="name"><strong>Compression Format</strong></td><td>compression-format</td><td id="default-value">NONE</td><td id="allowable-values"><ul><li>BZIP2</li><li>DEFLATE</li><li>NONE</li><li>SNAPPY</li><li>LZO</li></ul></td><td id="description">Compression type to use when writing Avro files. Default is None.</td></tr><tr><td id="name"><strong>Cache Size</strong></td><td>cache-size</td><td id="default-value">1000</td><td id="allowable-values"></td><td id="description">Specifies how many Schemas should be cached</td></tr><tr><td id="name"><strong>Encoder Pool Size</strong></td><td>encoder-pool-size</td><td id="default-value">32</td><td id="allowable-values"></td><td id="description">Avro Writers require the use of an Encoder. Creation of Encoders is expensive, but once created, they can be reused. This property controls the maximum number of Encoders that can be pooled and reused. Setting this value too small can result in degraded performance, but setting it higher can result in more heap being used. This property is ignored if the Avro Writer is configured with a Schema Write Strategy of 'Embed Avro Schema'.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>System Resource Considerations:</h3>None specified.</body></html> |