22/generated/connect_transforms.html

<div id="org.apache.kafka.connect.transforms.InsertField">
<h5>org.apache.kafka.connect.transforms.InsertField</h5>
Insert field(s) using attributes from the record metadata or a configured static value.<p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.InsertField$Key</code>) or value (<code>org.apache.kafka.connect.transforms.InsertField$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>offset.field</td></td><td>Field name for Kafka offset - only applicable to sink connectors.<br/>Suffix with <code>!</code> to make this a required field, or <code>?</code> to keep it optional (the default).</td></td><td>string</td></td><td>null</td></td><td></td></td><td>medium</td></td></tr>
<tr>
<td>partition.field</td></td><td>Field name for Kafka partition. Suffix with <code>!</code> to make this a required field, or <code>?</code> to keep it optional (the default).</td></td><td>string</td></td><td>null</td></td><td></td></td><td>medium</td></td></tr>
<tr>
<td>static.field</td></td><td>Field name for static data field. Suffix with <code>!</code> to make this a required field, or <code>?</code> to keep it optional (the default).</td></td><td>string</td></td><td>null</td></td><td></td></td><td>medium</td></td></tr>
<tr>
<td>static.value</td></td><td>Static field value, if field name configured.</td></td><td>string</td></td><td>null</td></td><td></td></td><td>medium</td></td></tr>
<tr>
<td>timestamp.field</td></td><td>Field name for record timestamp. Suffix with <code>!</code> to make this a required field, or <code>?</code> to keep it optional (the default).</td></td><td>string</td></td><td>null</td></td><td></td></td><td>medium</td></td></tr>
<tr>
<td>topic.field</td></td><td>Field name for Kafka topic. Suffix with <code>!</code> to make this a required field, or <code>?</code> to keep it optional (the default).</td></td><td>string</td></td><td>null</td></td><td></td></td><td>medium</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.ReplaceField">
<h5>org.apache.kafka.connect.transforms.ReplaceField</h5>
Filter or rename fields.<p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.ReplaceField$Key</code>) or value (<code>org.apache.kafka.connect.transforms.ReplaceField$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>blacklist</td></td><td>Fields to exclude. This takes precedence over the whitelist.</td></td><td>list</td></td><td>""</td></td><td></td></td><td>medium</td></td></tr>
<tr>
<td>renames</td></td><td>Field rename mappings.</td></td><td>list</td></td><td>""</td></td><td>list of colon-delimited pairs, e.g. <code>foo:bar,abc:xyz</code></td></td><td>medium</td></td></tr>
<tr>
<td>whitelist</td></td><td>Fields to include. If specified, only these fields will be used.</td></td><td>list</td></td><td>""</td></td><td></td></td><td>medium</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.MaskField">
<h5>org.apache.kafka.connect.transforms.MaskField</h5>
Mask specified fields with a valid null value for the field type (i.e. 0, false, empty string, and so on).<p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.MaskField$Key</code>) or value (<code>org.apache.kafka.connect.transforms.MaskField$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>fields</td></td><td>Names of fields to mask.</td></td><td>list</td></td><td></td></td><td>non-empty list</td></td><td>high</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.ValueToKey">
<h5>org.apache.kafka.connect.transforms.ValueToKey</h5>
Replace the record key with a new key formed from a subset of fields in the record value.
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>fields</td></td><td>Field names on the record value to extract as the record key.</td></td><td>list</td></td><td></td></td><td>non-empty list</td></td><td>high</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.HoistField">
<h5>org.apache.kafka.connect.transforms.HoistField</h5>
Wrap data using the specified field name in a Struct when schema present, or a Map in the case of schemaless data.<p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.HoistField$Key</code>) or value (<code>org.apache.kafka.connect.transforms.HoistField$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>field</td></td><td>Field name for the single field that will be created in the resulting Struct or Map.</td></td><td>string</td></td><td></td></td><td></td></td><td>medium</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.ExtractField">
<h5>org.apache.kafka.connect.transforms.ExtractField</h5>
Extract the specified field from a Struct when schema present, or a Map in the case of schemaless data. Any null values are passed through unmodified.<p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.ExtractField$Key</code>) or value (<code>org.apache.kafka.connect.transforms.ExtractField$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>field</td></td><td>Field name to extract.</td></td><td>string</td></td><td></td></td><td></td></td><td>medium</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.SetSchemaMetadata">
<h5>org.apache.kafka.connect.transforms.SetSchemaMetadata</h5>
Set the schema name, version or both on the record's key (<code>org.apache.kafka.connect.transforms.SetSchemaMetadata$Key</code>) or value (<code>org.apache.kafka.connect.transforms.SetSchemaMetadata$Value</code>) schema.
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>schema.name</td></td><td>Schema name to set.</td></td><td>string</td></td><td>null</td></td><td></td></td><td>high</td></td></tr>
<tr>
<td>schema.version</td></td><td>Schema version to set.</td></td><td>int</td></td><td>null</td></td><td></td></td><td>high</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.TimestampRouter">
<h5>org.apache.kafka.connect.transforms.TimestampRouter</h5>
Update the record's topic field as a function of the original topic value and the record timestamp.<p/>This is mainly useful for sink connectors, since the topic field is often used to determine the equivalent entity name in the destination system(e.g. database table or search index name).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>timestamp.format</td></td><td>Format string for the timestamp that is compatible with <code>java.text.SimpleDateFormat</code>.</td></td><td>string</td></td><td>yyyyMMdd</td></td><td></td></td><td>high</td></td></tr>
<tr>
<td>topic.format</td></td><td>Format string which can contain <code>${topic}</code> and <code>${timestamp}</code> as placeholders for the topic and timestamp, respectively.</td></td><td>string</td></td><td>${topic}-${timestamp}</td></td><td></td></td><td>high</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.RegexRouter">
<h5>org.apache.kafka.connect.transforms.RegexRouter</h5>
Update the record topic using the configured regular expression and replacement string.<p/>Under the hood, the regex is compiled to a <code>java.util.regex.Pattern</code>. If the pattern matches the input topic, <code>java.util.regex.Matcher#replaceFirst()</code> is used with the replacement string to obtain the new topic.
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>regex</td></td><td>Regular expression to use for matching.</td></td><td>string</td></td><td></td></td><td>valid regex</td></td><td>high</td></td></tr>
<tr>
<td>replacement</td></td><td>Replacement string.</td></td><td>string</td></td><td></td></td><td></td></td><td>high</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.Flatten">
<h5>org.apache.kafka.connect.transforms.Flatten</h5>
Flatten a nested data structure, generating names for each field by concatenating the field names at each level with a configurable delimiter character. Applies to Struct when schema present, or a Map in the case of schemaless data. The default delimiter is '.'.<p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.Flatten$Key</code>) or value (<code>org.apache.kafka.connect.transforms.Flatten$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>delimiter</td></td><td>Delimiter to insert between field names from the input record when generating field names for the output record</td></td><td>string</td></td><td>.</td></td><td></td></td><td>medium</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.Cast">
<h5>org.apache.kafka.connect.transforms.Cast</h5>
Cast fields or the entire key or value to a specific type, e.g. to force an integer field to a smaller width. Only simple primitive types are supported -- integers, floats, boolean, and string. <p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.Cast$Key</code>) or value (<code>org.apache.kafka.connect.transforms.Cast$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>spec</td></td><td>List of fields and the type to cast them to of the form field1:type,field2:type to cast fields of Maps or Structs. A single type to cast the entire value. Valid types are int8, int16, int32, int64, float32, float64, boolean, and string.</td></td><td>list</td></td><td></td></td><td>list of colon-delimited pairs, e.g. <code>foo:bar,abc:xyz</code></td></td><td>high</td></td></tr>
</tbody></table>
</div>
<div id="org.apache.kafka.connect.transforms.TimestampConverter">
<h5>org.apache.kafka.connect.transforms.TimestampConverter</h5>
Convert timestamps between different formats such as Unix epoch, strings, and Connect Date/Timestamp types.Applies to individual fields or to the entire value.<p/>Use the concrete transformation type designed for the record key (<code>org.apache.kafka.connect.transforms.TimestampConverter$Key</code>) or value (<code>org.apache.kafka.connect.transforms.TimestampConverter$Value</code>).
<p/>
<table class="data-table"><tbody>
<tr>
<th>Name</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
<th>Valid Values</th>
<th>Importance</th>
</tr>
<tr>
<td>target.type</td></td><td>The desired timestamp representation: string, unix, Date, Time, or Timestamp</td></td><td>string</td></td><td></td></td><td></td></td><td>high</td></td></tr>
<tr>
<td>field</td></td><td>The field containing the timestamp, or empty if the entire value is a timestamp</td></td><td>string</td></td><td>""</td></td><td></td></td><td>high</td></td></tr>
<tr>
<td>format</td></td><td>A SimpleDateFormat-compatible format for the timestamp. Used to generate the output when type=string or used to parse the input if the input is a string.</td></td><td>string</td></td><td>""</td></td><td></td></td><td>medium</td></td></tr>
</tbody></table>
</div>