Assert sink connector
A flink sink plugin which can assert illegal data by user defined rules
name | type | required | default value |
---|---|---|---|
rules | ConfigMap | yes | - |
rules.field_rules | string | yes | - |
rules.field_rules.field_name | string | yes | - |
rules.field_rules.field_type | string | no | - |
rules.field_rules.field_value | ConfigList | no | - |
rules.field_rules.field_value.rule_type | string | no | - |
rules.field_rules.field_value.rule_value | double | no | - |
rules.row_rules | string | yes | - |
rules.row_rules.rule_type | string | no | - |
rules.row_rules.rule_value | string | no | - |
common-options | no | - |
Rule definition of user's available data. Each rule represents one field validation or row num validation.
field rules for field validation
field name(string)
field type (string), e.g. string,boolean,byte,short,int,long,float,double,char,void,BigInteger,BigDecimal,Instant
A list value rule define the data value validation
The following rules are supported for now
value can't be null
define the minimum value of data
define the maximum value of data
define the minimum string length of a string data
define the maximum string length of a string data
define the minimun number of rows
define the maximum number of rows
the value related to rule type
Sink plugin common parameters, please refer to Sink Common Options for details
the whole config obey with hocon
style
Assert { rules = { row_rules = [ { rule_type = MAX_ROW rule_value = 10 }, { rule_type = MIN_ROW rule_value = 5 } ], field_rules = [{ field_name = name field_type = string field_value = [ { rule_type = NOT_NULL }, { rule_type = MIN_LENGTH rule_value = 5 }, { rule_type = MAX_LENGTH rule_value = 10 } ] }, { field_name = age field_type = int field_value = [ { rule_type = NOT_NULL }, { rule_type = MIN rule_value = 32767 }, { rule_type = MAX rule_value = 2147483647 } ] } ] } }