inlong-sort-connector-redis

data-type

See detail: redis data type

PLAIN

Redis strings commands are used for managing string values in Redis

The first element is Redis row key,must be string type. The remaining fields(c2~c6) will be serialized into one value and put into redis

HASH

• A Redis hash is a data type that represents a mapping between a string field and a string value.
  
• There are two members in hash DataType:
  
• the first member is Redis hash field.
  
• the second member is Redis hash value.

SET

• Redis Lists are simply lists of strings, sorted by insertion order.
• You can add elements in Redis lists in the head or the tail of the list.

BITMAP

• Bitmaps are not an actual data type, but a set of bit-oriented operations defined on the String type.
  
• Since strings are binary safe blobs and their maximum length is 512 MB,
  
• they are suitable to set up to 2^32 different bits.

SchemaMappingMode

DYNAMIC

• The DYNAMIC mode witch mapping a {@link java.util.Map} to {@link RedisDataType}.
• There are two members in DYNAMIC mode:
  
• the first member is Redis key.
  
• the second member is a {@link java.util.Map} object, witch will be iterated,
  
• the entry key is redis key, and the entry value is redis value.

STATIC_PREFIX_MATCH

• The are at least two fields, the first member is redis key,
  
• and each field from the second field represents a redis value.
  
• ex:
  

key, field, value1, value2, value3, [value]...

STATIC_KV_PAIR;

• There are two fields, the first field is key,
  
• and the other fields are pairs of field and value.
  
• ex:
  

 key, field1, value1,field2,value2,field3,value3,[field,value]...

SQL demo

PLAIN

Plain only support STATIC_PREFIX_MATCH schema mapping mode

CREATE TABLE sink (
    key STRING,
    aaa STRING,
    bbb DOUBLE,    
    ccc BIGINT,    
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (  
    'connector' = 'redis-inlong',  
    'sink.batch-size' = '1',  
    'format' = 'csv',  
    'data-type' = 'PLAIN',  
    'redis-mode' = 'standalone',  
    'host' = 'localhost',  
    'port' = '56615',  
    'maxIdle' = '8',  
    'minIdle' = '1',  
    'maxTotal' = '2',  
    'timeout' = '2000'
);

HASH with PREFIX_MATCH

The first element is Redis row key, must be string type. The second element is Redis field name in Hash DataType. The remaining fields(c2~c7) will be serialized into one value and put into redis

CREATE TABLE sink (
    KEY STRING, 
    field_name STRING, 
    value_1 DOUBLE,
    value_2 BIGINT, 
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
   'connector' = 'redis-inlong',
   'sink.batch-size' = '1',
   'format' = 'csv',
   'data-type' = 'HASH',
   'redis-mode' = 'standalone',
   'host' = 'localhost',
   'port' = '56869',
   'maxIdle' = '8',
   'minIdle' = '1',
   'maxTotal' = '2',
   'timeout' = '2000'
);

HASH with STATIC_KV_PAIR

The first element is Redis row key, must be string type. The odd elements(c2 / c4 / c6) are Redis field names in Hash DataType, must be String type. The even elements(c3 / c5 / c7) are Redis field values in Hash DataType, must be String type.

CREATE TABLE sink (
    key STRING,
    field1 STRING,
    value1 STRING,
    field2 STRING,
    value2 STRING,
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
  'connector' = 'redis-inlong',
  'sink.batch-size' = '1',
  'format' = 'csv',
  'data-type' = 'HASH',
  'schema-mapping-mode' = 'STATIC_KV_PAIR',
  'redis-mode' = 'standalone',
  'host' = 'localhost',
  'port' = '6379',
  'maxIdle' = '8',
  'minIdle' = '1',
  'maxTotal' = '2',
  'timeout' = '2000'
);

HASH with DYNAMIC

The first element is Redis row key, must be string type. The second element is must be Map<String,String> type: key is fieldName, value is fieldValue.

CREATE TABLE sink (
    key STRING,
    fieldValueMap MAP<STRING,STRING>,
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
  'connector' = 'redis-inlong',
  'sink.batch-size' = '1',
  'format' = 'csv',
  'data-type' = 'HASH',
  'schema-mapping-mode' = 'DYNAMIC',
  'redis-mode' = 'standalone',
  'host' = 'localhost',
  'port' = '6379',
  'maxIdle' = '8',
  'minIdle' = '1',
  'maxTotal' = '2',
  'timeout' = '2000'
)"

BITMAP with STATIC_KV_PAIR

The first element is Redis row key, must be string type. The odd elements(c2 /c4 /c6 ) are Redis offsets in Bitmap DataType, must be Long type. The even elements(c3 / c5 / c7) are Redis values in Bitmap DataType, must be Boolean type.

CREATE TABLE sink (
    key STRING,
    offset_1 BIGINT,
    value_1 BOOLEAN,
    offset_2 BIGINT,
    value_2 BOOLEAN,
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
  'connector' = 'redis-inlong',
  'sink.batch-size' = '1',
  'format' = 'csv',
  'data-type' = 'BITMAP',
  'schema-mapping-mode' = 'STATIC_KV_PAIR',
  'redis-mode' = 'standalone',
  'host' = 'localhost',
  'port' = '6379',
  'maxIdle' = '8',
  'minIdle' = '1',
  'maxTotal' = '2',
  'timeout' = '2000'
)