id: io-redis-sink title: Redis sink connector sidebar_label: “Redis sink connector”

:::note

You can download all the Pulsar connectors on download page.

:::

The Redis sink connector pulls messages from Pulsar topics and persists the messages to a Redis database.

Configuration

The configuration of the Redis sink connector has the following properties.

Property

NameTypeRequiredDefaultDescription
redisHostsStringtrue" " (empty string)A comma-separated list of Redis hosts to connect to.
redisPasswordStringfalse" " (empty string)The password used to connect to Redis.
redisDatabaseinttrue0The Redis database to connect to.
clientModeStringfalseStandaloneThe client mode when interacting with Redis cluster.

Below are the available options:
Standalone
Cluster
autoReconnectbooleanfalsetrueWhether the Redis client automatically reconnect or not.
requestQueueintfalse2147483647The maximum number of queued requests to Redis.
tcpNoDelaybooleanfalsefalseWhether to enable TCP with no delay or not.
keepAlivebooleanfalsefalseWhether to enable a keepalive to Redis or not.
connectTimeoutlongfalse10000The time to wait before timing out when connecting in milliseconds.
operationTimeoutlongfalse10000The time before an operation is marked as timed out in milliseconds .
batchTimeMsintfalse1000The Redis operation time in milliseconds.
batchSizeintfalse200The batch size of writing to Redis database.

Example

Before using the Redis sink connector, you need to create a configuration file in the path you will start the Pulsar service (PULSAR_HOME) through one of the following methods.

  • JSON

    {
       "configs": {
          "redisHosts": "localhost:6379",
          "redisPassword": "mypassword",
          "redisDatabase": "0",
          "clientMode": "Standalone",
          "operationTimeout": "2000",
          "batchSize": "1",
          "batchTimeMs": "1000",
          "connectTimeout": "3000"
       }
    }
    
  • YAML

    configs:
        redisHosts: "localhost:6379"
        redisPassword: "mypassword"
        redisDatabase: 0
        clientMode: "Standalone"
        operationTimeout: 2000
        batchSize: 1
        batchTimeMs: 1000
        connectTimeout: 3000
    

Usage

This example shows how to write records to a Redis database using the Pulsar Redis connector.

  1. Start a Redis server.

    docker pull redis:5.0.5
    docker run -d -p 6379:6379 --name my-redis redis:5.0.5 --requirepass "mypassword"
    
  2. Start a Pulsar service locally in standalone mode.

    bin/pulsar standalone
    

    Make sure the NAR file is available at connectors/pulsar-io-redis-@pulsar:version@.nar.

  3. Start the Pulsar Redis connector in local run mode using one of the following methods.

    • Use the JSON configuration file as shown previously.
    bin/pulsar-admin sinks localrun \
        --archive connectors/pulsar-io-redis-@pulsar:version@.nar \
        --tenant public \
        --namespace default \
        --name my-redis-sink \
        --sink-config '{"redisHosts": "localhost:6379","redisPassword": "mypassword","redisDatabase": "0","clientMode": "Standalone","operationTimeout": "3000","batchSize": "1"}' \
        --inputs my-redis-topic
    
    • Use the YAML configuration file as shown previously.
    bin/pulsar-admin sinks localrun \
        --archive connectors/pulsar-io-redis-@pulsar:version@.nar \
        --tenant public \
        --namespace default \
        --name my-redis-sink \
        --sink-config-file redis-sink-config.yaml \
        --inputs my-redis-topic
    
  4. Publish records to the topic.

    bin/pulsar-client produce \
        persistent://public/default/my-redis-topic \
        -k "streaming" \
        -m "Pulsar"
    
  5. Start a Redis client in Docker.

    docker exec -it my-redis redis-cli -a "mypassword"
    
  6. Check the key/value in Redis.

    127.0.0.1:6379> keys *
    1) "streaming"
    127.0.0.1:6379> get "streaming"
    "Pulsar"