Druid relies on dynamic config providers to supply multiple related sets of credentials, secrets, and configurations within a Druid extension. Dynamic config providers are intended to eventually replace PasswordProvider.
By default, Druid includes an environment variable dynamic config provider that supports Kafka consumer configuration in Kafka ingestion.
To develop a custom extension of the DynamicConfigProvider
interface that is registered at Druid process startup, see Adding a new DynamicConfigProvider implementation.
You can use the environment variable dynamic config provider (EnvironmentVariableDynamicConfigProvider
) to store passwords or other sensitive information using system environment variables instead of plain text configuration.
The environment variable dynamic config provider uses the following syntax:
druid.dynamic.config.provider={"type": "environment","variables":{"secret1": "SECRET1_VAR","secret2": "SECRET2_VAR"}}
Field | Type | Description | Required |
---|---|---|---|
type | String | dynamic config provider type | Yes: environment |
variables | Map | environment variables that store the configuration information | Yes |
When using the environment variable config provider, consider the following:
The following example shows how to configure environment variables to store the SSL key and truststore passwords for Kafka.
On the Overlord and Peon machines, set the following environment variables for the system user that runs the Druid services:
export SSL_KEY_PASSWORD=mysecretkeypassword export SSL_KEYSTORE_PASSWORD=mysecretkeystorepassword export SSL_TRUSTSTORE_PASSWORD=mysecrettruststorepassword
When you define the consumer properties in the supervisor spec, use the dynamic config provider to refer to the environment variables:
... "consumerProperties": { "bootstrap.servers": "localhost:9092", "ssl.keystore.location": "/opt/kafka/config/kafka01.keystore.jks", "ssl.truststore.location": "/opt/kafka/config/kafka.truststore.jks", "druid.dynamic.config.provider": { "type": "environment", "variables": { "ssl.key.password": "SSL_KEY_PASSWORD", "ssl.keystore.password": "SSL_KEYSTORE_PASSWORD", "ssl.truststore.password": "SSL_TRUSTSTORE_PASSWORD" } } }, ...
When connecting to Kafka, Druid replaces the environment variables with their corresponding values.