Drill provides a variety of ways for specifying credentials for storage plugins. Though all storage plugin credentials may be stored in Zookeeper, it may be unsafe to specify them directly in the plugin configs.
Here is the example for specifying storage plugin credentials directly:
{ "type": "jdbc", "driver": "xxx.Driver", "url": "jdbc:xxx:xxx", "username": "xxx", "password": "xxx" }
Drill provides credentialsProvider
property for specifying desired credential provider type and its configs instead of holding raw credentials in storage plugin configs.
One of the implementations for credentials provider is HadoopCredentialsProvider
that allows using Hadoop Configuration property values as credentials. To use it, credentialsProviderType
property should be set to HadoopCredentialsProvider
:
{ "type": "jdbc", "driver": "xxx.Driver", "url": "jdbc:xxx:xxx", "credentialsProvider": { "credentialsProviderType": "HadoopCredentialsProvider", "propertyNames": { "username": "hadoop.user.property", "password": "hadoop.password.property" } } }
propertyNames
map contains keys that specify which credential will be obtained from the Hadoop Configuration property with the name of the propertyNames
value.
For example, user may create in drill config directory core-site.xml
config file with the following content:
<configuration> <property> <name>hadoop.user.property</name> <value>user1</value> </property> <property> <name>hadoop.password.property</name> <value>user1Pass</value> </property> </configuration>
In this case, the storage jdbc
plugin will use user1
value as the username
and user1Pass
value as its password
.
EnvCredentialsProvider
credentials provider implementation allows using Environment Variable values as plugin credentials. This way for specifying credentials is consistent with the Kubernetes world, where different user secrets and configmaps may be exposed as environment variables to be used by a container.
To use it, credentialsProviderType
property should be set to EnvCredentialsProvider
:
{ "type": "jdbc", "driver": "xxx.Driver", "url": "jdbc:xxx:xxx", "credentialsProvider": { "credentialsProviderType": "EnvCredentialsProvider", "envVariableNames": { "username": "USER_NAME", "password": "USER_PASSWORD" } } }
propertyNames
map contains keys that specify which credential will be obtained from the Environment Variable value with the name of the propertyNames
value.
For example, user may export the following variables:
export USER_NAME='user1' export USER_PASSWORD='user1Pass'
In this case, the storage jdbc
plugin will use user1
value as the username
and user1Pass
value as its password
.
VaultCredentialsProvider
credentials provider implementation allows using Vault secrets as plugin credentials. Currently, this credentials provider authenticates itself to Vault using the AppRole auth method which is intended for use by applications and services. In future, it may be able to use the Vault token of the Drill query user instead, in the event that the VaultUserAuthenticator
is also in use.
Before using this credential provider, the following Drill properties should be configured in drill-override.conf
:
"drill.exec.storage.vault.address" - host name or address of the Vault server. "drill.exec.storage.vault.app_role_id" - the role ID belonging to the AppRole Drill will use "drill.exec.storage.vault.secret_id" - the secret ID belonging to the AppRole Drill will use
Note that you will generally need to create and assign a Vault policy to grant the AppRole used by Drill read access to Vault secrets.
Once it is set, we can configure storage plugin to use this way of obtaining credentials:
{ "type": "jdbc", "driver": "xxx.Driver", "url": "jdbc:xxx:xxx", "credentialsProvider": { "credentialsProviderType": "VaultCredentialsProvider", "secretPath": "secret/jdbc", "propertyNames": { "username": "usernameSecret", "password": "passwordSecret" } } }
secretPath
property specifies the Vault key value from which to read. If the plugin's authMode
is set to user_translation
then the secretPath
may include a variable named $user
which will be replaced with the Drill query username at query execution time. propertyNames
map contains keys that specify which credential will be obtained from the Vault secret with the secret name of the propertyNames
value.
For example, user may store the following secrets in the Vault:
vault kv put secret/jdbc usernameSecret=muser passwordSecret=mpassword
In this case, the storage jdbc
plugin will use user1
value as the username
and user1Pass
value as its password
.
To be consistent with credentials providers implementations, Drill provides a new way of specifying credentials directly in storage plugin config:
{ "type": "jdbc", "driver": "xxx.Driver", "url": "jdbc:xxx:xxx", "credentialsProvider": { "credentialsProviderType": "PlainCredentialsProvider", "credentials": { "username": "user1", "password": "user1Pass" } } }
This way of specifying credentials directly should be used instead of the old one since it groups credentials and makes it easier to replace PlainCredentialsProvider
with a more secured alternative.
CredentialsProvider
is a base interface for all credential provider implementations. Users may create and use their own credential provider implementations without changing Drill code. To achieve that, just add dependency on the drill-logical
jar (where CredentialsProvider
interface is placed), create own implementation of this interface, and create drill-module.conf
file that will add implementation class to Drill classpath scanning, for example if the class will have the following full name: foo.bar.package.ExampleCredentialsProvider
, drill-module.conf
content should be the following:
drill.classpath.scanning: { packages += "foo.bar.package" }