This document provides an overview of Apache Druid security features, configuration instructions, and some best practices to secure Druid.
By default, security features in Druid are disabled, which simplifies the initial deployment experience. However, security features must be configured in a production deployment. These features include TLS, authentication, and authorization.
The following recommendations apply to the Druid cluster setup:
WARNING!
Druid administrators have the same OS permissions as the Unix user account running Druid. See Authentication and authorization model. If the Druid process is running under the OS root user account, then Druid administrators can read or write all files that the root account has access to, including sensitive files such as/etc/passwd
.
consumerProperties
field of KafkaSupervisorIngestionSpec
. See Environment variable dynamic config provider for more information.The following recommendations apply to the network where Druid runs:
The following recommendation applies to Druid's authorization and authentication model:
WRITE
permissions to any DATASOURCE
to trusted users. Druid's trust model assumes those users have the same privileges as the operating system user that runs the Druid console process. Additionally, users with WRITE
permissions can make changes to datasources and they have access to both task and supervisor update (POST) APIs which may affect ingestion.STATE READ
, STATE WRITE
, CONFIG WRITE
, and DATASOURCE WRITE
permissions to highly-trusted users. These permissions allow users to access resources on behalf of the Druid server process regardless of the datasource.Enabling TLS encrypts the traffic between external clients and the Druid cluster and traffic between services within the cluster.
Before you enable TLS in Druid, generate the KeyStore and truststore. When one Druid process, e.g. Broker, contacts another Druid process , e.g. Historical, the first service is a client for the second service, considered the server.
The client uses a trustStore that contains certificates trusted by the client. For example, the Broker.
The server uses a KeyStore that contains private keys and certificate chain used to securely identify itself.
The following example demonstrates how to use Java keytool to generate the KeyStore for the server and then create a trustStore to trust the key for the client:
keytool
command:$> keytool -keystore keystore.jks -alias druid -genkey -keyalg RSA
$> keytool -export -alias druid -keystore keystore.jks -rfc -file public.cert
$> keytool -import -file public.cert -alias druid -keystore truststore.jks
Druid uses Jetty as its embedded web server. See Configuring SSL/TLS KeyStores from the Jetty documentation.
WARNING: Do not use use self-signed certificates for production environments. Instead, rely on your current public key infrastructure to generate and distribute trusted keys.
Edit common.runtime.properties
for all Druid services on all nodes. Add or update the following TLS options. Restart the cluster when you are finished.
# Turn on TLS globally druid.enableTlsPort=true # Disable non-TLS communicatoins druid.enablePlaintextPort=false # For Druid processes acting as a client # Load simple-client-sslcontext to enable client side TLS # Add the following to extension load list druid.extensions.loadList=[......., "simple-client-sslcontext"] # Setup client side TLS druid.client.https.protocol=TLSv1.2 druid.client.https.trustStoreType=jks druid.client.https.trustStorePath=truststore.jks # replace with correct trustStore file druid.client.https.trustStorePassword=secret123 # replace with your own password # Setup server side TLS druid.server.https.keyStoreType=jks druid.server.https.keyStorePath=my-keystore.jks # replace with correct keyStore file druid.server.https.keyStorePassword=secret123 # replace with your own password druid.server.https.certAlias=druid
For more information, see TLS support and Simple SSLContext Provider Module.
You can configure authentication and authorization to control access to the Druid APIs. Then configure users, roles, and permissions, as described in the following sections. Make the configuration changes in the common.runtime.properties
file on all Druid servers in the cluster.
Within Druid's operating context, authenticators control the way user identities are verified. Authorizers employ user roles to relate authenticated users to the datasources they are permitted to access. You can set the finest-grained permissions on a per-datasource basis.
The following graphic depicts the course of request through the authentication process:
To authenticate requests in Druid, you configure an Authenticator. Authenticator extensions exist for HTTP basic authentication, LDAP, and Kerberos.
The following takes you through sample configuration steps for enabling basic auth:
druid-basic-security
extension to druid.extensions.loadList
in common.runtime.properties
. For the quickstart installation, for example, the properties file is at conf/druid/cluster/_common
:druid.extensions.loadList=["druid-basic-security", "druid-histogram", "druid-datasketches", "druid-kafka-indexing-service"]
An example configuration:
# Druid basic security druid.auth.authenticatorChain=["MyBasicMetadataAuthenticator"] druid.auth.authenticator.MyBasicMetadataAuthenticator.type=basic # Default password for 'admin' user, should be changed for production. druid.auth.authenticator.MyBasicMetadataAuthenticator.initialAdminPassword=password1 # Default password for internal 'druid_system' user, should be changed for production. druid.auth.authenticator.MyBasicMetadataAuthenticator.initialInternalClientPassword=password2 # Uses the metadata store for storing users, you can use authentication API to create new users and grant permissions druid.auth.authenticator.MyBasicMetadataAuthenticator.credentialsValidator.type=metadata # If true and the request credential doesn't exists in this credentials store, the request will proceed to next Authenticator in the chain. druid.auth.authenticator.MyBasicMetadataAuthenticator.skipOnFailure=false druid.auth.authenticator.MyBasicMetadataAuthenticator.authorizerName=MyBasicMetadataAuthorizer # Escalator druid.escalator.type=basic druid.escalator.internalClientUsername=druid_system druid.escalator.internalClientPassword=password2 druid.escalator.authorizerName=MyBasicMetadataAuthorizer druid.auth.authorizers=["MyBasicMetadataAuthorizer"] druid.auth.authorizer.MyBasicMetadataAuthorizer.type=basic
See Authentication and Authorization for more information about the Authenticator, Escalator, and Authorizer concepts. See Basic Security for more information about the extension used in the examples above, and Kerberos for Kerberos authentication.
After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator user
endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
The following diagram depicts the authorization model, and the relationship between users, roles, permissions, and resources.
The following steps walk through a sample setup procedure:
The default Coordinator API port is 8081 for non-TLS connections and 8281 for secured connections.
druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>
, replacing USERNAME with the new username you are trying to create. For example:curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname
If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates, you may choose to include the
insecure
curl option to forgo certificate checking for the curl command.
druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>/credentials
. For example:curl -u admin:password1 -H'Content-Type: application/json' -XPOST --data-binary @pass.json https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname/credentialsThe password is conveyed in the
pass.json
file in the following form:{ "password": "myname_password" }
druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>
. For example:curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname
druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>
. For example:curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole
druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>/roles/<ROLENAME>
. For example:curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname/roles/myrole | jq
druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>/permissions
. For example:curl -u admin:password1 -H'Content-Type: application/json' -XPOST --data-binary @perms.json https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole/permissionsThe payload of
perms.json
should be in the form:[ { "resource": { "name": "<PATTERN>", "type": "DATASOURCE" }, "action": "READ" }, { "resource": { "name": "STATE", "type": "STATE" }, "action": "READ" } ]
Note: Druid treats the resource name as a regular expression (regex). You can use a specific datasource name or regex to grant permissions for multiple datasources at a time.
As an alternative to using the basic metadata authenticator, you can use LDAP to authenticate users. The following steps provide an overview of the setup procedure. For more information on these settings, see Properties for LDAP user authentication.
In common.runtime.properties
, add LDAP to the authenticator chain in the order in which you want requests to be evaluated. For example:
# Druid basic security druid.auth.authenticatorChain=["ldap", "MyBasicMetadataAuthenticator"]
Configure LDAP settings in common.runtime.properties
as appropriate for your LDAP scheme and system. For example:
druid.auth.authenticator.ldap.type=basic druid.auth.authenticator.ldap.enableCacheNotifications=true druid.auth.authenticator.ldap.credentialsValidator.type=ldap druid.auth.authenticator.ldap.credentialsValidator.url=ldap://ad_host:389 druid.auth.authenticator.ldap.credentialsValidator.bindUser=ad_admin_user druid.auth.authenticator.ldap.credentialsValidator.bindPassword=ad_admin_password druid.auth.authenticator.ldap.credentialsValidator.baseDn=dc=example,dc=com druid.auth.authenticator.ldap.credentialsValidator.userSearch=(&(sAMAccountName=%s)(objectClass=user)) druid.auth.authenticator.ldap.credentialsValidator.userAttribute=sAMAccountName druid.auth.authenticator.ldap.authorizerName=ldapauth druid.escalator.type=basic druid.escalator.internalClientUsername=ad_interal_user druid.escalator.internalClientPassword=Welcome123 druid.escalator.authorizerName=ldapauth druid.auth.authorizers=["ldapauth"] druid.auth.authorizer.ldapauth.type=basic druid.auth.authorizer.ldapauth.initialAdminUser=<ad_initial_admin_user> druid.auth.authorizer.ldapauth.initialAdminRole=admin druid.auth.authorizer.ldapauth.roleProvider.type=ldap
Use the Druid API to create the group mapping and allocate initial roles. For example, using curl and given a group named group1
in the directory, run:
curl -i -v -H "Content-Type: application/json" -u internal -X POST -d @groupmap.json http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1map
The groupmap.json
file contents would be something like:
{ "name": "group1map", "groupPattern": "CN=group1,CN=Users,DC=example,DC=com", "roles": [ "readRole" ] }
Check if the group mapping is created successfully by executing the following API. This lists all group mappings.
curl -i -v -H "Content-Type: application/json" -u internal -X GET http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings
Alternatively, to check the details of a specific group mapping, use the following API:
curl -i -v -H "Content-Type: application/json" -u internal -X GET http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1map
To add additional roles to the group mapping, use the following API:
curl -i -v -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1/roles/<newrole>
Add the LDAP user to Druid. To add a user, use the following authentication API:
curl -i -v -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authentication/db/ldap/users/<ad_user>
Use the following command to assign the role to a user:
curl -i -v -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/users/<ad_user>/roles/<rolename>
Congratulations, you have configured permissions for user-assigned roles in Druid!
Within Druid's trust model there users can have different authorization levels:
Additionally, Druid operates according to the following principles:
From the inner most layer:
Note: Only grant the
DATASOURCE WRITE
to trusted users because they can act as the Druid process.
Within the cluster:
Cluster to deep storage:
Cluster to client:
allowAll authorizer
.