Pulsar supports a pluggable authentication mechanism and a broker can be configured to support multiple authentication sources.
The role of the Authentication provider implementation is to establish the identity of the client, in the form of a role token. This role token is then used to verify whether this client is allowed to publish or consume on a certain topic.
In addition to provide connection encryption between pulsar clients and broker, TLS can be used to identify the client through a certificate signed by a trusted certificate authority.
Note: Unlike the rest of the Pulsar code, the TLS auth provider is not being used in production at Yahoo. Please report any issues encountered in using it.
The first step is to create the certificate for the CA. The CA will be used to signed both the broker and clients certificates, to ensure each party will trust the others.
# On Linux systems: $ CA.pl -newca # On MacOSX $ /System/Library/OpenSSL/misc/CA.pl -newca
After answering the questions, this will leave the CA related files under ./demoCA directory
demoCA/cacert.pem is the public certificate. It is meant to be distributed to all the parties involved.demoCA/private/cakey.pem is the private key. This is only needed when signing a new certificate for either broker or clients and it must be guarded safely.Now we can create certificate requests and sign them with the CA.
These commands will ask a few questions and create the certificates. When asked for the common name, you need to match the hostname of the broker. You could also use a wildcard to match a group of broker hostnames, eg: *.broker.usw.example.com, so that the same certificate can be reused in multiple machines.
$ openssl req -newkey rsa:2048 -sha256 -nodes -out broker-cert.csr -outform PEM # Convert key to PKCS#8 format $ openssl pkcs8 -topk8 -inform PEM -outform PEM -in privkey.pem -out broker-key.pem -nocrypt
This will create a broker certificate request files named broker-cert.csr and broker-key.pem.
We can now proceed to create the signed certificate:
$ openssl ca -out broker-cert.pem -infiles broker-cert.csr
At this point, we have the broker-cert.pem and broker-key.pem that are needed for the broker.
Repeat the same steps did for the broker and create a client-cert.pem and client-key.pem files.
For client common name you need to use a string you intend to use as the role token for this client, though it doesn't need to match the client hostname.
To configure TLS authentication in Pulsar broker in conf/broker.conf:
tlsEnabled=true tlsCertificateFilePath=/path/to/broker-cert.pem tlsKeyFilePath=/path/to/broker-key.pem tlsTrustCertsFilePath=/path/to/cacert.pem # Add TLS auth provider authenticationEnabled=true authorizationEnabled=true authenticationProviders=com.yahoo.pulsar.broker.authentication.AuthenticationProviderTls
Since discovery service is redirecting the HTTPS requests, it needs to be trusted by the client as well. Add TLS configuration in conf/discovery.conf:
tlsEnabled=true tlsCertificateFilePath=/path/to/broker-cert.pem tlsKeyFilePath=/path/to/broker-key.pem
ClientConfiguration conf = new ClientConfiguration(); conf.setUseTls(true); conf.setTlsTrustCertsFilePath("/path/to/cacert.pem"); Map<String, String> authParams = new HashMap<>(); authParams.put("tlsCertFile", "/path/to/client-cert.pem"); authParams.put("tlsKeyFile", "/path/to/client-key.pem"); conf.setAuthentication(AuthenticationTls.class.getName(), authParams); PulsarClient client = PulsarClient.create( "pulsar+ssl://my-broker.com:6651", conf);
ClientConfiguration config = ClientConfiguration(); config.setUseTls(true); std::string certfile = "/path/to/cacert.pem"; ParamMap params; params["tlsCertFile"] = "/path/to/client-cert.pem"; params["tlsKeyFile"] = "/path/to/client-key.pem"; config.setTlsTrustCertsFilePath(certfile); config.setTlsAllowInsecureConnection(false); AuthenticationPtr auth = pulsar::AuthFactory::create("/path/to/libauthtls.so", params); config.setAuth(auth); Client client("pulsar+ssl://my-broker.com:6651",config);
Command line tools like pulsar-admin, pulsar-perf and pulsar-client use the conf/client.conf config file and we can add there the authentication parameters:
serviceUrl=https://broker.example.com:8443/ authPlugin=com.yahoo.pulsar.client.impl.auth.AuthenticationTls authParams=tlsCertFile:/path/to/client-cert.pem,tlsKeyFile:/path/to/client-key.pem useTls=true tlsAllowInsecureConnection=false tlsTrustCertsFilePath=/path/to/cacert.pem
Athenz is one of the role-based authentication/authorization systems. In Pulsar, Athenz RoleToken (also called Z-Token) can be used as the identity of the client.
Assume that we have the decentralized Athenz system, i.e., there is not only the ZMS server but also the ZTS server.
At first we set up the service access control in Athenz. We should create domains for the provider (which provides some resources to other services with some authentication/authorization policies) and the tenant (which is provisioned to access some resources in a provider). In this case, provider corresponds to the Pulsar service itself and tenant corrensponds to each application using Pulsar (typically, Property on Pulsar).
For the tenant side, we should
Note that the private key generated in the step 2 needs to be specified when the pulsar client connects to the Broker (see the latter client configuration examples).
For more specific steps using UI, please refer example_service_athenz_setup.md#client-tenant-domain.
For the provider side, we should
Note that in step 2, any action and resource can be specified since they are not used on Pulsar. (In other words, Pulsar uses Athenz RoleToken for only authentication, not authorization).
For more specific steps using UI, please refer example_service_athenz_setup.md#server-provider-domain.
The class name of the Athenz authentication provider and comma separated provider domain names need to be specified in conf/broker.conf. Note that using TLS encryption is strongly recommended to protect RoleTokens against intercepting and reusing (see also data_model).
# Add Athenz auth provider authenticationEnabled=true authorizationEnabled=true authenticationProviders=com.yahoo.pulsar.broker.authentication.AuthenticationProviderAthenz athenzDomainNames=pulsar # Enable TLS tlsEnabled=true tlsCertificateFilePath=/path/to/broker-cert.pem tlsKeyFilePath=/path/to/broker-key.pem
4 parameters: tenantDomain, tenantService, providerDomain, privateKeyPath and an optional parameter keyId need to be configured. Note again that TLS is recommended.
ClientConfiguration conf = new ClientConfiguration(); // Enable TLS conf.setUseTls(true); conf.setTlsTrustCertsFilePath("/path/to/cacert.pem"); // Set Athenz auth plugin and its parameters Map<String, String> authParams = new HashMap<>(); authParams.put("tenantDomain", "shopping"); // Tenant domain name authParams.put("tenantService", "some_app"); // Tenant service name authParams.put("providerDomain", "pulsar"); // Provider domain name authParams.put("privateKeyPath", "/path/to/private.pem"); // Tenant private key path authParams.put("keyId", "v1"); // Key id for the tenant private key (optional, default: "0") conf.setAuthentication(AuthenticationAthenz.class.getName(), authParams); PulsarClient client = PulsarClient.create( "pulsar+ssl://my-broker.com:6651", conf);
Command line tools like pulsar-admin, pulsar-perf and pulsar-client use the conf/client.conf config file and we can add there the authentication parameters:
serviceUrl=https://broker.example.com:8443/ # Set Athenz auth plugin and its parameters authPlugin=com.yahoo.pulsar.client.impl.auth.AuthenticationAthenz authParams=tenantDomain:shopping,tenantService:some_app,providerDomain:pulsar,privateKeyPath:/path/to/private.pem,keyId:v1 # Enable TLS useTls=true tlsAllowInsecureConnection=false tlsTrustCertsFilePath=/path/to/cacert.pem