Certificate role based authorization in Apache Bookkeeper. (#2429)

This feature allows a predefined set of services to be 'whitelisted' to be able to access bookkeeper based on their client certificates. _This feature is disabled by default._


**Motivation**
As BookKeeper and its supported services move to a cloud friendly service based architecture, it becomes of utmost importance to monitor and allow only certain qualified services to be able to access the data in BK.
We have TLS based authentication, however, any service with the rootCA can access Bookkeeper clusters which is not desirable.

**Changes**

To enable if, we have to set 2 configuration options in server config.

1. Set `bookieAuthProviderFactoryClass` config option to use BookieAuthZFactory
`bookieAuthProviderFactoryClass=org.apache.bookkeeper.tls.BookieAuthZFactory`

2. Set `authorizedRoles` to a comma separated list of roles present in client certificates' OU field.
`authorizedRoles=pulsar-broker-1,pulsar-broker-2`

Read further for details on how to implement these in your client certificates and how to wire it up.

So this feature can be broken down into two parts:

    Certificate and roles
    Server configuration for authorized roles

**Details:**
_Certificate and roles:_
Here is an example of how the SUBJECT field of a final certificate for Apache Pulsar running in the cloud would look like:

    CN=apache.bookkeeper.org
    O=apache-pulsar
    OU=0:pulsar-broker-role;1:cluster-1
    L=San Francisco
    S=CA
    C=US

This shows that this bookkeeper client certificate is owned by the apache pulsar service has the role ‘pulsar-broker-role’ for entities in ‘cluster-1’.
Only those services with pulsar-broker-role should be able to access it.
We can add more fields separated by commas to increase the upstream application clusters to be able to access this bookkeeper cluster.

For example: `OU=0:herddb-readonlyNode,herddb-readwriteNode;1:herddb-cluster2`

Such separation of access based on services is paramount to keeping this secure as many upstream users of BookKeeper are financial institutions, databases and other services.

_Server configuration for authorized roles_
Once we have a certificate whose SUBJECT field has the OU attribute with the roles we want to authorize, on the Bookie side, we need to specify which roles are allowed.
We make this happen by introducing a server configuration option called `authorizedRoles`.
Since we have only static options, this will be set in stone as long as the bookie booted up with it.
If in case we need to change the allowed roles, we’ll need to stop the bookie, update the configuration file and then restart the bookie.
We can have multiple roles which are authorized as the OU field can have multiple comma separated values for roles.

This is a redo of stale PR #2355 

Master Issue: #2354
17 files changed
tree: 4335f36b94a6b6b22e4b261372c95d0b4b5791a9
  1. .github/
  2. .test-infra/
  3. bin/
  4. bookkeeper-benchmark/
  5. bookkeeper-common/
  6. bookkeeper-common-allocator/
  7. bookkeeper-dist/
  8. bookkeeper-http/
  9. bookkeeper-proto/
  10. bookkeeper-server/
  11. bookkeeper-stats/
  12. bookkeeper-stats-providers/
  13. buildtools/
  14. circe-checksum/
  15. conf/
  16. cpu-affinity/
  17. deploy/
  18. dev/
  19. docker/
  20. metadata-drivers/
  21. microbenchmarks/
  22. shaded/
  23. site/
  24. site2/
  25. stats/
  26. stream/
  27. tests/
  28. tools/
  29. .gitignore
  30. LICENSE
  31. NOTICE
  32. pom.xml
  33. README.md
README.md

Build Status Build Status Coverage Status Maven Central

Apache BookKeeper

Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.

It is suitable for being used in following scenarios:

  • WAL (Write-Ahead-Logging), e.g. HDFS NameNode.
  • Message Store, e.g. Apache Pulsar.
  • Offset/Cursor Store, e.g. Apache Pulsar.
  • Object/Blob Store, e.g. storing state machine snapshots.

Get Started

  • Concepts: Start with the basic concepts of Apache BookKeeper. This will help you to fully understand the other parts of the documentation.
  • Getting Started to setup BookKeeper to write logs.

Documentation

Developers

You can also read Turning Ledgers into Logs to learn how to turn ledgers into continuous log streams. If you are looking for a high level log stream API, you can checkout DistributedLog.

Administrators

Contributors

Get In Touch

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.

Need Help?

Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.

Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.

Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.

Contributing

We feel that a welcoming open community is important and welcome contributions.

Contributing Code

  1. See Developer Setup to get your local environment setup.

  2. Take a look at our open issues: JIRA Issues Github Issues.

  3. Review our coding style and follow our pull requests to learn about our conventions.

  4. Make your changes according to our contribution guide.

Improving Website and Documentation

  1. See Building the website and documentation on how to build the website and documentation.