Google Git
Sign in
apache / pulsar-helm-chart / refs/tags/pulsar-3.0.0
tagd58e602d5412df1fc2e97636ce88e1e7e0d6ae10
taggerMichael Marshall <mmarshall@apache.org>Thu Nov 03 16:32:29 2022 -0500
objectc8f5e7f9ce596cbc6343feccb8cbe7e698803d8d 
Apache Pulsar Helm Chart
commitc8f5e7f9ce596cbc6343feccb8cbe7e698803d8d[log] [tgz]
authorMichael Marshall <mmarshall@apache.org>Thu Oct 20 22:33:08 2022 -0700
committerGitHub <noreply@github.com>Fri Oct 21 00:33:08 2022 -0500
tree6e75deb4eef3bfc0eca6e1a4142c87bd41db0128
parent687060aa27f61e91c4ddbc4c09a234115846ba5d [diff]
New Release Process (#301)

* Copy release process doc from Apache Airflow

Source: https://github.com/apache/airflow/blob/fb741fd87254e235f99d7d67e558dafad601f253/dev/README_RELEASE_HELM_CHART.md

* Adapt to Apache Pulsar

* Remove old release process notes

* Fix typo

* Apply suggestions from code review

Co-authored-by: tison <wander4096@gmail.com>

* Add sign.sh script for release artifacts

Script is copied from https://github.com/apache/airflow/blob/395ad7110e53a30a5d33f648d1dd797482eb268c/dev/sign.sh

* Add some updates (more to might follow)

* Add some more updates to the rest of the release plan

* Fix rat check command

Co-authored-by: tison <wander4096@gmail.com>

Relates to #290

### Motivation

Make the Apache Pulsar Helm Chart release follow ASF rules for voting, and make the helm binary available via dist.apache.org. By following the information in https://issues.apache.org/jira/browse/LEGAL-573 and in the Apache Airflow project https://github.com/apache/airflow/blob/main/dev/README_RELEASE_HELM_CHART.md, I built this new release process. It will likely need some iterative improvement.

### Modifications

* Add a release process that is based on the Apache Airflow release process

### Verifying this change

- [ ] Make sure that the change passes the CI checks.
3 files changed
tree: 6e75deb4eef3bfc0eca6e1a4142c87bd41db0128
  1. .ci/
  2. .github/
  3. charts/
  4. examples/
  5. hack/
  6. scripts/
  7. .asf.yaml
  8. .gitignore
  9. .rat-excludes
  10. LICENSE
  11. license_test.go
  12. NOTICE
  13. README.md
  14. RELEASE.md
  15. Vagrantfile
README.md

Official Apache Pulsar Helm Chart

This is the officially supported Helm Chart for installing Apache Pulsar on Kubernetes.

Read Deploying Pulsar on Kubernetes for more details.

Features

This Helm Chart includes all the components of Apache Pulsar for a complete experience.

  • [x] Pulsar core components:
    • [x] ZooKeeper
    • [x] Bookies
    • [x] Brokers
    • [x] Functions
    • [x] Proxies
  • [x] Management & monitoring components:
    • [x] Pulsar Manager
    • [x] Optional PodMonitors for each component (enabled by default)
    • [x] Kube-Prometheus-Stack (as of 3.0.0)

It includes support for:

  • [x] Security
    • [x] Automatically provisioned TLS certs, using Jetstack's cert-manager
    • [x] TLS Encryption
      • [x] Proxy
      • [x] Broker
      • [x] Toolset
      • [x] Bookie
      • [x] ZooKeeper
    • [x] Authentication
      • [x] JWT
      • [ ] Mutal TLS
      • [ ] Kerberos
    • [x] Authorization
    • [x] Non-root broker, bookkeeper, proxy, and zookeeper containers (version 2.10.0 and above)
  • [x] Storage
    • [x] Non-persistence storage
    • [x] Persistence Volume
    • [x] Local Persistent Volumes
    • [ ] Tiered Storage
  • [x] Functions
    • [x] Kubernetes Runtime
    • [x] Process Runtime
    • [x] Thread Runtime
  • [x] Operations
    • [x] Independent Image Versions for all components, enabling controlled upgrades

Requirements

In order to use this chart to deploy Apache Pulsar on Kubernetes, the followings are required.

  1. kubectl 1.18 or higher, compatible with your cluster (+/- 1 minor release from your cluster)
  2. Helm v3 (3.0.2 or higher)
  3. A Kubernetes cluster, version 1.18 or higher.

Environment setup

Before proceeding to deploying Pulsar, you need to prepare your environment.

Tools

helm and kubectl need to be installed on your computer.

Add to local Helm repository

To add this chart to your local Helm repository:

helm repo add apache https://pulsar.apache.org/charts

Kubernetes cluster preparation

You need a Kubernetes cluster whose version is 1.18 or higher in order to use this chart, due to the usage of certain Kubernetes features.

We provide some instructions to guide you through the preparation: http://pulsar.apache.org/docs/helm-prepare/

Deploy Pulsar to Kubernetes

  1. Configure your values file. The best way to know which values are available is to read the values.yaml.

  2. Install the chart:

    helm install <release-name> -n <namespace> -f your-values.yaml apache/pulsar

  3. Access the Pulsar cluster

    The default values will create a ClusterIP for the proxy you can use to interact with the cluster. To find the IP address of proxy use:

    kubectl get service -n <k8s-namespace>

For more information, please follow our detailed quick start guide.

Customize the deployment

We provide a detailed guideline for you to customize the Helm Chart for a production-ready deployment.

You can also checkout out the example values file for different deployments.

Disabling Kube-Prometheus-Stack CRDs

In order to disable the kube-prometheus-stack fully, it is necessary to add the following to your values.yaml:

kube-prometheus-stack:
  enabled: false
  prometheusOperator:
    enabled: false
  grafana:
    enabled: false
  alertmanager:
    enabled: false
  prometheus:
    enabled: false

Otherwise, the helm chart installation will attempt to install the CRDs for the kube-prometheus-stack. Additionally, you‘ll need to disable each of the component’s PodMonitors. This is shown in some examples and is verified in some tests.

Grafana Dashboards

The Apache Pulsar Helm Chart uses the kube-prometheus-stack Helm Chart to deploy Grafana. Dashboards are loaded via a Kubernetes ConfigMap. Please see their documentation for loading those dashboards.

The apache/pulsar GitHub repo contains some dashboards here.

Third Party Dashboards

Note: if you have third party dashboards that you would like included in this list, please open a pull request.

Upgrading

Once your Pulsar Chart is installed, configuration changes and chart updates should be done using helm upgrade.

helm repo add apache https://pulsar.apache.org/charts
helm repo update
helm get values <pulsar-release-name> > pulsar.yaml
helm upgrade -f pulsar.yaml \
    <pulsar-release-name> apache/pulsar

For more detailed information, see our Upgrading guide.

Upgrading to 2.10.0 and above

The 2.10.0+ Apache Pulsar docker image is a non-root container, by default. That complicates an upgrade to 2.10.0 because the existing files are owned by the root user but are not writable by the root group. In order to leverage this new security feature, the Bookkeeper and Zookeeper StatefulSet securityContexts are configurable in the values.yaml. They default to:

  securityContext:
    fsGroup: 0
    fsGroupChangePolicy: "OnRootMismatch"

This configuration is ideal for regular Kubernetes clusters where the UID is stable across restarts. If the process UID is subject to change (like it is in OpenShift), you'll need to set fsGroupChangePolicy: "Always".

The official docker image assumes that it is run as a member of the root group.

If you upgrade to the latest version of the helm chart before upgrading to Pulsar 2.10.0, then when you perform your first upgrade to version >= 2.10.0, you will need to set fsGroupChangePolicy: "Always" on the first upgrade and then set it back to fsGroupChangePolicy: "OnRootMismatch" on subsequent upgrades. This is because the root file won't mismatch permissions, but the RocksDB lock file will. If you have direct access to the persistent volumes, you can alternatively run chgrp -R g+w /pulsar/data before upgrading.

Here is a sample error you can expect if the RocksDB lock file is not correctly owned by the root group:

2022-05-14T03:45:06,903+0000  ERROR org.apache.bookkeeper.server.Main - Failed to build bookie server
java.io.IOException: Error open RocksDB database
    at org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.<init>(KeyValueStorageRocksDB.java:199) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.<init>(KeyValueStorageRocksDB.java:88) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.lambda$static$0(KeyValueStorageRocksDB.java:62) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.<init>(LedgerMetadataIndex.java:68) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.<init>(SingleDirectoryDbLedgerStorage.java:169) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.newSingleDirectoryDbLedgerStorage(DbLedgerStorage.java:150) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize(DbLedgerStorage.java:129) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.bookie.Bookie.<init>(Bookie.java:818) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:152) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.proto.BookieServer.<init>(BookieServer.java:120) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.server.service.BookieService.<init>(BookieService.java:52) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:304) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.server.Main.doMain(Main.java:226) [org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
Caused by: org.rocksdb.RocksDBException: while open a file for lock: /pulsar/data/bookkeeper/ledgers/current/ledgers/LOCK: Permission denied
    at org.rocksdb.RocksDB.open(Native Method) ~[org.rocksdb-rocksdbjni-6.10.2.jar:?]
    at org.rocksdb.RocksDB.open(RocksDB.java:239) ~[org.rocksdb-rocksdbjni-6.10.2.jar:?]
    at org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.<init>(KeyValueStorageRocksDB.java:196) ~[org.apache.bookkeeper-bookkeeper-server-4.14.4.jar:4.14.4]
    ... 13 more

Uninstall

To uninstall the Pulsar Chart, run the following command:

helm delete <pulsar-release-name>

For the purposes of continuity, these charts have some Kubernetes objects that are not removed when performing helm delete. These items we require you to conciously remove them, as they affect re-deployment should you choose to.

  • PVCs for stateful data, which you must consciously remove
    • ZooKeeper: This is your metadata.
    • BookKeeper: This is your data.
    • Prometheus: This is your metrics data, which can be safely removed.
  • Secrets, if generated by our prepare release script. They contain secret keys, tokens, etc. You can use cleanup release script to remove these secrets and tokens as needed.

Troubleshooting

We‘ve done our best to make these charts as seamless as possible, occasionally troubles do surface outside of our control. We’ve collected tips and tricks for troubleshooting common issues. Please examine these first before raising an issue, and feel free to add to them by raising a Pull Request!

Release Process

See RELEASE.md