Before deploying a Pulsar cluster, you need to prepare Kubernetes resources and then continue with the following steps.
Specify how to run Pulsar using Helm's --set option.name=value command line option. In each section, collect the options that are combined to use with the helm install command.
By default, the Pulsar Helm Chart is installed in a namespace called pulsar.
namespace: pulsar
To install the Pulsar Helm Chart into a different Kubernetes namespace, you can include this option in the helm install command.
--set namespace=<different-k8s-namespace>
By default, the Pulsar Helm Chart doesn't create the namespace.
namespaceCreate: false
To use the Pulsar Helm Chart to create the Kubernetes namespace automatically, you can include this option in the helm install command.
--set namespaceCreate=true
By default, the Pulsar Helm Chart creates Volume Claims with the expectation that a dynamic provisioner creates the underlying Persistent Volumes.
volumes: persistence: true
:::note
Before installing the production instance of Pulsar, ensure to plan the storage settings to avoid extra storage migration work. Because after initial installation, you must edit Kubernetes objects manually if you want to change storage settings.
:::
The Pulsar Helm Chart is designed for production use. To use the Pulsar Helm Chart in a development environment (such as Minikube), you can disable persistence by including this option in your helm install command.
--set volumes.persistence=false
By default, anti-affinity is enabled to ensure pods of the same component can run on different nodes.
affinity: anti_affinity: true
To use the Pulsar Helm Chart in a development environment (such as Minikube), you can disable anti-affinity by including this option in your helm install command.
--set affinity.anti_affinity=false
The Pulsar Helm Chart is designed for production usage. It deploys a production-ready Pulsar cluster, including Pulsar core components and monitoring components.
You can customize the components to be deployed by turning on/off individual components.
## Components ## ## Control what components of Apache Pulsar to deploy for the cluster components: # zookeeper zookeeper: true # bookkeeper bookkeeper: true # bookkeeper - autorecovery autorecovery: true # broker broker: true # functions functions: true # proxy proxy: true # toolset toolset: true # pulsar manager pulsar_manager: true
The Pulsar Helm Chart installs monitoring components using a dependent Helm chart, kube-prometheus-stack. You can customize this Helm chart to specify which monitoring components to install. These components are enabled by default.
## Monitoring Components ## ## Control what components of the kube-prometheus-stack Helm chart to deploy for the cluster kube-prometheus-stack: # Control deployment of this Helm chart entirely enabled: true # prometheus prometheus: enabled: true promtheus-node-exporter: enabled: true # grafana grafana: enabled: true
The Pulsar Helm Chart is designed to enable controlled upgrades. So it can configure independent image versions for components. You can customize the images by setting individual components.
## Images ## ## Control what images to use for each component images: zookeeper: repository: apachepulsar/pulsar-all tag: latest pullPolicy: IfNotPresent bookie: repository: apachepulsar/pulsar-all tag: latest pullPolicy: IfNotPresent autorecovery: repository: apachepulsar/pulsar-all tag: latest pullPolicy: IfNotPresent broker: repository: apachepulsar/pulsar-all tag: latest pullPolicy: IfNotPresent proxy: repository: apachepulsar/pulsar-all tag: latest pullPolicy: IfNotPresent functions: repository: apachepulsar/pulsar-all tag: latest pulsar_manager: repository: apachepulsar/pulsar-manager tag: v0.3.0 pullPolicy: IfNotPresent hasCommand: false
The Pulsar Helm Chart also lets you specify the image versions used by initialization containers used to coordinate creation and connection of dependent Pulsar resources.
## Images ## ## Control what image to use for Pulsar init containers pulsar_metadata: component: pulsar-init image: repository: apachepulsar/pulsar-all tag: latest pullPolicy: IfNotPresent
:::tip
If using a private Docker repository or pull-thru cache, the repository configuration option must be changed accordingly for all component defintions including the pulsar_metadata component.
:::
The Pulsar Helm Chart can be configured to enable TLS (Transport Layer Security) to protect all the traffic between components. Before enabling TLS, you have to provision TLS certificates for the required components.
To use the cert-manager to provision the TLS certificates, you have to install the cert-manager before installing the Pulsar Helm Chart. After successfully installing the cert-manager, you can set certs.internal_issuer.enabled to true. Therefore, the Pulsar Helm Chart can use the cert-manager to generate selfsigning TLS certificates for the configured components.
certs: internal_issuer: enabled: false component: internal-cert-issuer type: selfsigning
You can also customize the generated TLS certificates by configuring the fields as the following.
tls: # common settings for generating certs common: # 90d duration: 2160h # 15d renewBefore: 360h organization: - pulsar keySize: 4096 keyAlgorithm: rsa keyEncoding: pkcs8
After installing the cert-manager, you can set tls.enabled to true to enable TLS encryption for the entire cluster.
tls: enabled: false
You can also configure whether to enable TLS encryption for individual components.
tls: # settings for generating certs for proxy proxy: enabled: false cert_name: tls-proxy # settings for generating certs for broker broker: enabled: false cert_name: tls-broker # settings for generating certs for bookies bookie: enabled: false cert_name: tls-bookie # settings for generating certs for zookeeper zookeeper: enabled: false cert_name: tls-zookeeper # settings for generating certs for recovery autorecovery: cert_name: tls-recovery # settings for generating certs for toolset toolset: cert_name: tls-toolset
By default, authentication is disabled. You can set auth.authentication.enabled to true to enable authentication. Currently, the Pulsar Helm Chart only supports the JWT authentication provider. You can set auth.authentication.provider to jwt to use the JWT authentication provider.
# Enable or disable broker authentication and authorization. auth: authentication: enabled: false provider: "jwt" jwt: # Enable JWT authentication # If the token is generated by a secret key, set the usingSecretKey as true. # If the token is generated by a private key, set the usingSecretKey as false. usingSecretKey: false superUsers: # broker to broker communication broker: "broker-admin" # proxy to broker communication proxy: "proxy-admin" # pulsar-admin client to broker/proxy communication client: "admin"
To enable authentication, you can run prepare helm release to generate token secret keys and tokens for three super users specified in the auth.superUsers field. The generated token keys and super user tokens are uploaded and stored as Kubernetes secrets prefixed with <pulsar-release-name>-token-. You can use the following command to find those secrets.
kubectl get secrets -n <k8s-namespace>
By default, authorization is disabled. Authorization can be enabled only when authentication is enabled.
auth: authorization: enabled: false
To enable authorization, you can include this option in the helm install command.
--set auth.authorization.enabled=true
By default, the resource requests and the number of replicas for the Pulsar components in the Pulsar Helm Chart are adequate for small production deployment. If you deploy a non-production instance, you can reduce the defaults to fit into a smaller cluster.
Once you have all of your configuration options collected, you can install dependent charts before installing the Pulsar Helm Chart.
For more information about storage provisioner, refer to Kubernetes documentation. Note that you need to create a storage class for your Kubernetes cluster and configure the storage class name in the Helm Chart.
If you want to use local persistent volumes as the persistent storage, you need to install a local storage provisioner. Here are two options:
The Pulsar Helm Chart uses the cert-manager to provision and manage TLS certificates automatically. To enable TLS encryption for brokers or proxies, you need to install the cert-manager in advance.
For details about how to install the cert-manager, follow the official instructions.
Alternatively, we provide a bash script install-cert-manager.sh to install a cert-manager release to the namespace cert-manager.
git clone https://github.com/apache/pulsar-helm-chart cd pulsar-helm-chart ./scripts/cert-manager/install-cert-manager.sh
Once you have installed all the dependent charts and collected all of your configuration options, you can run prepare_helm_release.sh to prepare the Helm release.
git clone https://github.com/apache/pulsar-helm-chart cd pulsar-helm-chart ./scripts/pulsar/prepare_helm_release.sh -n <k8s-namespace> -k <helm-release-name>
The prepare_helm_release creates the following resources:
broker-admin, proxy-admin, and admin. By default, it generates an asymmetric public/private key pair. You can choose to generate a symmetric secret key by specifying --symmetric.broker-admin role is used for inter-broker communications.proxy-admin role is used for proxies to communicate with brokers.admin role is used by the admin tools.Once you have finished the above steps, you can install a Helm release.
In this example, the Helm release is named pulsar.
helm repo add apache https://pulsar.apache.org/charts helm repo update helm install pulsar apache/pulsar \ --timeout 10m \ --set [your configuration options]
You can also use the --version <installation version> option if you want to install a specific version of Pulsar Helm Chart.
:::tip
A list of installed resources is output once the Pulsar cluster is deployed. This may take 5-10 minutes.
To check the status of the deployment, run the helm status pulsar command. It can also be done while the deployment is taking place if you run the command in another terminal.
:::
The default values will create a ClusterIP for the following resources, which you can use to interact with the cluster.
http://<pulsar-manager-ip>:9527.http://<grafana-dashboard-ip>:3000.To find the IP addresses of those components, run the following command:
kubectl get service -n <k8s-namespace>
You can configure the Proxy and the Pulsar Manager as a NodePort instead of a ClusterIP.
proxy: service: type: NodePort pulsar_manager: service: type: NodePort
Although we have done our best to make these charts as seamless as possible, troubles do go out of our control occasionally. We have been collecting tips and tricks for troubleshooting common issues. Check it first before raising an issue, and feel free to add your solutions by creating a Pull Request.
To uninstall the Pulsar Helm Chart, run the following command:
helm uninstall <pulsar-release-name>
For the purposes of continuity, some Kubernetes objects in these charts cannot be removed by using the helm uninstall command. It is recommended to consciously remove these items, as they affect re-deployment.