This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including
pulsar-admin
For deploying a Pulsar cluster for production usage, please read the documentation on how to configure and install a Pulsar Helm chart.
:::tip
For the following steps, step 2 and step 3 are for developers and step 4 and step 5 are for administrators.
:::
Before installing a Pulsar Helm chart, you have to create a Kubernetes cluster. You can follow the instructions to prepare a Kubernetes cluster.
We use Minikube in this quick start guide.
Create a kubernetes cluster on Minikube.
minikube start --memory=8192 --cpus=4 --kubernetes-version=<k8s-version>
The <k8s-version>
can be any Kubernetes version supported by your minikube installation. Example: `v1.16.1.
Set kubectl
to use Minikube.
kubectl config use-context minikube
In order to use the Kubernetes Dashboard with local Kubernetes cluster on Minikube, enter the command below:
minikube dashboard
The command automatically triggers opening a webpage in your browser.
Clone the Pulsar Helm chart repository.
git clone https://github.com/apache/pulsar-helm-chart cd pulsar-helm-chart
Run prepare_helm_release.sh
to create secrets required for installing Apache Pulsar Helm chart. The username pulsar
and password pulsar
are used for logging into Grafana dashboard and Pulsar Manager.
./scripts/pulsar/prepare_helm_release.sh \ -n pulsar \ -k pulsar-mini \ --control-center-admin pulsar \ --control-center-password pulsar \ -c
Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
helm install \ --values examples/values-minikube.yaml \ pulsar-mini charts/pulsar
Check the status of all pods.
kubectl get pods -n pulsar
If all pods start up successfully, you can see STATUS
changes to Running
or Completed
.
Output
NAME READY STATUS RESTARTS AGE pulsar-mini-bookie-0 1/1 Running 0 9m27s pulsar-mini-bookie-init-5gphs 0/1 Completed 0 9m27s pulsar-mini-broker-0 1/1 Running 0 9m27s pulsar-mini-grafana-6b7bcc64c7-4tkxd 1/1 Running 0 9m27s pulsar-mini-prometheus-5fcf5dd84c-w8mgz 1/1 Running 0 9m27s pulsar-mini-proxy-0 1/1 Running 0 9m27s pulsar-mini-pulsar-init-t7cqt 0/1 Completed 0 9m27s pulsar-mini-pulsar-manager-9bcbb4d9f-htpcs 1/1 Running 0 9m27s pulsar-mini-toolset-0 1/1 Running 0 9m27s pulsar-mini-zookeeper-0 1/1 Running 0 9m27s
Check the status of all services in the namespace pulsar
.
kubectl get services -n pulsar
Output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE pulsar-mini-bookie ClusterIP None <none> 3181/TCP,8000/TCP 11m pulsar-mini-broker ClusterIP None <none> 8080/TCP,6650/TCP 11m pulsar-mini-grafana LoadBalancer 10.106.141.246 <pending> 3000:31905/TCP 11m pulsar-mini-prometheus ClusterIP None <none> 9090/TCP 11m pulsar-mini-proxy LoadBalancer 10.97.240.109 <pending> 80:32305/TCP,6650:31816/TCP 11m pulsar-mini-pulsar-manager LoadBalancer 10.103.192.175 <pending> 9527:30190/TCP 11m pulsar-mini-toolset ClusterIP None <none> <none> 11m pulsar-mini-zookeeper ClusterIP None <none> 2888/TCP,3888/TCP,2181/TCP 11m
pulsar-admin
is the CLI tool for Pulsar. In this step, you can use pulsar-admin
to create resources including tenants, namespaces, and topics.
Enter the toolset
container.
kubectl exec -it -n pulsar pulsar-mini-toolset-0 -- /bin/bash
In the toolset
container, create a tenant named apache
.
bin/pulsar-admin tenants create apache
Then you can list the tenants to see if the tenant is created successfully.
bin/pulsar-admin tenants list
You should see a similar output as below. The tenant apache
has been successfully created.
"apache" "public" "pulsar"
In the toolset
container, create a namespace named pulsar
in the tenant apache
.
bin/pulsar-admin namespaces create apache/pulsar
Then you can list the namespaces of tenant apache
to see if the namespace is created successfully.
bin/pulsar-admin namespaces list apache
You should see a similar output as below. The namespace apache/pulsar
has been successfully created.
"apache/pulsar"
In the toolset
container, create a topic test-topic
with 4
partitions in the namespace apache/pulsar
.
bin/pulsar-admin topics create-partitioned-topic apache/pulsar/test-topic -p 4
In the toolset
container, list all the partitioned topics in the namespace apache/pulsar
.
bin/pulsar-admin topics list-partitioned-topics apache/pulsar
Then you can see all the partitioned topics in the namespace apache/pulsar
.
"persistent://apache/pulsar/test-topic"
You can use the Pulsar client to create producers and consumers to produce and consume messages.
By default the Helm chart expose the Pulsar cluster through a Kubernetes LoadBalancer
. In Minikube, you can use the following command to get the IP address of the proxy service.
kubectl get services -n pulsar | grep pulsar-mini-proxy
You will see a similar output as below.
pulsar-mini-proxy LoadBalancer 10.97.240.109 <pending> 80:32305/TCP,6650:31816/TCP 28m
This output tells what are the node ports that Pulsar cluster's binary port and http port are exposed to. The port after 80:
is the http port while the port after 6650:
is the binary port.
Then you can find the ip address of your minikube server by running the following command.
minikube ip
At this point, you will get the service urls to connect to your Pulsar client.
webServiceUrl=http://$(minikube ip):<exposed-http-port>/ brokerServiceUrl=pulsar://$(minikube ip):<exposed-binary-port>/
Then proceed with the following steps:
Download the Apache Pulsar tarball from downloads page.
Decompress the tarball based on your download file.
tar -xf <file-name>.tar.gz
Expose PULSAR_HOME
.
(1) Enter the directory of the decompressed download file.
(2) Expose PULSAR_HOME
as the environment variable.
export PULSAR_HOME=$(pwd)
Configure the Pulsar client.
In the ${PULSAR_HOME}/conf/client.conf
file, replace webServiceUrl
and brokerServiceUrl
with the service urls you get from the above steps.
Create a subscription to consume messages from apache/pulsar/test-topic
.
bin/pulsar-client consume -s sub apache/pulsar/test-topic -n 0
Open a new terminal. In the new terminal, create a producer and send 10 messages to the test-topic
topic.
bin/pulsar-client produce apache/pulsar/test-topic -m "---------hello apache pulsar-------" -n 10
Verify the results.
From producer side
Output
The messages have been produced successfully.
18:15:15.489 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 10 messages successfully produced
From consumer side
Output
At the same time, you can receive the messages as below.
----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar------- ----- got message ----- ---------hello apache pulsar-------
Pulsar Manager is a web-based GUI management tool for managing and monitoring Pulsar.
By default, the Pulsar Manager
is exposed as a separate LoadBalancer
. You can open the Pulsar Manager UI using the following command:
minikube service pulsar-mini-pulsar-mananger
The pulsar manager UI will be open in your browser. You can use username pulsar
and password pulsar
to log into Pulsar Manager.
In Pulsar Manager UI, you can create an environment.
New Environment
button in the top-left corner.pulsar-mini
for the field Environment Name
in the popup window.http://pulsar-mini-broker:8080
for the field Service URL
in the popup window.Confirm
button in the popup window.After successfully creating an environment, you will be redirected to the tenants
page of that environment. Then you can create tenants
, namespaces
and topics
using Pulsar Manager.
Grafana is an open-source visualization tool, which can be used for visualizing time series data into dashboards.
By default, the Grafana is exposed as a separate LoadBalancer
. You can open the Grafana UI using the following command:
minikube service pulsar-mini-grafana -n pulsar
The Grafana UI will be open in your browser. You can use username pulsar
and password pulsar
to log into Grafana Dashboard.
You will be able to view dashboards for different components of a Pulsar cluster.