blob: 2e1ae4d74c653f9ba1422cecc53e716d49313501 [file] [log] [blame] [view]
---
title: Monitoring
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
There are different ways to monitor a Pulsar cluster, exposing both metrics relative to the
usage of topics and the overall health of the individual components of the cluster.
## Collecting metrics
### Broker stats
The [`pulsar-admin`](../../reference/CliTools#pulsar-admin) tool
Pulsar {% popover broker %} metrics can be collected from brokers and exported in JSON format. There are two main types of metrics:
* *Destination dumps*, which containing stats for each individual topic. They can be fetched using
```shell
bin/pulsar-admin broker-stats destinations
```
* Broker metrics, containing broker info and topics stats aggregated at namespace
level:
```shell
bin/pulsar-admin broker-stats monitoring-metrics
```
All the message rates are updated every 1min.
The aggregated broker metrics are also exposed in the [Prometheus](https://prometheus.io) format at:
```shell
http://$BROKER_ADDRESS:8080/metrics
```
### ZooKeeper stats
The local/global ZooKeeper server and clients that are shipped with Pulsar have been instrumented to expose
detailed stats through Prometheus as well.
```shell
http://$LOCAL_ZK_SERVER:8000/metrics
http://$GLOBAL_ZK_SERVER:8001/metrics
```
The default port of local ZooKeeper is `8000` and that of global ZooKeeper is `8001`.
These can be changed by specifying system property `stats_server_port`.
### BookKeeper stats
For BookKeeper you can configure the stats frameworks by changing the `statsProviderClass` in
`conf/bookkeeper.conf`.
By default, the default BookKeeper configuration included with Pulsar distribution will enable
the Prometheus exporter.
```shell
http://$BOOKIE_ADDRESS:8000/metrics
```
For bookies, the default port is `8000` (instead of `8080`) and that can be configured by changing
the `prometheusStatsHttpPort` in `conf/bookkeeper.conf`.
## Configuring Prometheus
You can configure Prometheus to collect and store the metrics data by following the Prometheus
[Getting started](https://prometheus.io/docs/introduction/getting_started/) guide.
When running on bare metal, you can provide the list of nodes that needs to be probed. When deploying
in a Kubernetes cluster, the monitoring is automatically setup with the [provided](../../deployment/Kubernetes)
instructions.
## Dashboards
When collecting time series statistics, the major problem is to make sure the number of dimensions
attached to the data does not explode.
For that reason we only collect time series of metrics aggregated at the namespace level.
### Pulsar per-topic dashboard
The per-topic dashboard instructions are available at [Dashboard](../../admin/Dashboard).
### Grafana
You can use grafana to easily create dashboard driven by the data stored in Prometheus.
There is a `pulsar-grafana` Docker image that is ready to use with the principal dashboards already
in place. This is enabled by default when deploying Pulsar on Kubernetes.
To use the dashboard manually:
```shell
docker run -p3000:3000 \
-e PROMETHEUS_URL=http://$PROMETHEUS_HOST:9090/ \
apachepulsar/pulsar-grafana:latest
```