id: reference-configuration title: Pulsar configuration sidebar_label: Pulsar configuration

You can manage Pulsar configuration by configuration files in the conf directory of a Pulsar installation.

BookKeeper

BookKeeper is a replicated log storage system that Pulsar uses for durable storage of all messages.

NameDescriptionDefault
bookiePortThe port on which the bookie server listens.3181
allowLoopbackWhether the bookie is allowed to use a loopback interface as its primary interface (that is the interface used to establish its identity). By default, loopback interfaces are not allowed to work as the primary interface. Using a loopback interface as the primary interface usually indicates a configuration error. For example, it’s fairly common in some VPS setups to not configure a hostname or to have the hostname resolve to 127.0.0.1. If this is the case, then all bookies in the cluster will establish their identities as 127.0.0.1:3181 and only one will be able to join the cluster. For VPSs configured like this, you should explicitly set the listening interface.false
listeningInterfaceThe network interface on which the bookie listens. By default, the bookie listens on all interfaces.eth0
advertisedAddressConfigure a specific hostname or IP address that the bookie should use to advertise itself to clients. By default, the bookie advertises either its own IP address or hostname according to the listeningInterface and useHostNameAsBookieID settings.N/A
allowMultipleDirsUnderSameDiskPartitionConfigure the bookie to enable/disable multiple ledger/index/journal directories in the same filesystem disk partition.false
minUsableSizeForIndexFileCreationThe minimum safe usable size available in index directory for bookie to create index files while replaying journal at the time of bookie starts in Readonly Mode (in bytes).1073741824
journalDirectoryThe directory where BookKeeper outputs its write-ahead log (WAL).data/bookkeeper/journal
journalDirectoriesDirectories that BookKeeper outputs its write ahead log. Multiple directories are available, being separated by ,. For example: journalDirectories=/tmp/bk-journal1,/tmp/bk-journal2. If journalDirectories is set, the bookies skip journalDirectory and use this setting directory./tmp/bk-journal
ledgerDirectoriesThe directory where BookKeeper outputs ledger snapshots. This could define multiple directories to store snapshots separated by ,, for example ledgerDirectories=/tmp/bk1-data,/tmp/bk2-data. Ideally, ledger dirs and the journal dir are each in a different device, which reduces the contention between random I/O and sequential write. It is possible to run with a single disk, but performance will be significantly lower.data/bookkeeper/ledgers
ledgerManagerTypeThe type of ledger manager used to manage how ledgers are stored, managed, and garbage collected. See BookKeeper Internals for more info.hierarchical
zkLedgersRootPathThe root ZooKeeper path used to store ledger metadata. This parameter is used by the ZooKeeper-based ledger manager as a root znode to store all ledgers./ledgers
ledgerStorageClassLedger storage implementation classorg.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage
entryLogFilePreallocationEnabledEnable or disable entry logger preallocationtrue
logSizeLimitMax file size of the entry logger, in bytes. A new entry log file will be created when the old one reaches the file size limitation.2147483648
minorCompactionThresholdThreshold of minor compaction. Entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If set to less than zero, the minor compaction is disabled.0.2
minorCompactionIntervalTime interval to run minor compaction, in seconds. If set to less than zero, the minor compaction is disabled. Note: should be greater than gcWaitTime.3600
majorCompactionThresholdThe threshold of major compaction. Entry log files whose remaining size percentage reaches below this threshold will be compacted in a major compaction. Those entry log files whose remaining size percentage is still higher than the threshold will never be compacted. If set to less than zero, the minor compaction is disabled.0.5
majorCompactionIntervalThe time interval to run major compaction, in seconds. If set to less than zero, the major compaction is disabled. Note: should be greater than gcWaitTime.86400
readOnlyModeEnabledIf readOnlyModeEnabled=true, then on all full ledger disks, bookie will be converted to read-only mode and serve only read requests. Otherwise the bookie will be shutdown.true
forceReadOnlyBookieWhether the bookie is force started in read only mode.false
persistBookieStatusEnabledPersist the bookie status locally on the disks. So the bookies can keep their status upon restarts.false
compactionMaxOutstandingRequestsSets the maximum number of entries that can be compacted without flushing. When compacting, the entries are written to the entrylog and the new offsets are cached in memory. Once the entrylog is flushed the index is updated with the new offsets. This parameter controls the number of entries added to the entrylog before a flush is forced. A higher value for this parameter means more memory will be used for offsets. Each offset consists of 3 longs. This parameter should not be modified unless you’re fully aware of the consequences.100000
compactionRateThe rate at which compaction will read entries, in adds per second.1000
isThrottleByBytesThrottle compaction by bytes or by entries.false
compactionRateByEntriesThe rate at which compaction will read entries, in adds per second.1000
compactionRateByBytesSet the rate at which compaction reads entries. The unit is bytes added per second.1000000
journalMaxSizeMBMax file size of journal file, in megabytes. A new journal file will be created when the old one reaches the file size limitation.2048
journalMaxBackupsThe max number of old journal files to keep. Keeping a number of old journal files would help data recovery in special cases.5
journalPreAllocSizeMBHow space to pre-allocate at a time in the journal.16
journalWriteBufferSizeKBThe of the write buffers used for the journal.64
journalRemoveFromPageCacheWhether pages should be removed from the page cache after force write.true
journalAdaptiveGroupWritesWhether to group journal force writes, which optimizes group commit for higher throughput.true
journalMaxGroupWaitMSecThe maximum latency to impose on a journal write to achieve grouping.1
journalAlignmentSizeAll the journal writes and commits should be aligned to given size4096
journalBufferedWritesThresholdMaximum writes to buffer to achieve grouping524288
journalFlushWhenQueueEmptyIf we should flush the journal when journal queue is emptyfalse
numJournalCallbackThreadsThe number of threads that should handle journal callbacks8
openLedgerRereplicationGracePeriodThe grace period, in milliseconds, that the replication worker waits before fencing and replicating a ledger fragment that's still being written to upon bookie failure.30000
rereplicationEntryBatchSizeThe number of max entries to keep in fragment for re-replication100
autoRecoveryDaemonEnabledWhether the bookie itself can start auto-recovery service.true
lostBookieRecoveryDelayHow long to wait, in seconds, before starting auto recovery of a lost bookie.0
gcWaitTimeHow long the interval to trigger next garbage collection, in milliseconds. Since garbage collection is running in background, too frequent gc will heart performance. It is better to give a higher number of gc interval if there is enough disk capacity.900000
gcOverreplicatedLedgerWaitTimeHow long the interval to trigger next garbage collection of overreplicated ledgers, in milliseconds. This should not be run very frequently since we read the metadata for all the ledgers on the bookie from zk.86400000
flushIntervalHow long the interval to flush ledger index pages to disk, in milliseconds. Flushing index files will introduce much random disk I/O. If separating journal dir and ledger dirs each on different devices, flushing would not affect performance. But if putting journal dir and ledger dirs on same device, performance degrade significantly on too frequent flushing. You can consider increment flush interval to get better performance, but you need to pay more time on bookie server restart after failure.60000
bookieDeathWatchIntervalInterval to watch whether bookie is dead or not, in milliseconds1000
allowStorageExpansionAllow the bookie storage to expand. Newly added ledger and index dirs must be empty.false
zkServersA list of one of more servers on which zookeeper is running. The server list can be comma separated values, for example: zkServers=zk1:2181,zk2:2181,zk3:2181.localhost:2181
zkTimeoutZooKeeper client session timeout in milliseconds Bookie server will exit if it received SESSION_EXPIRED because it was partitioned off from ZooKeeper for more than the session timeout JVM garbage collection, disk I/O will cause SESSION_EXPIRED. Increment this value could help avoiding this issue30000
zkRetryBackoffStartMsThe start time that the Zookeeper client backoff retries in milliseconds.1000
zkRetryBackoffMaxMsThe maximum time that the Zookeeper client backoff retries in milliseconds.10000
zkEnableSecuritySet ACLs on every node written on ZooKeeper, allowing users to read and write BookKeeper metadata stored on ZooKeeper. In order to make ACLs work you need to setup ZooKeeper JAAS authentication. All the bookies and Client need to share the same user, and this is usually done using Kerberos authentication. See ZooKeeper documentation.false
httpServerEnabledThe flag enables/disables starting the admin http server.false
httpServerPortThe HTTP server port to listen on. By default, the value is 8080. If you want to keep it consistent with the Prometheus stats provider, you can set it to 8000.8080
httpServerClassThe http server class.org.apache.bookkeeper.http.vertx.VertxHttpServer
serverTcpNoDelayThis settings is used to enabled/disabled Nagle’s algorithm, which is a means of improving the efficiency of TCP/IP networks by reducing the number of packets that need to be sent over the network. If you are sending many small messages, such that more than one can fit in a single IP packet, setting server.tcpnodelay to false to enable Nagle algorithm can provide better performance.true
serverSockKeepaliveThis setting is used to send keep-alive messages on connection-oriented sockets.true
serverTcpLingerThe socket linger timeout on close. When enabled, a close or shutdown will not return until all queued messages for the socket have been successfully sent or the linger timeout has been reached. Otherwise, the call returns immediately and the closing is done in the background.0
byteBufAllocatorSizeMaxThe maximum buf size of the received ByteBuf allocator.1048576
nettyMaxFrameSizeBytesThe maximum netty frame size in bytes. Any message received larger than this will be rejected.5253120
openFileLimitMax number of ledger index files could be opened in bookie server If number of ledger index files reaches this limitation, bookie server started to swap some ledgers from memory to disk. Too frequent swap will affect performance. You can tune this number to gain performance according your requirements.0
pageSizeSize of a index page in ledger cache, in bytes A larger index page can improve performance writing page to disk, which is efficient when you have small number of ledgers and these ledgers have similar number of entries. If you have large number of ledgers and each ledger has fewer entries, smaller index page would improve memory usage.8192
pageLimitHow many index pages provided in ledger cache If number of index pages reaches this limitation, bookie server starts to swap some ledgers from memory to disk. You can increment this value when you found swap became more frequent. But make sure pageLimit*pageSize should not more than JVM max memory limitation, otherwise you would got OutOfMemoryException. In general, incrementing pageLimit, using smaller index page would gain better performance in lager number of ledgers with fewer entries case If pageLimit is -1, bookie server will use 1/3 of JVM memory to compute the limitation of number of index pages.0
readOnlyModeEnabledIf all ledger directories configured are full, then support only read requests for clients. If “readOnlyModeEnabled=true” then on all ledger disks full, bookie will be converted to read-only mode and serve only read requests. Otherwise the bookie will be shutdown. By default this will be disabled.true
diskUsageThresholdFor each ledger dir, maximum disk space which can be used. Default is 0.95f. i.e. 95% of disk can be used at most after which nothing will be written to that partition. If all ledger dir partitions are full, then bookie will turn to readonly mode if ‘readOnlyModeEnabled=true’ is set, else it will shutdown. Valid values should be in between 0 and 1 (exclusive).0.95
diskCheckIntervalDisk check interval in milli seconds, interval to check the ledger dirs usage.10000
auditorPeriodicCheckIntervalInterval at which the auditor will do a check of all ledgers in the cluster. By default this runs once a week. The interval is set in seconds. To disable the periodic check completely, set this to 0. Note that periodic checking will put extra load on the cluster, so it should not be run more frequently than once a day.604800
sortedLedgerStorageEnabledWhether sorted-ledger storage is enabled.true
auditorPeriodicBookieCheckIntervalThe interval between auditor bookie checks. The auditor bookie check, checks ledger metadata to see which bookies should contain entries for each ledger. If a bookie which should contain entries is unavailable, thea the ledger containing that entry is marked for recovery. Setting this to 0 disabled the periodic check. Bookie checks will still run when a bookie fails. The interval is specified in seconds.86400
numAddWorkerThreadsThe number of threads that should handle write requests. if zero, the writes would be handled by netty threads directly.0
numReadWorkerThreadsThe number of threads that should handle read requests. if zero, the reads would be handled by netty threads directly.8
numHighPriorityWorkerThreadsThe umber of threads that should be used for high priority requests (i.e. recovery reads and adds, and fencing).8
maxPendingReadRequestsPerThreadIf read workers threads are enabled, limit the number of pending requests, to avoid the executor queue to grow indefinitely.2500
maxPendingAddRequestsPerThreadThe limited number of pending requests, which is used to avoid the executor queue to grow indefinitely when add workers threads are enabled.10000
isForceGCAllowWhenNoSpaceWhether force compaction is allowed when the disk is full or almost full. Forcing GC could get some space back, but could also fill up the disk space more quickly. This is because new log files are created before GC, while old garbage log files are deleted after GC.false
verifyMetadataOnGCTrue if the bookie should double check readMetadata prior to GC.false
flushEntrylogBytesEntry log flush interval in bytes. Flushing in smaller chunks but more frequently reduces spikes in disk I/O. Flushing too frequently may also affect performance negatively.268435456
readBufferSizeBytesThe number of bytes we should use as capacity for BufferedReadChannel.4096
writeBufferSizeBytesThe number of bytes used as capacity for the write buffer65536
useHostNameAsBookieIDWhether the bookie should use its hostname to register with the coordination service (e.g.: zookeeper service). When false, bookie will use its ip address for the registration.false
bookieIdIf you want to custom a bookie ID or use a dynamic network address for the bookie, you can set the bookieId.

Bookie advertises itself using the bookieId rather than the BookieSocketAddress (hostname:port or IP:port). If you set the bookieId, then the useHostNameAsBookieID does not take effect.

The bookieId is a non-empty string that can contain ASCII digits and letters ([a-zA-Z9-0]), colons, dashes, and dots.

For more information about bookieId, see here.
N/A
allowEphemeralPortsWhether the bookie is allowed to use an ephemeral port (port 0) as its server port. By default, an ephemeral port is not allowed. Using an ephemeral port as the service port usually indicates a configuration error. However, in unit tests, using an ephemeral port will address port conflict problems and allow running tests in parallel.false
enableLocalTransportWhether the bookie is allowed to listen for the BookKeeper clients executed on the local JVM.false
disableServerSocketBindWhether the bookie is allowed to disable bind on network interfaces. This bookie will be available only to BookKeeper clients executed on the local JVM.false
skipListArenaChunkSizeThe number of bytes that we should use as chunk allocation for org.apache.bookkeeper.bookie.SkipListArena.4194304
skipListArenaMaxAllocSizeThe maximum size that we should allocate from the skiplist arena. Allocations larger than this should be allocated directly by the VM to avoid fragmentation.131072
bookieAuthProviderFactoryClassThe factory class name of the bookie authentication provider. If this is null, then there is no authentication.null
statsProviderClassorg.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider
prometheusStatsHttpPort8000
dbStorage_writeCacheMaxSizeMbSize of Write Cache. Memory is allocated from JVM direct memory. Write cache is used to buffer entries before flushing into the entry log. For good performance, it should be big enough to hold a substantial amount of entries in the flush interval.25% of direct memory
dbStorage_readAheadCacheMaxSizeMbSize of Read cache. Memory is allocated from JVM direct memory. This read cache is pre-filled doing read-ahead whenever a cache miss happens. By default, it is allocated to 25% of the available direct memory.N/A
dbStorage_readAheadCacheBatchSizeHow many entries to pre-fill in cache after a read cache miss1000
dbStorage_rocksDB_blockCacheSizeSize of RocksDB block-cache. For best performance, this cache should be big enough to hold a significant portion of the index database which can reach ~2GB in some cases. By default, it uses 10% of direct memory.N/A
dbStorage_rocksDB_writeBufferSizeMB64
dbStorage_rocksDB_sstSizeInMB64
dbStorage_rocksDB_blockSize65536
dbStorage_rocksDB_bloomFilterBitsPerKey10
dbStorage_rocksDB_numLevels-1
dbStorage_rocksDB_numFilesInLevel04
dbStorage_rocksDB_maxSizeInLevel1MB256

Broker

Pulsar brokers are responsible for handling incoming messages from producers, dispatching messages to consumers, replicating data between clusters, and more.

NameDescriptionDefault
advertisedListenersSpecify multiple advertised listeners for the broker.

The format is <listener_name>:pulsar://<host>:<port>.

If there are multiple listeners, separate them with commas.

Note: do not use this configuration with advertisedAddress and brokerServicePort. If the value of this configuration is empty, the broker uses advertisedAddress and brokerServicePort
/
internalListenerNameSpecify the internal listener name for the broker.

Note: the listener name must be contained in advertisedListeners.

If the value of this configuration is empty, the broker uses the first listener as the internal listener.
/
authenticateOriginalAuthDataIf this flag is set to true, the broker authenticates the original Auth data; else it just accepts the originalPrincipal and authorizes it (if required).false
enablePersistentTopicsWhether persistent topics are enabled on the brokertrue
enableNonPersistentTopicsWhether non-persistent topics are enabled on the brokertrue
functionsWorkerEnabledWhether the Pulsar Functions worker service is enabled in the brokerfalse
exposePublisherStatsWhether to enable topic level metrics.true
statsUpdateFrequencyInSecs60
statsUpdateInitialDelayInSecs60
zookeeperServersZookeeper quorum connection string
zooKeeperCacheExpirySecondsZooKeeper cache expiry time in seconds300
configurationStoreServersConfiguration store connection string (as a comma-separated list)
brokerServicePortBroker data port6650
brokerServicePortTlsBroker data port for TLS6651
webServicePortPort to use to server HTTP request8080
webServicePortTlsPort to use to server HTTPS request8443
webSocketServiceEnabledEnable the WebSocket API service in brokerfalse
webSocketNumIoThreadsThe number of IO threads in Pulsar Client used in WebSocket proxy.Runtime.getRuntime().availableProcessors()
webSocketConnectionsPerBrokerThe number of connections per Broker in Pulsar Client used in WebSocket proxy.Runtime.getRuntime().availableProcessors()
webSocketSessionIdleTimeoutMillisTime in milliseconds that idle WebSocket session times out.300000
webSocketMaxTextFrameSizeThe maximum size of a text message during parsing in WebSocket proxy.1048576
exposeTopicLevelMetricsInPrometheusWhether to enable topic level metrics.true
exposeConsumerLevelMetricsInPrometheusWhether to enable consumer level metrics.false
jvmGCMetricsLoggerClassNameClassname of Pluggable JVM GC metrics logger that can log GC specific metrics.N/A
bindAddressHostname or IP address the service binds on, default is 0.0.0.0.0.0.0.0
bindAddressesAdditional Hostname or IP addresses the service binds on: listener_name:scheme://host:port,....
advertisedAddressHostname or IP address the service advertises to the outside world. If not set, the value of InetAddress.getLocalHost().getHostName() is used.
clusterNameName of the cluster to which this broker belongs to
brokerDeduplicationEnabledSets the default behavior for message deduplication in the broker. If enabled, the broker will reject messages that were already stored in the topic. This setting can be overridden on a per-namespace basis.false
brokerDeduplicationMaxNumberOfProducersThe maximum number of producers for which information will be stored for deduplication purposes.10000
brokerDeduplicationEntriesIntervalThe number of entries after which a deduplication informational snapshot is taken. A larger interval will lead to fewer snapshots being taken, though this would also lengthen the topic recovery time (the time required for entries published after the snapshot to be replayed).1000
brokerDeduplicationProducerInactivityTimeoutMinutesThe time of inactivity (in minutes) after which the broker will discard deduplication information related to a disconnected producer.360
brokerDeduplicationSnapshotFrequencyInSecondsHow often is the thread pool scheduled to check whether a snapshot needs to be taken. The value of 0 means it is disabled.120
dispatchThrottlingRateInMsgDispatch throttling-limit of messages for a broker (per second). 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingRateInByteDispatch throttling-limit of bytes for a broker (per second). 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingRatePerTopicInMsgDispatch throttling-limit of messages for every topic (per second). 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingRatePerTopicInByteDispatch throttling-limit of bytes for every topic (per second). 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingOnBatchMessageEnabledApply dispatch rate limiting on batch message instead individual messages with in batch message. (Default is disabled).false
dispatchThrottlingRateRelativeToPublishRateEnable dispatch rate-limiting relative to publish rate.false
dispatchThrottlingRatePerSubscriptionInMsgDispatch throttling-limit of messages for a subscription. 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingRatePerSubscriptionInByteDispatch throttling-limit of bytes for a subscription. 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingRatePerReplicatorInMsgThe default messages per second dispatch throttling-limit for every replicator in replication. The value of 0 means disabling replication message dispatch-throttling0
dispatchThrottlingRatePerReplicatorInByteThe default bytes per second dispatch throttling-limit for every replicator in replication. The value of 0 means disabling replication message-byte dispatch-throttling0
zooKeeperSessionTimeoutMillisZookeeper session timeout in milliseconds30000
brokerShutdownTimeoutMsTime to wait for broker graceful shutdown. After this time elapses, the process will be killed60000
skipBrokerShutdownOnOOMFlag to skip broker shutdown when broker handles Out of memory error.false
backlogQuotaCheckEnabledEnable backlog quota check. Enforces action on topic when the quota is reachedtrue
backlogQuotaCheckIntervalInSecondsHow often to check for topics that have reached the quota60
backlogQuotaDefaultLimitBytesThe default per-topic backlog quota limit. Being less than 0 means no limitation. By default, it is -1.-1
backlogQuotaDefaultRetentionPolicyThe defaulted backlog quota retention policy. By Default, it is producer_request_hold. ‘producer_request_hold’ Policy which holds producer‘s send request until the resource becomes available (or holding times out) ‘producer_exception’ Policy which throws javax.jms.ResourceAllocationException to the producer ‘consumer_backlog_eviction’ Policy which evicts the oldest message from the slowest consumer’s backlogproducer_request_hold
allowAutoTopicCreationEnable topic auto creation if a new producer or consumer connectedtrue
allowAutoTopicCreationTypeThe type of topic that is allowed to be automatically created.(partitioned/non-partitioned)non-partitioned
allowAutoSubscriptionCreationEnable subscription auto creation if a new consumer connectedtrue
defaultNumPartitionsThe number of partitioned topics that is allowed to be automatically created if allowAutoTopicCreationType is partitioned1
brokerDeleteInactiveTopicsEnabledEnable the deletion of inactive topics. If topics are not consumed for some while, these inactive topics might be cleaned up. Deleting inactive topics is enabled by default. The default period is 1 minute.true
brokerDeleteInactiveTopicsFrequencySecondsHow often to check for inactive topics60
brokerDeleteInactiveTopicsModeSet the mode to delete inactive topics. delete_when_no_subscriptions: delete the topic which has no subscriptions or active producers. delete_when_subscriptions_caught_up: delete the topic whose subscriptions have no backlogs and which has no active producers or consumers.delete_when_no_subscriptions
brokerDeleteInactiveTopicsMaxInactiveDurationSecondsSet the maximum duration for inactive topics. If it is not specified, the brokerDeleteInactiveTopicsFrequencySeconds parameter is adopted.N/A
forceDeleteTenantAllowedEnable you to delete a tenant forcefully.false
forceDeleteNamespaceAllowedEnable you to delete a namespace forcefully.false
messageExpiryCheckIntervalInMinutesThe frequency of proactively checking and purging expired messages.5
brokerServiceCompactionMonitorIntervalInSecondsInterval between checks to determine whether topics with compaction policies need compaction.60
brokerServiceCompactionThresholdInBytesIf the estimated backlog size is greater than this threshold, compression is triggered.

Set this threshold to 0 means disabling the compression check.
N/A
delayedDeliveryEnabledWhether to enable the delayed delivery for messages. If disabled, messages will be immediately delivered and there will be no tracking overhead.true
delayedDeliveryTickTimeMillisControl the tick time for retrying on delayed delivery, which affects the accuracy of the delivery time compared to the scheduled time. By default, it is 1 second.1000
activeConsumerFailoverDelayTimeMillisHow long to delay rewinding cursor and dispatching messages when active consumer is changed.1000
clientLibraryVersionCheckEnabledEnable check for minimum allowed client library versionfalse
clientLibraryVersionCheckAllowUnversionedAllow client libraries with no version informationtrue
statusFilePathPath for the file used to determine the rotation status for the broker when responding to service discovery health checks
preferLaterVersionsIf true, (and ModularLoadManagerImpl is being used), the load manager will attempt to use only brokers running the latest software version (to minimize impact to bundles)false
maxNumPartitionsPerPartitionedTopicMax number of partitions per partitioned topic. Use 0 or negative number to disable the check0
tlsEnabledDeprecated - Use webServicePortTls and brokerServicePortTls instead.false
tlsCertificateFilePathPath for the TLS certificate file
tlsKeyFilePathPath for the TLS private key file
tlsTrustCertsFilePathPath for the trusted TLS certificate file. This cert is used to verify that any certs presented by connecting clients are signed by a certificate authority. If this verification fails, then the certs are untrusted and the connections are dropped.
tlsAllowInsecureConnectionAccept untrusted TLS certificate from client. If it is set to true, a client with a cert which cannot be verified with the ‘tlsTrustCertsFilePath’ cert will be allowed to connect to the server, though the cert will not be used for client authentication.false
tlsProtocolsSpecify the tls protocols the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLSv1.3, TLSv1.2
tlsCiphersSpecify the tls cipher the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
tlsEnabledWithKeyStoreEnable TLS with KeyStore type configuration in brokerfalse
tlsProviderTLS Provider for KeyStore type
tlsKeyStoreTypeLS KeyStore type configuration in broker: JKS, PKCS12JKS
tlsKeyStoreTLS KeyStore path in broker
tlsKeyStorePasswordTLS KeyStore password for broker
brokerClientTlsEnabledWithKeyStoreWhether internal client use KeyStore type to authenticate with Pulsar brokersfalse
brokerClientSslProviderThe TLS Provider used by internal client to authenticate with other Pulsar brokers
brokerClientTlsTrustStoreTypeTLS TrustStore type configuration for internal client: JKS, PKCS12, used by the internal client to authenticate with Pulsar brokersJKS
brokerClientTlsTrustStoreTLS TrustStore path for internal client, used by the internal client to authenticate with Pulsar brokers
brokerClientTlsTrustStorePasswordTLS TrustStore password for internal client, used by the internal client to authenticate with Pulsar brokers
brokerClientTlsCiphersSpecify the tls cipher the internal client will use to negotiate during TLS Handshake. (a comma-separated list of ciphers) e.g. [TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256]
brokerClientTlsProtocolsSpecify the tls protocols the broker will use to negotiate during TLS handshake. (a comma-separated list of protocol names). e.g. TLSv1.3, TLSv1.2
ttlDurationDefaultInSecondsThe default Time to Live (TTL) for namespaces if the TTL is not configured at namespace policies. When the value is set to 0, TTL is disabled. By default, TTL is disabled.0
tokenSettingPrefixConfigure the prefix of the token-related settings, such as tokenSecretKey, tokenPublicKey, tokenAuthClaim, tokenPublicAlg, tokenAudienceClaim, and tokenAudience.
tokenSecretKeyConfigure the secret key to be used to validate auth tokens. The key can be specified like: tokenSecretKey=data:;base64,xxxxxxxxx or tokenSecretKey=file:///my/secret.key. Note: key file must be DER-encoded.
tokenPublicKeyConfigure the public key to be used to validate auth tokens. The key can be specified like: tokenPublicKey=data:;base64,xxxxxxxxx or tokenPublicKey=file:///my/secret.key. Note: key file must be DER-encoded.
tokenPublicAlgConfigure the algorithm to be used to validate auth tokens. This can be any of the asymettric algorithms supported by Java JWT (https://github.com/jwtk/jjwt#signature-algorithms-keys)RS256
tokenAuthClaimSpecify which of the token's claims will be used as the authentication “principal” or “role”. The default “sub” claim will be used if this is left blank
tokenAudienceClaimThe token audience “claim” name, e.g. “aud”, that will be used to get the audience from token. If not set, audience will not be verified.
tokenAudienceThe token audience stands for this broker. The field tokenAudienceClaim of a valid token, need contains this.
maxUnackedMessagesPerConsumerMax number of unacknowledged messages allowed to receive messages by a consumer on a shared subscription. Broker will stop sending messages to consumer once, this limit reaches until consumer starts acknowledging messages back. Using a value of 0, is disabling unackeMessage limit check and consumer can receive messages without any restriction50000
maxUnackedMessagesPerSubscriptionMax number of unacknowledged messages allowed per shared subscription. Broker will stop dispatching messages to all consumers of the subscription once this limit reaches until consumer starts acknowledging messages back and unack count reaches to limit/2. Using a value of 0, is disabling unackedMessage-limit check and dispatcher can dispatch messages without any restriction200000
subscriptionRedeliveryTrackerEnabledEnable subscription message redelivery trackertrue
subscriptionExpirationTimeMinutesHow long to delete inactive subscriptions from last consuming.

Setting this configuration to a value greater than 0 deletes inactive subscriptions automatically.
Setting this configuration to 0 does not delete inactive subscriptions automatically.

Since this configuration takes effect on all topics, if there is even one topic whose subscriptions should not be deleted automatically, you need to set it to 0.
Instead, you can set a subscription expiration time for each namespace using the pulsar-admin namespaces set-subscription-expiration-time options command.
0
maxConcurrentLookupRequestMax number of concurrent lookup request broker allows to throttle heavy incoming lookup traffic50000
maxConcurrentTopicLoadRequestMax number of concurrent topic loading request broker allows to control number of zk-operations5000
authenticationEnabledEnable authenticationfalse
authenticationProvidersAuthentication provider name list, which is comma separated list of class names
authenticationRefreshCheckSecondsInterval of time for checking for expired authentication credentials60
authorizationEnabledEnforce authorizationfalse
superUserRolesRole names that are treated as “super-user”, meaning they will be able to do all admin operations and publish/consume from all topics
brokerClientAuthenticationPluginAuthentication settings of the broker itself. Used when the broker connects to other brokers, either in same or other clusters
brokerClientAuthenticationParameters
athenzDomainNamesSupported Athenz provider domain names(comma separated) for authentication
exposePreciseBacklogInPrometheusEnable expose the precise backlog stats, set false to use published counter and consumed counter to calculate, this would be more efficient but may be inaccurate.false
schemaRegistryStorageClassNameThe schema storage implementation used by this broker.org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorageFactory
isSchemaValidationEnforcedEnforce schema validation on following cases: if a producer without a schema attempts to produce to a topic with schema, the producer will be failed to connect. PLEASE be carefully on using this, since non-java clients don't support schema. If this setting is enabled, then non-java clients fail to produce.false
offloadersDirectoryThe directory for all the offloader implementations../offloaders
bookkeeperMetadataServiceUriMetadata service uri that bookkeeper is used for loading corresponding metadata driver and resolving its metadata service location. This value can be fetched using bookkeeper shell whatisinstanceid command in BookKeeper cluster. For example: zk+hierarchical://localhost:2181/ledgers. The metadata service uri list can also be semicolon separated values like below: zk+hierarchical://zk1:2181;zk2:2181;zk3:2181/ledgers
bookkeeperClientAuthenticationPluginAuthentication plugin to use when connecting to bookies
bookkeeperClientAuthenticationParametersNameBookKeeper auth plugin implementation specifics parameters name and values
bookkeeperClientAuthenticationParameters
bookkeeperClientNumWorkerThreadsNumber of BookKeeper client worker threads. Default is Runtime.getRuntime().availableProcessors()
bookkeeperClientTimeoutInSecondsTimeout for BK add / read operations30
bookkeeperClientSpeculativeReadTimeoutInMillisSpeculative reads are initiated if a read request doesn’t complete within a certain time Using a value of 0, is disabling the speculative reads0
bookkeeperNumberOfChannelsPerBookieNumber of channels per bookie16
bookkeeperClientHealthCheckEnabledEnable bookies health check. Bookies that have more than the configured number of failure within the interval will be quarantined for some time. During this period, new ledgers won’t be created on these bookiestrue
bookkeeperClientHealthCheckIntervalSeconds60
bookkeeperClientHealthCheckErrorThresholdPerInterval5
bookkeeperClientHealthCheckQuarantineTimeInSeconds1800
bookkeeperClientRackawarePolicyEnabledEnable rack-aware bookie selection policy. BK will chose bookies from different racks when forming a new bookie ensembletrue
bookkeeperClientRegionawarePolicyEnabledEnable region-aware bookie selection policy. BK will chose bookies from different regions and racks when forming a new bookie ensemble. If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignoredfalse
bookkeeperClientMinNumRacksPerWriteQuorumMinimum number of racks per write quorum. BK rack-aware bookie selection policy will try to get bookies from at least ‘bookkeeperClientMinNumRacksPerWriteQuorum’ racks for a write quorum.2
bookkeeperClientEnforceMinNumRacksPerWriteQuorumEnforces rack-aware bookie selection policy to pick bookies from ‘bookkeeperClientMinNumRacksPerWriteQuorum’ racks for a writeQuorum. If BK can't find bookie then it would throw BKNotEnoughBookiesException instead of picking random one.false
bookkeeperClientReorderReadSequenceEnabledEnable/disable reordering read sequence on reading entries.false
bookkeeperClientIsolationGroupsEnable bookie isolation by specifying a list of bookie groups to choose from. Any bookie outside the specified groups will not be used by the broker
bookkeeperClientSecondaryIsolationGroupsEnable bookie secondary-isolation group if bookkeeperClientIsolationGroups doesn't have enough bookie available.
bookkeeperClientMinAvailableBookiesInIsolationGroupsMinimum bookies that should be available as part of bookkeeperClientIsolationGroups else broker will include bookkeeperClientSecondaryIsolationGroups bookies in isolated list.
bookkeeperClientGetBookieInfoIntervalSecondsSet the interval to periodically check bookie info86400
bookkeeperClientGetBookieInfoRetryIntervalSecondsSet the interval to retry a failed bookie info lookup60
bookkeeperEnableStickyReadsEnable/disable having read operations for a ledger to be sticky to a single bookie. If this flag is enabled, the client will use one single bookie (by preference) to read all entries for a ledger.true
managedLedgerDefaultEnsembleSizeNumber of bookies to use when creating a ledger2
managedLedgerDefaultWriteQuorumNumber of copies to store for each message2
managedLedgerDefaultAckQuorumNumber of guaranteed copies (acks to wait before write is complete)2
managedLedgerCacheSizeMBAmount of memory to use for caching data payload in managed ledger. This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker. By default, uses 1/5th of available direct memory
managedLedgerCacheCopyEntriesWhether we should make a copy of the entry payloads when inserting in cachefalse
managedLedgerCacheEvictionWatermarkThreshold to which bring down the cache level when eviction is triggered0.9
managedLedgerCacheEvictionFrequencyConfigure the cache eviction frequency for the managed ledger cache (evictions/sec)100.0
managedLedgerCacheEvictionTimeThresholdMillisAll entries that have stayed in cache for more than the configured time, will be evicted1000
managedLedgerCursorBackloggedThresholdConfigure the threshold (in number of entries) from where a cursor should be considered ‘backlogged’ and thus should be set as inactive.1000
managedLedgerDefaultMarkDeleteRateLimitRate limit the amount of writes per second generated by consumer acking the messages1.0
managedLedgerMaxEntriesPerLedgerThe max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: The max rollover time has been reachedThe max entries have been written to the ledgerThe max ledger size has been written to the ledger50000
managedLedgerMinLedgerRolloverTimeMinutesMinimum time between ledger rollover for a topic10
managedLedgerMaxLedgerRolloverTimeMinutesMaximum time before forcing a ledger rollover for a topic240
managedLedgerCursorMaxEntriesPerLedgerMax number of entries to append to a cursor ledger50000
managedLedgerCursorRolloverTimeInSecondsMax time before triggering a rollover on a cursor ledger14400
managedLedgerMaxUnackedRangesToPersistMax number of “acknowledgment holes” that are going to be persistently stored. When acknowledging out of order, a consumer will leave holes that are supposed to be quickly filled by acking all the messages. The information of which messages are acknowledged is persisted by compressing in “ranges” of messages that were acknowledged. After the max number of ranges is reached, the information will only be tracked in memory and messages will be redelivered in case of crashes.1000
autoSkipNonRecoverableDataSkip reading non-recoverable/unreadable data-ledger under managed-ledger’s list.It helps when data-ledgers gets corrupted at bookkeeper and managed-cursor is stuck at that ledger.false
loadBalancerEnabledEnable load balancertrue
loadBalancerPlacementStrategyStrategy to assign a new bundle weightedRandomSelection
loadBalancerReportUpdateThresholdPercentagePercentage of change to trigger load report update10
loadBalancerReportUpdateMaxIntervalMinutesmaximum interval to update load report15
loadBalancerHostUsageCheckIntervalMinutesFrequency of report to collect1
loadBalancerSheddingIntervalMinutesLoad shedding interval. Broker periodically checks whether some traffic should be offload from some over-loaded broker to other under-loaded brokers30
loadBalancerSheddingGracePeriodMinutesPrevent the same topics to be shed and moved to other broker more than once within this timeframe30
loadBalancerBrokerMaxTopicsUsage threshold to allocate max number of topics to broker50000
loadBalancerBrokerUnderloadedThresholdPercentageUsage threshold to determine a broker as under-loaded1
loadBalancerBrokerOverloadedThresholdPercentageUsage threshold to determine a broker as over-loaded85
loadBalancerResourceQuotaUpdateIntervalMinutesInterval to update namespace bundle resource quota15
loadBalancerBrokerComfortLoadLevelPercentageUsage threshold to determine a broker is having just right level of load65
loadBalancerAutoBundleSplitEnabledenable/disable namespace bundle auto splitfalse
loadBalancerNamespaceBundleMaxTopicsmaximum topics in a bundle, otherwise bundle split will be triggered1000
loadBalancerNamespaceBundleMaxSessionsmaximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered1000
loadBalancerNamespaceBundleMaxMsgRatemaximum msgRate (in + out) in a bundle, otherwise bundle split will be triggered1000
loadBalancerNamespaceBundleMaxBandwidthMbytesmaximum bandwidth (in + out) in a bundle, otherwise bundle split will be triggered100
loadBalancerNamespaceMaximumBundlesmaximum number of bundles in a namespace128
replicationMetricsEnabledEnable replication metricstrue
replicationConnectionsPerBrokerMax number of connections to open for each broker in a remote cluster More connections host-to-host lead to better throughput over high-latency links.16
replicationProducerQueueSizeReplicator producer queue size1000
replicatorPrefixReplicator prefix used for replicator producer name and cursor name pulsar.repl
replicationTlsEnabledEnable TLS when talking with other clusters to replicate messagesfalse
brokerServicePurgeInactiveFrequencyInSecondsDeprecated. Use brokerDeleteInactiveTopicsFrequencySeconds.60
transactionCoordinatorEnabledWhether to enable transaction coordinator in broker.true
transactionMetadataStoreProviderClassNameorg.apache.pulsar.transaction.coordinator.impl.InMemTransactionMetadataStoreProvider
defaultRetentionTimeInMinutesDefault message retention time0
defaultRetentionSizeInMBDefault retention size0
keepAliveIntervalSecondsHow often to check whether the connections are still alive30
bootstrapNamespacesThe bootstrap name.N/A
loadManagerClassNameName of load manager to useorg.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl
supportedNamespaceBundleSplitAlgorithmsSupported algorithms name for namespace bundle split[range_equally_divide,topic_count_equally_divide]
defaultNamespaceBundleSplitAlgorithmDefault algorithm name for namespace bundle splitrange_equally_divide
managedLedgerOffloadDriverThe directory for all the offloader implementations offloadersDirectory=./offloaders. Driver to use to offload old data to long term storage (Possible values: S3, aws-s3, google-cloud-storage). When using google-cloud-storage, Make sure both Google Cloud Storage and Google Cloud Storage JSON API are enabled for the project (check from Developers Console -> Api&auth -> APIs).
managedLedgerOffloadMaxThreadsMaximum number of thread pool threads for ledger offloading2
managedLedgerOffloadPrefetchRoundsThe maximum prefetch rounds for ledger reading for offloading.1
managedLedgerUnackedRangesOpenCacheSetEnabledUse Open Range-Set to cache unacknowledged messagestrue
managedLedgerOffloadDeletionLagMsDelay between a ledger being successfully offloaded to long term storage and the ledger being deleted from bookkeeper14400000
managedLedgerOffloadAutoTriggerSizeThresholdBytesThe number of bytes before triggering automatic offload to long term storage-1 (disabled)
s3ManagedLedgerOffloadRegionFor Amazon S3 ledger offload, AWS region
s3ManagedLedgerOffloadBucketFor Amazon S3 ledger offload, Bucket to place offloaded ledger into
s3ManagedLedgerOffloadServiceEndpointFor Amazon S3 ledger offload, Alternative endpoint to connect to (useful for testing)
s3ManagedLedgerOffloadMaxBlockSizeInBytesFor Amazon S3 ledger offload, Max block size in bytes. (64MB by default, 5MB minimum)67108864
s3ManagedLedgerOffloadReadBufferSizeInBytesFor Amazon S3 ledger offload, Read buffer size in bytes (1MB by default)1048576
gcsManagedLedgerOffloadRegionFor Google Cloud Storage ledger offload, region where offload bucket is located. Go to this page for more details: https://cloud.google.com/storage/docs/bucket-locations .N/A
gcsManagedLedgerOffloadBucketFor Google Cloud Storage ledger offload, Bucket to place offloaded ledger into.N/A
gcsManagedLedgerOffloadMaxBlockSizeInBytesFor Google Cloud Storage ledger offload, the maximum block size in bytes. (64MB by default, 5MB minimum)67108864
gcsManagedLedgerOffloadReadBufferSizeInBytesFor Google Cloud Storage ledger offload, Read buffer size in bytes. (1MB by default)1048576
gcsManagedLedgerOffloadServiceAccountKeyFileFor Google Cloud Storage, path to json file containing service account credentials. For more details, see the “Service Accounts” section of https://support.google.com/googleapi/answer/6158849 .N/A
fileSystemProfilePathFor File System Storage, file system profile path.../conf/filesystem_offload_core_site.xml
fileSystemURIFor File System Storage, file system uri.N/A
s3ManagedLedgerOffloadRoleFor Amazon S3 ledger offload, provide a role to assume before writing to s3
s3ManagedLedgerOffloadRoleSessionNameFor Amazon S3 ledger offload, provide a role session name when using a rolepulsar-s3-offload
acknowledgmentAtBatchIndexLevelEnabledEnable or disable the batch index acknowledgement.false
enableReplicatedSubscriptionsWhether to enable tracking of replicated subscriptions state across clusters.true
replicatedSubscriptionsSnapshotFrequencyMillisThe frequency of snapshots for replicated subscriptions tracking.1000
replicatedSubscriptionsSnapshotTimeoutSecondsThe timeout for building a consistent snapshot for tracking replicated subscriptions state.30
replicatedSubscriptionsSnapshotMaxCachedPerSubscriptionThe maximum number of snapshot to be cached per subscription.10
maxMessagePublishBufferSizeInMBThe maximum memory size for a broker to handle messages that are sent by producers. If the processing message size exceeds this value, the broker stops reading data from the connection. The processing messages refer to the messages that are sent to the broker but the broker has not sent response to the client. Usually the messages are waiting to be written to bookies. It is shared across all the topics running in the same broker. The value -1 disables the memory limitation. By default, it is 50% of direct memory.N/A
messagePublishBufferCheckIntervalInMillisInterval between checks to see if message publish buffer size exceeds the maximum. Use 0 or negative number to disable the max publish buffer limiting.100
retentionCheckIntervalInSecondsCheck between intervals to see if consumed ledgers need to be trimmed. Use 0 or negative number to disable the check.120
maxMessageSizeSet the maximum size of a message.5242880
preciseTopicPublishRateLimiterEnableEnable precise topic publish rate limiting.false
lazyCursorRecoveryWhether to recover cursors lazily when trying to recover a managed ledger backing a persistent topic. It can improve write availability of topics. The caveat is now when recovered ledger is ready to write we‘re not sure if all old consumers’ last mark delete position(ack position) can be recovered or not. So user can make the trade off or have custom logic in application to checkpoint consumer state.false
haProxyProtocolEnabledEnable or disable the HAProxy protocol.false
maxTopicsPerNamespaceThe maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check.0
subscriptionTypesEnabledEnable all subscription types, which are exclusive, shared, failover, and key_shared.Exclusive, Shared, Failover, Key_Shared
managedLedgerInfoCompressionTypeCompression type of managed ledger information.

Available options are NONE, LZ4, ZLIB, ZSTD, and SNAPPY).

If this value is NONE or invalid, the managedLedgerInfo is not compressed.

Note that after enabling this configuration, if you want to degrade a broker, you need to change the value to NONE and make sure all ledger metadata is saved without compression.
None
additionalServletsAdditional servlet name.

If you have multiple additional servlets, separate them by commas.

For example, additionalServlet_1, additionalServlet_2
N/A
additionalServletDirectoryLocation of broker additional servlet NAR directory./brokerAdditionalServlet
brokerEntryMetadataInterceptorsSet broker entry metadata interceptors.

Multiple interceptors should be separated by commas.

Available values:org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptororg.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor

Example
brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor, org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor
N/A
enableExposingBrokerEntryMetadataToClientWhether to expose broker entry metadata to client or not.

Available values:truefalse
Example
enableExposingBrokerEntryMetadataToClient=true
false

Client

You can use the pulsar-client CLI tool to publish messages to and consume messages from Pulsar topics. You can use this tool in place of a client library.

NameDescriptionDefault
webServiceUrlThe web URL for the cluster.http://localhost:8080/
brokerServiceUrlThe Pulsar protocol URL for the cluster.pulsar://localhost:6650/
authPluginThe authentication plugin.
authParamsThe authentication parameters for the cluster, as a comma-separated string.
useTlsWhether to enforce the TLS authentication in the cluster.false
tlsAllowInsecureConnectionAllow TLS connections to servers whose certificate cannot be verified to have been signed by a trusted certificate authority.false
tlsEnableHostnameVerificationWhether the server hostname must match the common name of the certificate that is used by the server.false
tlsTrustCertsFilePath
useKeyStoreTlsEnable TLS with KeyStore type configuration in the broker.false
tlsTrustStoreTypeTLS TrustStore type configuration. JKS PKCS12JKS
tlsTrustStoreTLS TrustStore path.
tlsTrustStorePasswordTLS TrustStore password.

Log4j

You can set the log level and configuration in the log4j2.yaml file. The following logging configuration parameters are available.

NameDefault
pulsar.root.loggerWARN,CONSOLE
pulsar.log.dirlogs
pulsar.log.filepulsar.log
log4j.rootLogger${pulsar.root.logger}
log4j.appender.CONSOLEorg.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.ThresholdDEBUG
log4j.appender.CONSOLE.layoutorg.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern%d{ISO8601} - %-5p - [%t:%C{1}@%L] - %m%n
log4j.appender.ROLLINGFILEorg.apache.log4j.DailyRollingFileAppender
log4j.appender.ROLLINGFILE.ThresholdDEBUG
log4j.appender.ROLLINGFILE.File${pulsar.log.dir}/${pulsar.log.file}
log4j.appender.ROLLINGFILE.layoutorg.apache.log4j.PatternLayout
log4j.appender.ROLLINGFILE.layout.ConversionPattern%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n
log4j.appender.TRACEFILEorg.apache.log4j.FileAppender
log4j.appender.TRACEFILE.ThresholdTRACE
log4j.appender.TRACEFILE.Filepulsar-trace.log
log4j.appender.TRACEFILE.layoutorg.apache.log4j.PatternLayout
log4j.appender.TRACEFILE.layout.ConversionPattern%d{ISO8601} - %-5p [%t:%C{1}@%L][%x] - %m%n

Note: ‘topic’ in log4j2.appender is configurable.

  • If you want to append all logs to a single topic, set the same topic name.
  • If you want to append logs to different topics, you can set different topic names.

Log4j shell

NameDefault
bookkeeper.root.loggerERROR,CONSOLE
log4j.rootLogger${bookkeeper.root.logger}
log4j.appender.CONSOLEorg.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.ThresholdDEBUG
log4j.appender.CONSOLE.layoutorg.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern%d{ABSOLUTE} %-5p %m%n
log4j.logger.org.apache.zookeeperERROR
log4j.logger.org.apache.bookkeeperERROR
log4j.logger.org.apache.bookkeeper.bookie.BookieShellINFO

Standalone

NameDescriptionDefault
authenticateOriginalAuthDataIf this flag is set to true, the broker authenticates the original Auth data; else it just accepts the originalPrincipal and authorizes it (if required).false
zookeeperServersThe quorum connection string for local ZooKeeper
zooKeeperCacheExpirySecondsZooKeeper cache expiry time in seconds300
configurationStoreServersConfiguration store connection string (as a comma-separated list)
brokerServicePortThe port on which the standalone broker listens for connections6650
webServicePortThe port used by the standalone broker for HTTP requests8080
bindAddressThe hostname or IP address on which the standalone service binds0.0.0.0
bindAddressesAdditional Hostname or IP addresses the service binds on: listener_name:scheme://host:port,....
advertisedAddressThe hostname or IP address that the standalone service advertises to the outside world. If not set, the value of InetAddress.getLocalHost().getHostName() is used.
numAcceptorThreadsNumber of threads to use for Netty Acceptor1
numIOThreadsNumber of threads to use for Netty IO2 * Runtime.getRuntime().availableProcessors()
numHttpServerThreadsNumber of threads to use for HTTP requests processing2 * Runtime.getRuntime().availableProcessors()
isRunningStandaloneThis flag controls features that are meant to be used when running in standalone mode.N/A
clusterNameThe name of the cluster that this broker belongs to.standalone
failureDomainsEnabledEnable cluster's failure-domain which can distribute brokers into logical region.false
zooKeeperSessionTimeoutMillisThe ZooKeeper session timeout, in milliseconds.30000
zooKeeperOperationTimeoutSecondsZooKeeper operation timeout in seconds.30
brokerShutdownTimeoutMsThe time to wait for graceful broker shutdown. After this time elapses, the process will be killed.60000
skipBrokerShutdownOnOOMFlag to skip broker shutdown when broker handles Out of memory error.false
backlogQuotaCheckEnabledEnable the backlog quota check, which enforces a specified action when the quota is reached.true
backlogQuotaCheckIntervalInSecondsHow often to check for topics that have reached the backlog quota.60
backlogQuotaDefaultLimitBytesThe default per-topic backlog quota limit. Being less than 0 means no limitation. By default, it is -1.-1
ttlDurationDefaultInSecondsThe default Time to Live (TTL) for namespaces if the TTL is not configured at namespace policies. When the value is set to 0, TTL is disabled. By default, TTL is disabled.0
brokerDeleteInactiveTopicsEnabledEnable the deletion of inactive topics. If topics are not consumed for some while, these inactive topics might be cleaned up. Deleting inactive topics is enabled by default. The default period is 1 minute.true
brokerDeleteInactiveTopicsFrequencySecondsHow often to check for inactive topics, in seconds.60
maxPendingPublishRequestsPerConnectionMaximum pending publish requests per connection to avoid keeping large number of pending requests in memory1000
messageExpiryCheckIntervalInMinutesHow often to proactively check and purged expired messages.5
activeConsumerFailoverDelayTimeMillisHow long to delay rewinding cursor and dispatching messages when active consumer is changed.1000
subscriptionExpirationTimeMinutesHow long to delete inactive subscriptions from last consumption. When it is set to 0, inactive subscriptions are not deleted automatically0
subscriptionRedeliveryTrackerEnabledEnable subscription message redelivery tracker to send redelivery count to consumer.true
subscriptionKeySharedEnableWhether to enable the Key_Shared subscription.true
subscriptionKeySharedUseConsistentHashingIn Key_Shared subscription type, with default AUTO_SPLIT mode, use splitting ranges or consistent hashing to reassign keys to new consumers.false
subscriptionKeySharedConsistentHashingReplicaPointsIn Key_Shared subscription type, the number of points in the consistent-hashing ring. The greater the number, the more equal the assignment of keys to consumers.100
subscriptionExpiryCheckIntervalInMinutesHow frequently to proactively check and purge expired subscription5
brokerDeduplicationEnabledSet the default behavior for message deduplication in the broker. This can be overridden per-namespace. If it is enabled, the broker rejects messages that are already stored in the topic.false
brokerDeduplicationMaxNumberOfProducersMaximum number of producer information that it's going to be persisted for deduplication purposes10000
brokerDeduplicationEntriesIntervalNumber of entries after which a deduplication information snapshot is taken. A greater interval leads to less snapshots being taken though it would increase the topic recovery time, when the entries published after the snapshot need to be replayed.1000
brokerDeduplicationProducerInactivityTimeoutMinutesThe time of inactivity (in minutes) after which the broker discards deduplication information related to a disconnected producer.360
defaultNumberOfNamespaceBundlesWhen a namespace is created without specifying the number of bundles, this value is used as the default setting.4
clientLibraryVersionCheckEnabledEnable checks for minimum allowed client library version.false
clientLibraryVersionCheckAllowUnversionedAllow client libraries with no version informationtrue
statusFilePathThe path for the file used to determine the rotation status for the broker when responding to service discovery health checks/usr/local/apache/htdocs
maxUnackedMessagesPerConsumerThe maximum number of unacknowledged messages allowed to be received by consumers on a shared subscription. The broker will stop sending messages to a consumer once this limit is reached or until the consumer begins acknowledging messages. A value of 0 disables the unacked message limit check and thus allows consumers to receive messages without any restrictions.50000
maxUnackedMessagesPerSubscriptionThe same as above, except per subscription rather than per consumer.200000
maxUnackedMessagesPerBrokerMaximum number of unacknowledged messages allowed per broker. Once this limit reaches, the broker stops dispatching messages to all shared subscriptions which has a higher number of unacknowledged messages until subscriptions start acknowledging messages back and unacknowledged messages count reaches to limit/2. When the value is set to 0, unacknowledged message limit check is disabled and broker does not block dispatchers.0
maxUnackedMessagesPerSubscriptionOnBrokerBlockedOnce the broker reaches maxUnackedMessagesPerBroker limit, it blocks subscriptions which have higher unacknowledged messages than this percentage limit and subscription does not receive any new messages until that subscription acknowledges messages back.0.16
unblockStuckSubscriptionEnabledBroker periodically checks if subscription is stuck and unblock if flag is enabled.false
maxNumPartitionsPerPartitionedTopicMax number of partitions per partitioned topic. Use 0 or negative number to disable the check0
zookeeperSessionExpiredPolicyThere are two policies when ZooKeeper session expired happens, “shutdown” and “reconnect”. If it is set to “shutdown” policy, when ZooKeeper session expired happens, the broker is shutdown. If it is set to “reconnect” policy, the broker tries to reconnect to ZooKeeper server and re-register metadata to ZooKeeper. Note: the “reconnect” policy is an experiment feature.shutdown
topicPublisherThrottlingTickTimeMillisTick time to schedule task that checks topic publish rate limiting across all topics. A lower value can improve accuracy while throttling publish but it uses more CPU to perform frequent check. (Disable publish throttling with value 0)10
brokerPublisherThrottlingTickTimeMillisTick time to schedule task that checks broker publish rate limiting across all topics. A lower value can improve accuracy while throttling publish but it uses more CPU to perform frequent check. When the value is set to 0, publish throttling is disabled.50
brokerPublisherThrottlingMaxMessageRateMaximum rate (in 1 second) of messages allowed to publish for a broker if the message rate limiting is enabled. When the value is set to 0, message rate limiting is disabled.0
brokerPublisherThrottlingMaxByteRateMaximum rate (in 1 second) of bytes allowed to publish for a broker if the byte rate limiting is enabled. When the value is set to 0, the byte rate limiting is disabled.0
subscribeThrottlingRatePerConsumerToo many subscribe requests from a consumer can cause broker rewinding consumer cursors and loading data from bookies, hence causing high network bandwidth usage. When the positive value is set, broker will throttle the subscribe requests for one consumer. Otherwise, the throttling will be disabled. By default, throttling is disabled.0
subscribeRatePeriodPerConsumerInSecondRate period for {subscribeThrottlingRatePerConsumer}. By default, it is 30s.30
dispatchThrottlingRateInMsgDispatch throttling-limit of messages for a broker (per second). 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingRateInByteDispatch throttling-limit of bytes for a broker (per second). 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingRatePerTopicInMsgDefault messages (per second) dispatch throttling-limit for every topic. When the value is set to 0, default message dispatch throttling-limit is disabled.0
dispatchThrottlingRatePerTopicInByteDefault byte (per second) dispatch throttling-limit for every topic. When the value is set to 0, default byte dispatch throttling-limit is disabled.0
dispatchThrottlingOnBatchMessageEnabledApply dispatch rate limiting on batch message instead individual messages with in batch message. (Default is disabled).false
dispatchThrottlingRateRelativeToPublishRateEnable dispatch rate-limiting relative to publish rate.false
dispatchThrottlingRatePerSubscriptionInMsgThe defaulted number of message dispatching throttling-limit for a subscription. The value of 0 disables message dispatch-throttling.0
dispatchThrottlingRatePerSubscriptionInByteThe default number of message-bytes dispatching throttling-limit for a subscription. The value of 0 disables message-byte dispatch-throttling.0
dispatchThrottlingRatePerReplicatorInMsgDispatch throttling-limit of messages for every replicator in replication (per second). 0 means the dispatch throttling-limit in replication is disabled.0
dispatchThrottlingRatePerReplicatorInByteDispatch throttling-limit of bytes for every replicator in replication (per second). 0 means the dispatch throttling-limit is disabled.0
dispatchThrottlingOnNonBacklogConsumerEnabledEnable dispatch-throttling for both caught up consumers as well as consumers who have backlogs.true
dispatcherMaxReadBatchSizeThe maximum number of entries to read from BookKeeper. By default, it is 100 entries.100
dispatcherMaxReadSizeBytesThe maximum size in bytes of entries to read from BookKeeper. By default, it is 5MB.5242880
dispatcherMinReadBatchSizeThe minimum number of entries to read from BookKeeper. By default, it is 1 entry. When there is an error occurred on reading entries from bookkeeper, the broker will backoff the batch size to this minimum number.1
dispatcherMaxRoundRobinBatchSizeThe maximum number of entries to dispatch for a shared subscription. By default, it is 20 entries.20
preciseDispatcherFlowControlPrecise dispathcer flow control according to history message number of each entry.false
streamingDispatchWhether to use streaming read dispatcher. It can be useful when there's a huge backlog to drain and instead of read with micro batch we can streamline the read from bookkeeper to make the most of consumer capacity till we hit bookkeeper read limit or consumer process limit, then we can use consumer flow control to tune the speed. This feature is currently in preview and can be changed in subsequent release.false
maxConcurrentLookupRequestMaximum number of concurrent lookup request that the broker allows to throttle heavy incoming lookup traffic.50000
maxConcurrentTopicLoadRequestMaximum number of concurrent topic loading request that the broker allows to control the number of zk-operations.5000
maxConcurrentNonPersistentMessagePerConnectionMaximum number of concurrent non-persistent message that can be processed per connection.1000
numWorkerThreadsForNonPersistentTopicNumber of worker threads to serve non-persistent topic.8
enablePersistentTopicsEnable broker to load persistent topics.true
enableNonPersistentTopicsEnable broker to load non-persistent topics.true
maxProducersPerTopicMaximum number of producers allowed to connect to topic. Once this limit reaches, the broker rejects new producers until the number of connected producers decreases. When the value is set to 0, maxProducersPerTopic-limit check is disabled.0
maxConsumersPerTopicMaximum number of consumers allowed to connect to topic. Once this limit reaches, the broker rejects new consumers until the number of connected consumers decreases. When the value is set to 0, maxConsumersPerTopic-limit check is disabled.0
maxConsumersPerSubscriptionMaximum number of consumers allowed to connect to subscription. Once this limit reaches, the broker rejects new consumers until the number of connected consumers decreases. When the value is set to 0, maxConsumersPerSubscription-limit check is disabled.0
maxNumPartitionsPerPartitionedTopicMaximum number of partitions per partitioned topic. When the value is set to a negative number or is set to 0, the check is disabled.0
tlsCertRefreshCheckDurationSecTLS certificate refresh duration in seconds. When the value is set to 0, check the TLS certificate on every new connection.300
tlsCertificateFilePathPath for the TLS certificate file.
tlsKeyFilePathPath for the TLS private key file.
tlsTrustCertsFilePathPath for the trusted TLS certificate file.
tlsAllowInsecureConnectionAccept untrusted TLS certificate from the client. If it is set to true, a client with a certificate which cannot be verified with the ‘tlsTrustCertsFilePath’ certificate is allowed to connect to the server, though the certificate is not be used for client authentication.false
tlsProtocolsSpecify the TLS protocols the broker uses to negotiate during TLS handshake.
tlsCiphersSpecify the TLS cipher the broker uses to negotiate during TLS Handshake.
tlsRequireTrustedClientCertOnConnectTrusted client certificates are required for to connect TLS. Reject the Connection if the client certificate is not trusted. In effect, this requires that all connecting clients perform TLS client authentication.false
tlsEnabledWithKeyStoreEnable TLS with KeyStore type configuration in broker.false
tlsProviderTLS Provider for KeyStore type.
tlsKeyStoreTypeTLS KeyStore type configuration in the broker.JKS PKCS12JKS
tlsKeyStoreTLS KeyStore path in the broker.
tlsKeyStorePasswordTLS KeyStore password for the broker.
tlsTrustStoreTypeTLS TrustStore type configuration in the brokerJKS PKCS12JKS
tlsTrustStoreTLS TrustStore path in the broker.
tlsTrustStorePasswordTLS TrustStore password for the broker.
brokerClientTlsEnabledWithKeyStoreConfigure whether the internal client uses the KeyStore type to authenticate with Pulsar brokers.false
brokerClientSslProviderThe TLS Provider used by the internal client to authenticate with other Pulsar brokers.
brokerClientTlsTrustStoreTypeTLS TrustStore type configuration for the internal client to authenticate with Pulsar brokers. JKS PKCS12JKS
brokerClientTlsTrustStoreTLS TrustStore path for the internal client to authenticate with Pulsar brokers.
brokerClientTlsTrustStorePasswordTLS TrustStore password for the internal client to authenticate with Pulsar brokers.
brokerClientTlsCiphersSpecify the TLS cipher that the internal client uses to negotiate during TLS Handshake.
brokerClientTlsProtocolsSpecify the TLS protocols that the broker uses to negotiate during TLS handshake.
systemTopicEnabledEnable/Disable system topics.false
topicLevelPoliciesEnabledEnable or disable topic level policies. Topic level policies depends on the system topic. Please enable the system topic first.false
proxyRolesRole names that are treated as “proxy roles”. If the broker sees a request with role as proxyRoles, it demands to see a valid original principal.
authenticateOriginalAuthDataIf this flag is set, the broker authenticates the original Auth data. Otherwise, it just accepts the originalPrincipal and authorizes it (if required).false
authenticationEnabledEnable authentication for the broker.false
authenticationProvidersA comma-separated list of class names for authentication providers.false
authorizationEnabledEnforce authorization in brokers.false
authorizationProviderAuthorization provider fully qualified class-name.org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
authorizationAllowWildcardsMatchingAllow wildcard matching in authorization. Wildcard matching is applicable only when the wildcard-character (*) presents at the first or last position.false
superUserRolesRole names that are treated as “superusers.” Superusers are authorized to perform all admin tasks.
brokerClientAuthenticationPluginThe authentication settings of the broker itself. Used when the broker connects to other brokers either in the same cluster or from other clusters.
brokerClientAuthenticationParametersThe parameters that go along with the plugin specified using brokerClientAuthenticationPlugin.
athenzDomainNamesSupported Athenz authentication provider domain names as a comma-separated list.
anonymousUserRoleWhen this parameter is not empty, unauthenticated users perform as anonymousUserRole.
tokenSettingPrefixConfigure the prefix of the token related setting like tokenSecretKey, tokenPublicKey, tokenAuthClaim, tokenPublicAlg, tokenAudienceClaim, and tokenAudience.
tokenSecretKeyConfigure the secret key to be used to validate auth tokens. The key can be specified like: tokenSecretKey=data:;base64,xxxxxxxxx or tokenSecretKey=file:///my/secret.key. Note: key file must be DER-encoded.
tokenPublicKeyConfigure the public key to be used to validate auth tokens. The key can be specified like: tokenPublicKey=data:;base64,xxxxxxxxx or tokenPublicKey=file:///my/secret.key. Note: key file must be DER-encoded.
tokenAuthClaimSpecify the token claim that will be used as the authentication “principal” or “role”. The “subject” field will be used if this is left blank
tokenAudienceClaimThe token audience “claim” name, e.g. “aud”. It is used to get the audience from token. If it is not set, the audience is not verified.
tokenAudienceThe token audience stands for this broker. The field tokenAudienceClaim of a valid token need contains this parameter.
saslJaasClientAllowedIdsThis is a regexp, which limits the range of possible ids which can connect to the Broker using SASL. By default, it is set to SaslConstants.JAAS_CLIENT_ALLOWED_IDS_DEFAULT, which is “.pulsar.”, so only clients whose id contains ‘pulsar’ are allowed to connect.N/A
saslJaasBrokerSectionNameService Principal, for login context name. By default, it is set to SaslConstants.JAAS_DEFAULT_BROKER_SECTION_NAME, which is “Broker”.N/A
httpMaxRequestSizeIf the value is larger than 0, it rejects all HTTP requests with bodies larged than the configured limit.-1
exposePreciseBacklogInPrometheusEnable expose the precise backlog stats, set false to use published counter and consumed counter to calculate, this would be more efficient but may be inaccurate.false
bookkeeperMetadataServiceUriMetadata service uri is what BookKeeper used for loading corresponding metadata driver and resolving its metadata service location. This value can be fetched using bookkeeper shell whatisinstanceid command in BookKeeper cluster. For example: zk+hierarchical://localhost:2181/ledgers. The metadata service uri list can also be semicolon separated values like: zk+hierarchical://zk1:2181;zk2:2181;zk3:2181/ledgers.N/A
bookkeeperClientAuthenticationPluginAuthentication plugin to be used when connecting to bookies (BookKeeper servers).
bookkeeperClientAuthenticationParametersNameBookKeeper authentication plugin implementation parameters and values.
bookkeeperClientAuthenticationParametersParameters associated with the bookkeeperClientAuthenticationParametersName
bookkeeperClientNumWorkerThreadsNumber of BookKeeper client worker threads. Default is Runtime.getRuntime().availableProcessors()
bookkeeperClientTimeoutInSecondsTimeout for BookKeeper add and read operations.30
bookkeeperClientSpeculativeReadTimeoutInMillisSpeculative reads are initiated if a read request doesn’t complete within a certain time. A value of 0 disables speculative reads.0
bookkeeperUseV2WireProtocolUse older Bookkeeper wire protocol with bookie.true
bookkeeperClientHealthCheckEnabledEnable bookie health checks.true
bookkeeperClientHealthCheckIntervalSecondsThe time interval, in seconds, at which health checks are performed. New ledgers are not created during health checks.60
bookkeeperClientHealthCheckErrorThresholdPerIntervalError threshold for health checks.5
bookkeeperClientHealthCheckQuarantineTimeInSecondsIf bookies have more than the allowed number of failures within the time interval specified by bookkeeperClientHealthCheckIntervalSeconds1800
bookkeeperClientGetBookieInfoIntervalSecondsSpecify options for the GetBookieInfo check. This setting helps ensure the list of bookies that are up to date on the brokers.86400
bookkeeperClientGetBookieInfoRetryIntervalSecondsSpecify options for the GetBookieInfo check. This setting helps ensure the list of bookies that are up to date on the brokers.60
bookkeeperClientRackawarePolicyEnabledtrue
bookkeeperClientRegionawarePolicyEnabledfalse
bookkeeperClientMinNumRacksPerWriteQuorum2
bookkeeperClientMinNumRacksPerWriteQuorumfalse
bookkeeperClientReorderReadSequenceEnabledfalse
bookkeeperClientIsolationGroups
bookkeeperClientSecondaryIsolationGroupsEnable bookie secondary-isolation group if bookkeeperClientIsolationGroups doesn't have enough bookie available.
bookkeeperClientMinAvailableBookiesInIsolationGroupsMinimum bookies that should be available as part of bookkeeperClientIsolationGroups else broker will include bookkeeperClientSecondaryIsolationGroups bookies in isolated list.
bookkeeperTLSProviderFactoryClassSet the client security provider factory class name.org.apache.bookkeeper.tls.TLSContextFactory
bookkeeperTLSClientAuthenticationEnable TLS authentication with bookie.false
bookkeeperTLSKeyFileTypeSupported type: PEM, JKS, PKCS12.PEM
bookkeeperTLSTrustCertTypesSupported type: PEM, JKS, PKCS12.PEM
bookkeeperTLSKeyStorePasswordPathPath to file containing keystore password, if the client keystore is password protected.
bookkeeperTLSTrustStorePasswordPathPath to file containing truststore password, if the client truststore is password protected.
bookkeeperTLSKeyFilePathPath for the TLS private key file.
bookkeeperTLSCertificateFilePathPath for the TLS certificate file.
bookkeeperTLSTrustCertsFilePathPath for the trusted TLS certificate file.
bookkeeperTlsCertFilesRefreshDurationSecondsTls cert refresh duration at bookKeeper-client in seconds (0 to disable check).
bookkeeperDiskWeightBasedPlacementEnabledEnable/Disable disk weight based placement.false
bookkeeperExplicitLacIntervalInMillsSet the interval to check the need for sending an explicit LAC. When the value is set to 0, no explicit LAC is sent.0
bookkeeperClientExposeStatsToPrometheusExpose BookKeeper client managed ledger stats to Prometheus.false
managedLedgerDefaultEnsembleSize1
managedLedgerDefaultWriteQuorum1
managedLedgerDefaultAckQuorum1
managedLedgerDigestTypeDefault type of checksum to use when writing to BookKeeper.CRC32C
managedLedgerNumWorkerThreadsNumber of threads to be used for managed ledger tasks dispatching.Runtime.getRuntime().availableProcessors()
managedLedgerNumSchedulerThreadsNumber of threads to be used for managed ledger scheduled tasks.Runtime.getRuntime().availableProcessors()
managedLedgerCacheSizeMBN/A
managedLedgerCacheCopyEntriesWhether to copy the entry payloads when inserting in cache.false
managedLedgerCacheEvictionWatermark0.9
managedLedgerCacheEvictionFrequencyConfigure the cache eviction frequency for the managed ledger cache (evictions/sec)100.0
managedLedgerCacheEvictionTimeThresholdMillisAll entries that have stayed in cache for more than the configured time, will be evicted1000
managedLedgerCursorBackloggedThresholdConfigure the threshold (in number of entries) from where a cursor should be considered ‘backlogged’ and thus should be set as inactive.1000
managedLedgerUnackedRangesOpenCacheSetEnabledUse Open Range-Set to cache unacknowledged messagestrue
managedLedgerDefaultMarkDeleteRateLimit0.1
managedLedgerMaxEntriesPerLedger50000
managedLedgerMinLedgerRolloverTimeMinutes10
managedLedgerMaxLedgerRolloverTimeMinutes240
managedLedgerCursorMaxEntriesPerLedger50000
managedLedgerCursorRolloverTimeInSeconds14400
managedLedgerMaxSizePerLedgerMbytesMaximum ledger size before triggering a rollover for a topic.2048
managedLedgerMaxUnackedRangesToPersistMaximum number of “acknowledgment holes” that are going to be persistently stored. When acknowledging out of order, a consumer leaves holes that are supposed to be quickly filled by acknowledging all the messages. The information of which messages are acknowledged is persisted by compressing in “ranges” of messages that were acknowledged. After the max number of ranges is reached, the information is only tracked in memory and messages are redelivered in case of crashes.10000
managedLedgerMaxUnackedRangesToPersistInZooKeeperMaximum number of “acknowledgment holes” that can be stored in Zookeeper. If the number of unacknowledged message range is higher than this limit, the broker persists unacknowledged ranges into bookkeeper to avoid additional data overhead into Zookeeper.1000
autoSkipNonRecoverableDatafalse
managedLedgerMetadataOperationsTimeoutSecondsOperation timeout while updating managed-ledger metadata.60
managedLedgerReadEntryTimeoutSecondsRead entries timeout when the broker tries to read messages from BookKeeper.0
managedLedgerAddEntryTimeoutSecondsAdd entry timeout when the broker tries to publish message to BookKeeper.0
managedLedgerNewEntriesCheckDelayInMillisNew entries check delay for the cursor under the managed ledger. If no new messages in the topic, the cursor tries to check again after the delay time. For consumption latency sensitive scenario, you can set the value to a smaller value or 0. Of course, a smaller value may degrade consumption throughput.10 ms
managedLedgerPrometheusStatsLatencyRolloverSecondsManaged ledger prometheus stats latency rollover seconds.60
managedLedgerTraceTaskExecutionWhether to trace managed ledger task execution time.true
managedLedgerNewEntriesCheckDelayInMillisNew entries check delay for the cursor under the managed ledger. If no new messages in the topic, the cursor will try to check again after the delay time. For consumption latency sensitive scenario, it can be set to a smaller value or 0. A smaller value degrades consumption throughput. By default, it is 10ms.10
loadBalancerEnabledfalse
loadBalancerPlacementStrategyweightedRandomSelection
loadBalancerReportUpdateThresholdPercentage10
loadBalancerReportUpdateMaxIntervalMinutes15
loadBalancerHostUsageCheckIntervalMinutes1
loadBalancerSheddingIntervalMinutes30
loadBalancerSheddingGracePeriodMinutes30
loadBalancerBrokerMaxTopics50000
loadBalancerBrokerUnderloadedThresholdPercentage1
loadBalancerBrokerOverloadedThresholdPercentage85
loadBalancerResourceQuotaUpdateIntervalMinutes15
loadBalancerBrokerComfortLoadLevelPercentage65
loadBalancerAutoBundleSplitEnabledfalse
loadBalancerAutoUnloadSplitBundlesEnabledEnable/Disable automatic unloading of split bundles.true
loadBalancerNamespaceBundleMaxTopics1000
loadBalancerNamespaceBundleMaxSessions1000
loadBalancerNamespaceBundleMaxMsgRate1000
loadBalancerNamespaceBundleMaxBandwidthMbytes100
loadBalancerNamespaceMaximumBundles128
loadBalancerBrokerThresholdShedderPercentageThe broker resource usage threshold. When the broker resource usage is greater than the pulsar cluster average resource usage, the threshold shedder is triggered to offload bundles from the broker. It only takes effect in the ThresholdShedder strategy.10
loadBalancerMsgRateDifferenceShedderThresholdMessage-rate percentage threshold between highest and least loaded brokers for uniform load shedding.50
loadBalancerMsgThroughputMultiplierDifferenceShedderThresholdMessage-throughput threshold between highest and least loaded brokers for uniform load shedding.4
loadBalancerHistoryResourcePercentageThe history usage when calculating new resource usage. It only takes effect in the ThresholdShedder strategy.0.9
loadBalancerBandwithInResourceWeightThe BandWithIn usage weight when calculating new resource usage. It only takes effect in the ThresholdShedder strategy.1.0
loadBalancerBandwithOutResourceWeightThe BandWithOut usage weight when calculating new resource usage. It only takes effect in the ThresholdShedder strategy.1.0
loadBalancerCPUResourceWeightThe CPU usage weight when calculating new resource usage. It only takes effect in the ThresholdShedder strategy.1.0
loadBalancerMemoryResourceWeightThe heap memory usage weight when calculating new resource usage. It only takes effect in the ThresholdShedder strategy.1.0
loadBalancerDirectMemoryResourceWeightThe direct memory usage weight when calculating new resource usage. It only takes effect in the ThresholdShedder strategy.1.0
loadBalancerBundleUnloadMinThroughputThresholdBundle unload minimum throughput threshold. Avoid bundle unload frequently. It only takes effect in the ThresholdShedder strategy.10
namespaceBundleUnloadingTimeoutMsTime to wait for the unloading of a namespace bundle in milliseconds.60000
replicationMetricsEnabledtrue
replicationConnectionsPerBroker16
replicationProducerQueueSize1000
replicationPolicyCheckDurationSecondsDuration to check replication policy to avoid replicator inconsistency due to missing ZooKeeper watch. When the value is set to 0, disable checking replication policy.600
defaultRetentionTimeInMinutes0
defaultRetentionSizeInMB0
keepAliveIntervalSeconds30
haProxyProtocolEnabledEnable or disable the HAProxy protocol.false
bookieIdIf you want to custom a bookie ID or use a dynamic network address for a bookie, you can set the bookieId.

Bookie advertises itself using the bookieId rather than the BookieSocketAddress (hostname:port or IP:port).

The bookieId is a non-empty string that can contain ASCII digits and letters ([a-zA-Z9-0]), colons, dashes, and dots.

For more information about bookieId, see here.
/
maxTopicsPerNamespaceThe maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check.0

WebSocket

NameDescriptionDefault
configurationStoreServers
zooKeeperSessionTimeoutMillis30000
zooKeeperCacheExpirySecondsZooKeeper cache expiry time in seconds300
serviceUrl
serviceUrlTls
brokerServiceUrl
brokerServiceUrlTls
webServicePort8080
webServicePortTls8443
bindAddress0.0.0.0
clusterName
authenticationEnabledfalse
authenticationProviders
authorizationEnabledfalse
superUserRoles
brokerClientAuthenticationPlugin
brokerClientAuthenticationParameters
tlsEnabledfalse
tlsAllowInsecureConnectionfalse
tlsCertificateFilePath
tlsKeyFilePath
tlsTrustCertsFilePath

Pulsar proxy

The Pulsar proxy can be configured in the conf/proxy.conf file.

NameDescriptionDefault
forwardAuthorizationCredentialsForward client authorization credentials to Broker for re-authorization, and make sure authentication is enabled for this to take effect.false
zookeeperServersThe ZooKeeper quorum connection string (as a comma-separated list)
configurationStoreServersConfiguration store connection string (as a comma-separated list)
brokerServiceURLThe service URL pointing to the broker cluster.
brokerServiceURLTLSThe TLS service URL pointing to the broker cluster
brokerWebServiceURLThe Web service URL pointing to the broker cluster
brokerWebServiceURLTLSThe TLS Web service URL pointing to the broker cluster
functionWorkerWebServiceURLThe Web service URL pointing to the function worker cluster. It is only configured when you setup function workers in a separate cluster.
functionWorkerWebServiceURLTLSThe TLS Web service URL pointing to the function worker cluster. It is only configured when you setup function workers in a separate cluster.
zookeeperSessionTimeoutMsZooKeeper session timeout (in milliseconds)30000
zooKeeperCacheExpirySecondsZooKeeper cache expiry time in seconds300
advertisedAddressHostname or IP address the service advertises to the outside world. If not set, the value of InetAddress.getLocalHost().getHostname() is used.N/A
servicePortThe port to use for server binary Protobuf requests6650
servicePortTlsThe port to use to server binary Protobuf TLS requests6651
statusFilePathPath for the file used to determine the rotation status for the proxy instance when responding to service discovery health checks
proxyLogLevelProxy log level 0: Do not log any TCP channel information. 1: Parse and log any TCP channel information and command information without message body. 2: Parse and log channel information, command information and message body.0
authenticationEnabledWhether authentication is enabled for the Pulsar proxyfalse
authenticateMetricsEndpointWhether the ‘/metrics’ endpoint requires authentication. Defaults to true. ‘authenticationEnabled’ must also be set for this to take effect.true
authenticationProvidersAuthentication provider name list (a comma-separated list of class names)
authorizationEnabledWhether authorization is enforced by the Pulsar proxyfalse
authorizationProviderAuthorization provider as a fully qualified class nameorg.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
anonymousUserRoleWhen this parameter is not empty, unauthenticated users perform as anonymousUserRole.
brokerClientAuthenticationPluginThe authentication plugin used by the Pulsar proxy to authenticate with Pulsar brokers
brokerClientAuthenticationParametersThe authentication parameters used by the Pulsar proxy to authenticate with Pulsar brokers
brokerClientTrustCertsFilePathThe path to trusted certificates used by the Pulsar proxy to authenticate with Pulsar brokers
superUserRolesRole names that are treated as “super-users,” meaning that they will be able to perform all admin
maxConcurrentInboundConnectionsMax concurrent inbound connections. The proxy will reject requests beyond that.10000
maxConcurrentLookupRequestsMax concurrent outbound connections. The proxy will error out requests beyond that.50000
tlsEnabledInProxyDeprecated - use servicePortTls and webServicePortTls instead.false
tlsEnabledWithBrokerWhether TLS is enabled when communicating with Pulsar brokers.false
tlsCertRefreshCheckDurationSecTLS certificate refresh duration in seconds. If the value is set 0, check TLS certificate every new connection.300
tlsCertificateFilePathPath for the TLS certificate file
tlsKeyFilePathPath for the TLS private key file
tlsTrustCertsFilePathPath for the trusted TLS certificate pem file
tlsHostnameVerificationEnabledWhether the hostname is validated when the proxy creates a TLS connection with brokersfalse
tlsRequireTrustedClientCertOnConnectWhether client certificates are required for TLS. Connections are rejected if the client certificate isn’t trusted.false
tlsProtocolsSpecify the tls protocols the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLSv1.3, TLSv1.2
tlsCiphersSpecify the tls cipher the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
httpReverseProxyConfigsHTTP directs to redirect to non-pulsar services
httpOutputBufferSizeHTTP output buffer size. The amount of data that will be buffered for HTTP requests before it is flushed to the channel. A larger buffer size may result in higher HTTP throughput though it may take longer for the client to see data. If using HTTP streaming via the reverse proxy, this should be set to the minimum value (1) so that clients see the data as soon as possible.32768
httpNumThreadsNumber of threads to use for HTTP requests processing2 * Runtime.getRuntime().availableProcessors()
tokenSettingPrefixConfigure the prefix of the token related setting like tokenSecretKey, tokenPublicKey, tokenAuthClaim, tokenPublicAlg, tokenAudienceClaim, and tokenAudience.
tokenSecretKeyConfigure the secret key to be used to validate auth tokens. The key can be specified like: tokenSecretKey=data:;base64,xxxxxxxxx or tokenSecretKey=file:///my/secret.key. Note: key file must be DER-encoded.
tokenPublicKeyConfigure the public key to be used to validate auth tokens. The key can be specified like: tokenPublicKey=data:;base64,xxxxxxxxx or tokenPublicKey=file:///my/secret.key. Note: key file must be DER-encoded.
tokenAuthClaimSpecify the token claim that will be used as the authentication “principal” or “role”. The “subject” field will be used if this is left blank
tokenAudienceClaimThe token audience “claim” name, e.g. “aud”. It is used to get the audience from token. If it is not set, the audience is not verified.
tokenAudienceThe token audience stands for this broker. The field tokenAudienceClaim of a valid token need contains this parameter.
haProxyProtocolEnabledEnable or disable the HAProxy protocol.false

ZooKeeper

ZooKeeper handles a broad range of essential configuration- and coordination-related tasks for Pulsar. The default configuration file for ZooKeeper is in the conf/zookeeper.conf file in your Pulsar installation. The following parameters are available:

NameDescriptionDefault
tickTimeThe tick is the basic unit of time in ZooKeeper, measured in milliseconds and used to regulate things like heartbeats and timeouts. tickTime is the length of a single tick.2000
initLimitThe maximum time, in ticks, that the leader ZooKeeper server allows follower ZooKeeper servers to successfully connect and sync. The tick time is set in milliseconds using the tickTime parameter.10
syncLimitThe maximum time, in ticks, that a follower ZooKeeper server is allowed to sync with other ZooKeeper servers. The tick time is set in milliseconds using the tickTime parameter.5
dataDirThe location where ZooKeeper will store in-memory database snapshots as well as the transaction log of updates to the database.data/zookeeper
clientPortThe port on which the ZooKeeper server will listen for connections.2181
admin.enableServerThe port at which the admin listens.true
admin.serverPortThe port at which the admin listens.9990
autopurge.snapRetainCountIn ZooKeeper, auto purge determines how many recent snapshots of the database stored in dataDir to retain within the time interval specified by autopurge.purgeInterval (while deleting the rest).3
autopurge.purgeIntervalThe time interval, in hours, by which the ZooKeeper database purge task is triggered. Setting to a non-zero number will enable auto purge; setting to 0 will disable. Read this guide before enabling auto purge.1
forceSyncRequires updates to be synced to media of the transaction log before finishing processing the update. If this option is set to ‘no’, ZooKeeper will not require updates to be synced to the media. WARNING: it's not recommended to run a production ZK cluster with forceSync disabled.yes
maxClientCnxnsThe maximum number of client connections. Increase this if you need to handle more ZooKeeper clients.60

In addition to the parameters in the table above, configuring ZooKeeper for Pulsar involves adding a server.N line to the conf/zookeeper.conf file for each node in the ZooKeeper cluster, where N is the number of the ZooKeeper node. Here's an example for a three-node ZooKeeper cluster:

server.1=zk1.us-west.example.com:2888:3888
server.2=zk2.us-west.example.com:2888:3888
server.3=zk3.us-west.example.com:2888:3888

We strongly recommend consulting the ZooKeeper Administrator's Guide for a more thorough and comprehensive introduction to ZooKeeper configuration