commit | e93a7f1f9b2cfaa171e46a21b8e347c9fd3b0921 | [log] [tgz] |
---|---|---|
author | wenbingshen <oliver.shen999@gmail.com> | Wed May 31 12:04:54 2023 +0800 |
committer | GitHub <noreply@github.com> | Wed May 31 12:04:54 2023 +0800 |
tree | 594e2bbb503b3da597b8acd5a71fda00425c8451 | |
parent | 575a81c79e90e256e919287ef9762bc14d15789a [diff] |
check indexBaseDir specified with ledgerBaseDir (#3967) ### Motivation I deployed the pulsar-3.0.0 version, and I observed the log of bookkeeper-4.16.1 as follows, I did not config special indexDirectories, but the printed log told me that indexDirectories was specified, which is wrong. ``` 2023-05-23T19:52:45,192+0800 [main] INFO org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage - indexDir is specified, creating single directory db ledger storage on /data0/pulsar/data/ledgers/current 2023-05-23T19:52:45,322+0800 [main] INFO org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB - Searching for a RocksDB configuration file in /usr/local/pulsar/apache-pulsar-3.0.0/conf/ledger_metadata_rocksdb.conf 2023-05-23T19:52:45,322+0800 [main] INFO org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB - Found a RocksDB configuration file and using it to initialize the RocksDB 2023-05-23T19:52:45,337+0800 [main] INFO org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB - Searching for a RocksDB configuration file in /usr/local/pulsar/apache-pulsar-3.0.0/conf/entry_location_rocksdb.conf 2023-05-23T19:52:45,337+0800 [main] INFO org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB - Found a RocksDB configuration file and using it to initialize the RocksDB ``` here ledgerDirsManager == indexDirsManager, so ledgerDir is always equals indexDir. https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/DbLedgerStorage.java#L181 ```java for (int i = 0; i < ledgerDirsManager.getAllLedgerDirs().size(); i++) { File ledgerDir = ledgerDirsManager.getAllLedgerDirs().get(i); File indexDir = indexDirsManager.getAllLedgerDirs().get(i); // Create a ledger dirs manager for the single directory File[] lDirs = new File[1]; // Remove the `/current` suffix which will be appended again by LedgersDirManager lDirs[0] = ledgerDir.getParentFile(); LedgerDirsManager ldm = new LedgerDirsManager(conf, lDirs, ledgerDirsManager.getDiskChecker(), NullStatsLogger.INSTANCE); // Create a index dirs manager for the single directory File[] iDirs = new File[1]; // Remove the `/current` suffix which will be appended again by LedgersDirManager iDirs[0] = indexDir.getParentFile(); LedgerDirsManager idm = new LedgerDirsManager(conf, iDirs, indexDirsManager.getDiskChecker(), NullStatsLogger.INSTANCE); ...... ledgerStorageList.add(newSingleDirectoryDbLedgerStorage(conf, ledgerManager, ldm, idm, entrylogger, statsLogger, perDirectoryWriteCacheSize, perDirectoryReadCacheSize, readAheadCacheBatchSize, readAheadCacheBatchBytesSize)); ...... } ```
Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.
It is suitable for being used in following scenarios:
Please visit the Documentation from the project website for more information.
For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue.
Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.
Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.
Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.
We feel that a welcoming open community is important and welcome contributions.
See our installation guide to get your local environment setup.
Take a look at our open issues: Github Issues.
Review our coding style and follow our pull requests to learn more about our conventions.
Make your changes according to our contributing guide