| commit | ad3f0a30181aa4d7ba099ee67fd45fa98cf9c2c0 | [log] [tgz] |
|---|---|---|
| author | Bilal Waheed <23331711+mbwaheed@users.noreply.github.com> | Mon May 18 12:19:01 2020 -0700 |
| committer | GitHub <noreply@github.com> | Mon May 18 15:19:01 2020 -0400 |
| tree | 68b57edbd05fb2ea34468e37855b965cb27e5066 | |
| parent | 738dbb35c7c1db82141d90b200c0bc69c6fa1b9b [diff] |
SOLR-13101: SHARED replica's distributed indexing (#1430) * SOLR-13101: SHARED replica's distributed indexing The basic purpose of this change was to refactor most of the SHARED replica logic out of DistributedZkUpdateProcessor. Along with that refactoring I came across couple of functional issues in the logic that have been fixed too. Functional fixes: -If a replica looses its leadership in the middle of indexing batch, it should still push its changes to the shared store. -SHARED replica does not need to process soft commits and does not need to broadcast hard commits to all the shards of a collection. -Previously we would pull from the shared store even when the doc being added/deleted is not meant for the current core/shard. -Previously DistributedZkUpdateProcessor#processDelete was running its pull logic even before the request had been setup (DistributedZkUpdateProcessor#setupRequest). -DistributedZkUpdateProcessorTest have been deleted in favor of new SharedStoreDistributedIndexingTest SharedCoreIndexingBatchProcessorTest. Refactoring: -Most of DistributedZkUpdateProcessor's SHARED replica logic is moved into a new SharedCoreIndexingBatchProcessor. The purpose of this class is to pull from the shared store at the start of an indexing batch (if the core is stale) and push to the shared store at the end of a successfully committed indexing batch. -CoreUpdateTracker has been deleted and its only persistShardIndexToSharedStore method has been renamed to pushCoreToSharedStore and moved to CorePusher. -BlobStoreUtilsTest#syncLocalCoreWithSharedStore is renamed to pullCoreFromSharedStore and moved into a new CorePuller class and the tests to CorePullerTests. -I did rename phrase "blob store" to "shared store" at some places in the changed classes. But it was not meant to be an exhaustive attempt. * -Throw error for pull request for an unknown. -Log warning when indexing a non-active shard. * Address CR feedback.
Apache Lucene is a high-performance, full featured text search engine library written in Java.
Apache Solr is an enterprise search platform written using Apache Lucene. Major features include full-text search, index replication and sharding, and result faceting and highlighting.
This README file only contains basic setup instructions. For more comprehensive documentation, visit:
(You do not need to do this if you downloaded a pre-built package.)
Lucene and Solr are built using Apache Ant. To build Lucene and Solr, run:
ant compile
If you see an error about Ivy missing while invoking Ant (e.g., .ant/lib does not exist), run ant ivy-bootstrap and retry.
Sometimes you may face issues with Ivy (e.g., an incompletely downloaded artifact). Cleaning up the Ivy cache and retrying is a workaround for most of such issues:
rm -rf ~/.ivy2/cache
The Solr server can then be packaged and prepared for startup by running the following command from the solr/ directory:
ant server
After building Solr, the server can be started using the bin/solr control scripts. Solr can be run in either standalone or distributed (SolrCloud mode).
To run Solr in standalone mode, run the following command from the solr/ directory:
bin/solr start
To run Solr in SolrCloud mode, run the following command from the solr/ directory:
bin/solr start -c
The bin/solr control script allows heavy modification of the started Solr. Common options are described in some detail in solr/README.txt. For an exhaustive treatment of options, run bin/solr start -h from the solr/ directory.
Ant can be used to generate project files compatible with most common IDEs. Run the ant command corresponding to your IDE of choice before attempting to import Lucene/Solr.
ant eclipse (See this for details)ant idea (See this for details)ant netbeans (See this for details)The standard test suite can be run with the command:
ant test
Like Solr itself, the test-running can be customized or tailored in a number or ways. For an exhaustive discussion of the options available, run:
ant test-help
Please review the Contributing to Solr Guide for information on contributing.
#solr and #solr-dev on freenode.net