| <!DOCTYPE html> |
| <!--[if IE]><![endif]--> |
| <html> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> |
| <title>Namespace Lucene.Net.Search |
| | Apache Lucene.NET 4.8.0-beta00008 Documentation </title> |
| <meta name="viewport" content="width=device-width"> |
| <meta name="title" content="Namespace Lucene.Net.Search |
| | Apache Lucene.NET 4.8.0-beta00008 Documentation "> |
| <meta name="generator" content="docfx 2.50.0.0"> |
| |
| <link rel="shortcut icon" href="../../logo/favicon.ico"> |
| <link rel="stylesheet" href="../../styles/docfx.vendor.css"> |
| <link rel="stylesheet" href="../../styles/docfx.css"> |
| <link rel="stylesheet" href="../../styles/main.css"> |
| <meta property="docfx:navrel" content="../../toc.html"> |
| <meta property="docfx:tocrel" content="../toc.html"> |
| |
| <meta property="docfx:rel" content="../../"> |
| |
| </head> |
| <body data-spy="scroll" data-target="#affix" data-offset="120"> |
| <div id="wrapper"> |
| <header> |
| |
| <nav id="autocollapse" class="navbar ng-scope" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| |
| <a class="navbar-brand" href="../../index.html"> |
| <img id="logo" class="svg" src="../../logo/lucene-net-color.png" alt=""> |
| </a> |
| </div> |
| <div class="collapse navbar-collapse" id="navbar"> |
| <form class="navbar-form navbar-right" role="search" id="search"> |
| <div class="form-group"> |
| <input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off"> |
| </div> |
| </form> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="subnav navbar navbar-default"> |
| <div class="container hide-when-search" id="breadcrumb"> |
| <ul class="breadcrumb"> |
| <li></li> |
| </ul> |
| </div> |
| </div> |
| </header> |
| <div class="container body-content"> |
| |
| <div id="search-results"> |
| <div class="search-list"></div> |
| <div class="sr-items"> |
| <p><i class="glyphicon glyphicon-refresh index-loading"></i></p> |
| </div> |
| <ul id="pagination"></ul> |
| </div> |
| </div> |
| <div role="main" class="container body-content hide-when-search"> |
| |
| <div class="sidenav hide-when-search"> |
| <a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a> |
| <div class="sidetoggle collapse" id="sidetoggle"> |
| <div id="sidetoc"></div> |
| </div> |
| </div> |
| <div class="article row grid-right"> |
| <div class="col-md-10"> |
| <article class="content wrap" id="_content" data-uid="Lucene.Net.Search"> |
| |
| <h1 id="Lucene_Net_Search" data-uid="Lucene.Net.Search" class="text-break">Namespace Lucene.Net.Search |
| </h1> |
| <div class="markdown level0 summary"><!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <p>Code to search indices.</p> |
| <h2 id="table-of-contents">Table Of Contents</h2> |
| <ol> |
| <li><a href="#search">Search Basics</a> 2. <a href="#query">The Query Classes</a> 3. <a href="#scoring">Scoring: Introduction</a> 4. <a href="#scoringBasics">Scoring: Basics</a> 5. <a href="#changingscoring">Changing the Scoring</a> 6. <a href="#algorithm">Appendix: Search Algorithm</a> </li> |
| </ol> |
| <h2 id="search-basics">Search Basics</h2> |
| <p> Lucene offers a wide variety of <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> implementations, most of which are in this package, its subpackages (<a class="xref" href="../Lucene.Net/Lucene.Net.Search.Spans.html">spans</a>, <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Payloads.html">payloads</a>), or the <a href="{@docRoot}/../queries/overview-summary.html">queries module</a>. These implementations can be combined in a wide variety of ways to provide complex querying capabilities along with information about where matches took place in the document collection. The <a href="#query">Query Classes</a> section below highlights some of the more important Query classes. For details on implementing your own Query class, see <a href="#customQueriesExpert">Custom Queries -- Expert Level</a> below. </p> |
| <p> To perform a search, applications usually call <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">#search(Query,int)</a> or <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">#search(Query,Filter,int)</a>. </p> |
| <p> Once a Query has been created and submitted to the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a>, the scoring process begins. After some infrastructure setup, control finally passes to the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a> implementation and its <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> or <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html">BulkScore</a> instances. See the <a href="#algorithm">Algorithm</a> section for more notes on the process. </p> |
| <pre><code><!-- TODO: this page over-links the same things too many times --> |
| </code></pre><h2 id="query-classes">Query Classes</h2> |
| <h4 id="-"></h4> |
| <pre><code>[TermQuery](xref:Lucene.Net.Search.TermQuery) |
| </code></pre><p>Of the various implementations of <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a>, the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a> is the easiest to understand and the most often used in applications. A <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a> matches all the documents that contain the specified <a class="xref" href="../Lucene.Net/Lucene.Net.Index.Term.html">Term</a>, which is a word that occurs in a certain <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Field.html">Field</a>. Thus, a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a> identifies and scores all <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Document.html">Document</a>s that have a <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Field.html">Field</a> with the specified string in it. Constructing a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a> is as simple as: TermQuery tq = new TermQuery(new Term("fieldName", "term")); In this example, the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> identifies all <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Document.html">Document</a>s that have the <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Field.html">Field</a> named <tt>"fieldName"</tt> containing the word <tt>"term"</tt>. </p> |
| <h4 id="--1"></h4> |
| <pre><code>[BooleanQuery](xref:Lucene.Net.Search.BooleanQuery) |
| </code></pre><p>Things start to get interesting when one combines multiple <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a> instances into a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanQuery.html">BooleanQuery</a>. A <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanQuery.html">BooleanQuery</a> contains multiple <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanClause.html">BooleanClause</a>s, where each clause contains a sub-query (<a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> instance) and an operator (from <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanClause.html#Lucene_Net_Search_BooleanClause_Occur">BooleanClause.Occur</a>) describing how that sub-query is combined with the other clauses: 1. <p><a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanClause.html#methods">SHOULD</a> — Use this operator when a clause can occur in the result set, but is not required. If a query is made up of all SHOULD clauses, then every document in the result set matches at least one of these clauses.</p> 2. <p><a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanClause.html#methods">MUST</a> — Use this operator when a clause is required to occur in the result set. Every document in the result set will match all such clauses.</p> 3. <p><a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanClause.html#methods">NOT</a> — Use this operator when a clause must not occur in the result set. No document in the result set will match any such clauses.</p> Boolean queries are constructed by adding two or more <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanClause.html">BooleanClause</a> instances. If too many clauses are added, a <a href="xref:Lucene.Net.Search.BooleanQuery.TooManyClauses">TooManyClauses</a> exception will be thrown during searching. This most often occurs when a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> is rewritten into a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanQuery.html">BooleanQuery</a> with many <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a> clauses, for example by <a class="xref" href="../Lucene.Net/Lucene.Net.Search.WildcardQuery.html">WildcardQuery</a>. The default setting for the maximum number of clauses 1024, but this can be changed via the static method <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanQuery.html">#setMaxClauseCount(int)</a>. </p> |
| <h4 id="phrases">Phrases</h4> |
| <p>Another common search is to find documents containing certain phrases. This |
| is handled three different ways:</p> |
| <ol> |
| <li></li> |
| </ol> |
| <p><a class="xref" href="../Lucene.Net/Lucene.Net.Search.PhraseQuery.html">PhraseQuery</a> — Matches a sequence of <a class="xref" href="../Lucene.Net/Lucene.Net.Index.Term.html">Term</a>s. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.PhraseQuery.html">PhraseQuery</a> uses a slop factor to determine how many positions may occur between any two terms in the phrase and still be considered a match. The slop is 0 by default, meaning the phrase must match exactly.</p> |
| <ol> |
| <li></li> |
| </ol> |
| <p><a class="xref" href="../Lucene.Net/Lucene.Net.Search.MultiPhraseQuery.html">MultiPhraseQuery</a> — A more general form of PhraseQuery that accepts multiple Terms for a position in the phrase. For example, this can be used to perform phrase queries that also incorporate synonyms. 3. <p><a class="xref" href="../Lucene.Net/Lucene.Net.Search.Spans.SpanNearQuery.html">SpanNearQuery</a> — Matches a sequence of other <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Spans.SpanQuery.html">SpanQuery</a> instances. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Spans.SpanNearQuery.html">SpanNearQuery</a> allows for much more complicated phrase queries since it is constructed from other <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Spans.SpanQuery.html">SpanQuery</a> instances, instead of only <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a> instances.</p> </p> |
| <h4 id="--2"></h4> |
| <pre><code>[TermRangeQuery](xref:Lucene.Net.Search.TermRangeQuery) |
| </code></pre><p>The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermRangeQuery.html">TermRangeQuery</a> matches all documents that occur in the exclusive range of a lower <a class="xref" href="../Lucene.Net/Lucene.Net.Index.Term.html">Term</a> and an upper <a class="xref" href="../Lucene.Net/Lucene.Net.Index.Term.html">Term</a> according to <a class="xref" href="../Lucene.Net/Lucene.Net.Index.TermsEnum.html#methods">TermsEnum.getComparator</a>. It is not intended for numerical ranges; use <a class="xref" href="../Lucene.Net/Lucene.Net.Search.NumericRangeQuery.html">NumericRangeQuery</a> instead. For example, one could find all documents that have terms beginning with the letters <tt>a</tt> through <tt>c</tt>. </p> |
| <h4 id="--3"></h4> |
| <pre><code>[NumericRangeQuery](xref:Lucene.Net.Search.NumericRangeQuery) |
| </code></pre><p>The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.NumericRangeQuery.html">NumericRangeQuery</a> matches all documents that occur in a numeric range. For NumericRangeQuery to work, you must index the values using a one of the numeric fields (<a href="xref:Lucene.Net.Documents.IntField">IntField</a>, <a href="xref:Lucene.Net.Documents.LongField">LongField</a>, <a href="xref:Lucene.Net.Documents.FloatField">FloatField</a>, or <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.DoubleField.html">DoubleField</a>). </p> |
| <h4 id="--4"></h4> |
| <pre><code>[PrefixQuery](xref:Lucene.Net.Search.PrefixQuery), |
| [WildcardQuery](xref:Lucene.Net.Search.WildcardQuery), |
| [RegexpQuery](xref:Lucene.Net.Search.RegexpQuery) |
| </code></pre><p>While the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.PrefixQuery.html">PrefixQuery</a> has a different implementation, it is essentially a special case of the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.WildcardQuery.html">WildcardQuery</a>. The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.PrefixQuery.html">PrefixQuery</a> allows an application to identify all documents with terms that begin with a certain string. The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.WildcardQuery.html">WildcardQuery</a> generalizes this by allowing for the use of <tt><em></em></tt> (matches 0 or more characters) and <tt>?</tt> (matches exactly one character) wildcards. Note that the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.WildcardQuery.html">WildcardQuery</a> can be quite slow. Also note that <a class="xref" href="../Lucene.Net/Lucene.Net.Search.WildcardQuery.html">WildcardQuery</a> should not start with <tt></tt> and <tt>?</tt>, as these are extremely slow. Some QueryParsers may not allow this by default, but provide a <code>setAllowLeadingWildcard</code> method to remove that protection. The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.RegexpQuery.html">RegexpQuery</a> is even more general than WildcardQuery, allowing an application to identify all documents with terms that match a regular expression pattern. </p> |
| <h4 id="--5"></h4> |
| <pre><code>[FuzzyQuery](xref:Lucene.Net.Search.FuzzyQuery) |
| </code></pre><p>A <a class="xref" href="../Lucene.Net/Lucene.Net.Search.FuzzyQuery.html">FuzzyQuery</a> matches documents that contain terms similar to the specified term. Similarity is determined using <a href="http://en.wikipedia.org/wiki/Levenshtein">Levenshtein (edit) distance</a>. This type of query can be useful when accounting for spelling variations in the collection. </p> |
| <h2 id="scoring--introduction">Scoring — Introduction</h2> |
| <p>Lucene scoring is the heart of why we all love Lucene. It is blazingly fast and it hides almost all of the complexity from the user. In a nutshell, it works. At least, that is, until it doesn't work, or doesn't work as one would expect it to work. Then we are left digging into Lucene internals or asking for help on <a href="mailto:java-user@lucene.apache.org">java-user@lucene.apache.org</a> to figure out why a document with five of our query terms scores lower than a different document with only one of the query terms. </p> |
| <p>While this document won't answer your specific scoring issues, it will, hopefully, point you to the places that can help you figure out the <em>what</em> and <em>why</em> of Lucene scoring. </p> |
| <p>Lucene scoring supports a number of pluggable information retrieval <a href="http://en.wikipedia.org/wiki/Information_retrieval#Model_types">models</a>, including: * <a href="http://en.wikipedia.org/wiki/Vector_Space_Model">Vector Space Model (VSM)</a> * <a href="http://en.wikipedia.org/wiki/Probabilistic_relevance_model">Probablistic Models</a> such as <a href="http://en.wikipedia.org/wiki/Probabilistic_relevance_model_(BM25)">Okapi BM25</a> and <a href="http://en.wikipedia.org/wiki/Divergence-from-randomness_model">DFR</a> * <a href="http://en.wikipedia.org/wiki/Language_model">Language models</a> These models can be plugged in via the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.html">Similarity API</a>, and offer extension hooks and parameters for tuning. In general, Lucene first finds the documents that need to be scored based on boolean logic in the Query specification, and then ranks this subset of matching documents via the retrieval model. For some valuable references on VSM and IR in general refer to <a href="http://wiki.apache.org/lucene-java/InformationRetrieval">Lucene Wiki IR references</a>. </p> |
| <p>The rest of this document will cover <a href="#scoringbasics">Scoring basics</a> and explain how to change your <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.html">Similarity</a>. Next, it will cover ways you can customize the lucene internals in <a href="#customQueriesExpert">Custom Queries -- Expert Level</a>, which gives details on implementing your own <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> class and related functionality. Finally, we will finish up with some reference material in the <a href="#algorithm">Appendix</a>. </p> |
| <h2 id="scoring--basics">Scoring — Basics</h2> |
| <p>Scoring is very much dependent on the way documents are indexed, so it is important to understand |
| indexing. (see <a href="{@docRoot}/overview-summary.html#overview_description">Lucene overview</a> |
| before continuing on with this section) Be sure to use the useful |
| <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html#methods">Doc)</a> |
| to understand how the score for a certain matching document was |
| computed.</p> |
| <p>Generally, the Query determines which documents match (a binary decision), while the Similarity determines how to assign scores to the matching documents. </p> |
| <h4 id="fields-and-documents">Fields and Documents</h4> |
| <p>In Lucene, the objects we are scoring are <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Document.html">Document</a>s. A Document is a collection of <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Field.html">Field</a>s. Each Field has <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.FieldType.html">semantics</a> about how it is created and stored (<a class="xref" href="../Lucene.Net/Lucene.Net.Documents.FieldType.html#methods">Tokenized</a>, <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.FieldType.html#methods">Stored</a>, etc). It is important to note that Lucene scoring works on Fields and then combines the results to return Documents. This is important because two Documents with the exact same content, but one having the content in two Fields and the other in one Field may return different scores for the same query due to length normalization. </p> |
| <h4 id="score-boosting">Score Boosting</h4> |
| <p>Lucene allows influencing search results by "boosting" at different times: * <strong>Index-time boost</strong> by calling <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Field.html#methods">Field.setBoost</a> before a document is added to the index. * <strong>Query-time boost</strong> by setting a boost on a query clause, calling <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html#methods">Query.setBoost</a>. </p> |
| <p>Indexing time boosts are pre-processed for storage efficiency and written to storage for a field as follows: * All boosts of that field (i.e. all boosts under the same field name in that doc) are multiplied. * The boost is then encoded into a normalization value by the Similarity object at index-time: <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.html#methods">ComputeNorm</a>. The actual encoding depends upon the Similarity implementation, but note that most use a lossy encoding (such as multiplying the boost with document length or similar, packed into a single byte!). * Decoding of any index-time normalization values and integration into the document's score is also performed at search time by the Similarity. </p> |
| <h2 id="changing-scoring--similarity">Changing Scoring — Similarity</h2> |
| <p> Changing <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.html">Similarity</a> is an easy way to influence scoring, this is done at index-time with <a class="xref" href="../Lucene.Net/Lucene.Net.Index.IndexWriterConfig.html#methods">IndexWriterConfig.setSimilarity</a> and at query-time with <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html#methods">IndexSearcher.setSimilarity</a>. Be sure to use the same Similarity at query-time as at index-time (so that norms are encoded/decoded correctly); Lucene makes no effort to verify this. </p> |
| <p> You can influence scoring by configuring a different built-in Similarity implementation, or by tweaking its parameters, subclassing it to override behavior. Some implementations also offer a modular API which you can extend by plugging in a different component (e.g. term frequency normalizer). </p> |
| <p> Finally, you can extend the low level <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.html">Similarity</a> directly to implement a new retrieval model, or to use external scoring factors particular to your application. For example, a custom Similarity can access per-document values via <a class="xref" href="../Lucene.Net/Lucene.Net.Search.FieldCache.html">FieldCache</a> or <a class="xref" href="../Lucene.Net/Lucene.Net.Index.NumericDocValues.html">NumericDocValues</a> and integrate them into the score. </p> |
| <p> See the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.html">Lucene.Net.Search.Similarities</a> package documentation for information on the built-in available scoring models and extending or changing Similarity. </p> |
| <h2 id="custom-queries--expert-level">Custom Queries — Expert Level</h2> |
| <p>Custom queries are an expert level task, so tread carefully and be prepared to share your code if you want help. </p> |
| <p>With the warning out of the way, it is possible to change a lot more than just the Similarity when it comes to matching and scoring in Lucene. Lucene's search is a complex mechanism that is grounded by <span>three main classes</span>: 1. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> — The abstract object representation of the user's information need. 2. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a> — The internal interface representation of the user's Query, so that Query objects may be reused. This is global (across all segments of the index) and generally will require global statistics (such as docFreq for a given term across all segments). 3. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> — An abstract class containing common functionality for scoring. Provides both scoring and explanation capabilities. This is created per-segment. 4. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html">BulkScorer</a> — An abstract class that scores a range of documents. A default implementation simply iterates through the hits from <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a>, but some queries such as <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanQuery.html">BooleanQuery</a> have more efficient implementations. Details on each of these classes, and their children, can be found in the subsections below. </p> |
| <h4 id="the-query-class">The Query Class</h4> |
| <p>In some sense, the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> class is where it all begins. Without a Query, there would be nothing to score. Furthermore, the Query class is the catalyst for the other scoring classes as it is often responsible for creating them or coordinating the functionality between them. The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> class has several methods that are important for derived classes: 1. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html#methods">Searcher)</a> — A <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a> is the internal representation of the Query, so each Query implementation must provide an implementation of Weight. See the subsection on <a href="#weightclass">The Weight Interface</a> below for details on implementing the Weight interface. 2. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html#methods">Reader)</a> — Rewrites queries into primitive queries. Primitive queries are: <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TermQuery.html">TermQuery</a>, <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanQuery.html">BooleanQuery</a>, <span>and other queries that implement <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html#methods">Searcher)</a></span> </p> |
| <h4 id="the-weight-interface">The Weight Interface</h4> |
| <p>The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a> interface provides an internal representation of the Query so that it can be reused. Any <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a> dependent state should be stored in the Weight implementation, not in the Query class. The interface defines five methods that must be implemented: 1. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html#methods">GetQuery</a> — Pointer to the Query that this Weight represents. 2. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html#methods">GetValueForNormalization</a> — A weight can return a floating point value to indicate its magnitude for query normalization. Typically a weight such as TermWeight that scores via a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.html">Similarity</a> will just defer to the Similarity's implementation: <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.SimWeight.html#methods">SimWeight#getValueForNormalization</a>. For example, with <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.TFIDFSimilarity.html">Lucene's classic vector-space formula</a>, this is implemented as the sum of squared weights: <code>` 3. [TopLevelBoost)](xref:Lucene.Net.Search.Weight#methods) — Performs query normalization: *</code>topLevelBoost<code>: A query-boost factor from any wrapping queries that should be multiplied into every document's score. For example, a TermQuery that is wrapped within a BooleanQuery with a boost of</code>5<code>would receive this value at this time. This allows the TermQuery (the leaf node in this case) to compute this up-front a single time (e.g. by multiplying into the IDF), rather than for every document. *</code>norm`: Passes in a a normalization factor which may allow for comparing scores between queries. Typically a weight such as TermWeight that scores via a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.html">Similarity</a> will just defer to the Similarity's implementation: <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.SimWeight.html#methods">SimWeight#normalize</a>. 4. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html#methods">AcceptDocs)</a> — Construct a new <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> for this Weight. See <a href="#scorerclass">The Scorer Class</a> below for help defining a Scorer. As the name implies, the Scorer is responsible for doing the actual scoring of documents given the Query. 5. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html#methods">AcceptDocs)</a> — Construct a new <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html">BulkScorer</a> for this Weight. See <a href="#bulkscorerclass">The BulkScorer Class</a> below for help defining a BulkScorer. This is an optional method, and most queries do not implement it. 6. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html#methods">Doc)</a> — Provide a means for explaining why a given document was scored the way it was. Typically a weight such as TermWeight that scores via a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.html">Similarity</a> will make use of the Similarity's implementation: <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.SimScorer.html#methods">Freq)</a>. </p> |
| <h4 id="the-scorer-class">The Scorer Class</h4> |
| <p>The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> abstract class provides common scoring functionality for all Scorer implementations and is the heart of the Lucene scoring process. The Scorer defines the following abstract (some of them are not yet abstract, but will be in future versions and should be considered as such now) methods which must be implemented (some of them inherited from <a class="xref" href="../Lucene.Net/Lucene.Net.Search.DocIdSetIterator.html">DocIdSetIterator</a>): 1. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html#methods">NextDoc</a> — Advances to the next document that matches this Query, returning true if and only if there is another document that matches. 2. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html#methods">DocID</a> — Returns the id of the <a class="xref" href="../Lucene.Net/Lucene.Net.Documents.Document.html">Document</a> that contains the match. 3. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html#methods">Score</a> — Return the score of the current document. This value can be determined in any appropriate way for an application. For instance, the <a href="xref:Lucene.Net.Search.TermScorer">TermScorer</a> simply defers to the configured Similarity: <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Similarities.Similarity.SimScorer.html#methods">Freq)</a>. 4. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html#methods">Freq</a> — Returns the number of matches for the current document. This value can be determined in any appropriate way for an application. For instance, the <a href="xref:Lucene.Net.Search.TermScorer">TermScorer</a> simply defers to the term frequency from the inverted index: <a class="xref" href="../Lucene.Net/Lucene.Net.Index.DocsEnum.html#methods">DocsEnum.freq</a>. 5. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html#methods">Advance</a> — Skip ahead in the document matches to the document whose id is greater than or equal to the passed in value. In many instances, advance can be implemented more efficiently than simply looping through all the matching documents until the target document is identified. 6. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html#methods">GetChildren</a> — Returns any child subscorers underneath this scorer. This allows for users to navigate the scorer hierarchy and receive more fine-grained details on the scoring process. </p> |
| <h4 id="the-bulkscorer-class">The BulkScorer Class</h4> |
| <p>The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html">BulkScorer</a> scores a range of documents. There is only one abstract method: 1. <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html#methods">Score</a> — Score all documents up to but not including the specified max document. </p> |
| <h4 id="why-would-i-want-to-add-my-own-query">Why would I want to add my own Query?</h4> |
| <p>In a nutshell, you want to add your own custom Query implementation when you think that Lucene's aren't appropriate for the task that you want to do. You might be doing some cutting edge research or you need more information back out of Lucene (similar to Doug adding SpanQuery functionality).</p> |
| <h2 id="appendix-search-algorithm">Appendix: Search Algorithm</h2> |
| <p>This section is mostly notes on stepping through the Scoring process and serves as fertilizer for the earlier sections.</p> |
| <p>In the typical search application, a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> is passed to the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a>, beginning the scoring process.</p> |
| <p>Once inside the IndexSearcher, a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Collector.html">Collector</a> is used for the scoring and sorting of the search results. These important objects are involved in a search: 1. The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a> object of the Query. The Weight object is an internal representation of the Query that allows the Query to be reused by the IndexSearcher. 2. The IndexSearcher that initiated the call. 3. A <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Filter.html">Filter</a> for limiting the result set. Note, the Filter may be null. 4. A <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Sort.html">Sort</a> object for specifying how to sort the results if the standard score-based sort method is not desired. </p> |
| <p>Assuming we are not sorting (since sorting doesn't affect the raw Lucene score), we call one of the search methods of the IndexSearcher, passing in the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a> object created by <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html#methods">IndexSearcher.createNormalizedWeight</a>, <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Filter.html">Filter</a> and the number of results we want. This method returns a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TopDocs.html">TopDocs</a> object, which is an internal collection of search results. The IndexSearcher creates a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.TopScoreDocCollector.html">TopScoreDocCollector</a> and passes it along with the Weight, Filter to another expert search method (for more on the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Collector.html">Collector</a> mechanism, see <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a>). The TopScoreDocCollector uses a <a href="xref:Lucene.Net.Util.PriorityQueue">PriorityQueue</a> to collect the top results for the search. </p> |
| <p>If a Filter is being used, some initial setup is done to determine which docs to include. Otherwise, we ask the Weight for a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> for each <a class="xref" href="../Lucene.Net/Lucene.Net.Index.IndexReader.html">IndexReader</a> segment and proceed by calling <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html#methods">BulkScorer.score</a>. </p> |
| <p>At last, we are actually going to score some documents. The score method takes in the Collector (most likely the TopScoreDocCollector or TopFieldCollector) and does its business.Of course, here is where things get involved. The <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> that is returned by the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a> object depends on what type of Query was submitted. In most real world applications with multiple query terms, the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> is going to be a <code>BooleanScorer2</code> created from <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BooleanQuery.BooleanWeight.html">BooleanWeight</a> (see the section on <a href="#customqueriesexpert">custom queries</a> for info on changing this). </p> |
| <p>Assuming a BooleanScorer2, we first initialize the Coordinator, which is used to apply the coord() factor. We then get a internal Scorer based on the required, optional and prohibited parts of the query. Using this internal Scorer, the BooleanScorer2 then proceeds into a while loop based on the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html#methods">Scorer.nextDoc</a> method. The nextDoc() method advances to the next document matching the query. This is an abstract method in the Scorer class and is thus overridden by all derived implementations. If you have a simple OR query your internal Scorer is most likely a DisjunctionSumScorer, which essentially combines the scorers from the sub scorers of the OR'd terms.</p> |
| </div> |
| <div class="markdown level0 conceptual"></div> |
| <div class="markdown level0 remarks"></div> |
| <h3 id="classes">Classes |
| </h3> |
| <h4><a class="xref" href="Lucene.Net.Search.AssertingBulkOutOfOrderScorer.html">AssertingBulkOutOfOrderScorer</a></h4> |
| <section><p>A crazy <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html">BulkScorer</a> that wraps another <a class="xref" href="../Lucene.Net/Lucene.Net.Search.BulkScorer.html">BulkScorer</a> |
| but shuffles the order of the collected documents.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.AssertingBulkScorer.html">AssertingBulkScorer</a></h4> |
| <section><p>Wraps a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> with additional checks. </p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.AssertingCollector.html">AssertingCollector</a></h4> |
| <section><p>Wraps another <a class="xref" href="../Lucene.Net/Lucene.Net.Search.ICollector.html">ICollector</a> and checks that |
| <a class="xref" href="Lucene.Net.Search.AssertingCollector.html#Lucene_Net_Search_AssertingCollector_AcceptsDocsOutOfOrder">AcceptsDocsOutOfOrder</a> is respected.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.AssertingIndexSearcher.html">AssertingIndexSearcher</a></h4> |
| <section><p>Helper class that adds some extra checks to ensure correct |
| usage of <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a> and <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Weight.html">Weight</a>.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.AssertingQuery.html">AssertingQuery</a></h4> |
| <section><p>Assertion-enabled query. </p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.AssertingScorer.html">AssertingScorer</a></h4> |
| <section><p>Wraps a <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Scorer.html">Scorer</a> with additional checks.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.CheckHits.html">CheckHits</a></h4> |
| <section><p>Utility class for asserting expected hits in tests.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.ExplanationAsserter.html">ExplanationAsserter</a></h4> |
| <section><p>Asserts that the score explanation for every document matching a |
| query corresponds with the true score. |
| <p> |
| NOTE: this HitCollector should only be used with the <a class="xref" href="../Lucene.Net/Lucene.Net.Search.Query.html">Query</a> and <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a> |
| specified at when it is constructed.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.ExplanationAssertingSearcher.html">ExplanationAssertingSearcher</a></h4> |
| <section><p>An <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a> that implicitly checks hte explanation of every match |
| whenever it executes a search.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.FCInvisibleMultiReader.html">FCInvisibleMultiReader</a></h4> |
| <section><p>This is a <a class="xref" href="../Lucene.Net/Lucene.Net.Index.MultiReader.html">MultiReader</a> that can be used for randomly wrapping other readers |
| without creating <a class="xref" href="../Lucene.Net/Lucene.Net.Search.FieldCache.html">FieldCache</a> insanity. |
| The trick is to use an opaque/fake cache key.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.QueryUtils.html">QueryUtils</a></h4> |
| <section><p>Utility class for sanity-checking queries.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.RandomSimilarityProvider.html">RandomSimilarityProvider</a></h4> |
| <section><p>Similarity implementation that randomizes Similarity implementations |
| per-field. |
| <p> |
| The choices are 'sticky', so the selected algorithm is always used |
| for the same field.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.SearchEquivalenceTestBase.html">SearchEquivalenceTestBase</a></h4> |
| <section><p>Simple base class for checking search equivalence. |
| Extend it, and write tests that create <a class="xref" href="Lucene.Net.Search.SearchEquivalenceTestBase.html#Lucene_Net_Search_SearchEquivalenceTestBase_RandomTerm">RandomTerm()</a>s |
| (all terms are single characters a-z), and use |
| <a class="xref" href="Lucene.Net.Search.SearchEquivalenceTestBase.html#Lucene_Net_Search_SearchEquivalenceTestBase_AssertSameSet_Lucene_Net_Search_Query_Lucene_Net_Search_Query_">AssertSameSet(Query, Query)</a> and |
| <a class="xref" href="Lucene.Net.Search.SearchEquivalenceTestBase.html#Lucene_Net_Search_SearchEquivalenceTestBase_AssertSubsetOf_Lucene_Net_Search_Query_Lucene_Net_Search_Query_">AssertSubsetOf(Query, Query)</a>.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.SearcherExpiredException.html">SearcherExpiredException</a></h4> |
| <section><p>Thrown when the lease for a searcher has expired.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.SetCollector.html">SetCollector</a></h4> |
| <section><p>Just collects document ids into a set.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.ShardSearchingTestBase.html">ShardSearchingTestBase</a></h4> |
| <section><p>Base test class for simulating distributed search across multiple shards.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.ShardSearchingTestBase.NodeState.html">ShardSearchingTestBase.NodeState</a></h4> |
| <section></section> |
| <h4><a class="xref" href="Lucene.Net.Search.ShardSearchingTestBase.NodeState.ShardIndexSearcher.html">ShardSearchingTestBase.NodeState.ShardIndexSearcher</a></h4> |
| <section><p>Matches docs in the local shard but scores based on |
| aggregated stats ("mock distributed scoring") from all |
| nodes.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Search.ShardSearchingTestBase.SearcherAndVersion.html">ShardSearchingTestBase.SearcherAndVersion</a></h4> |
| <section><p>An <a class="xref" href="../Lucene.Net/Lucene.Net.Search.IndexSearcher.html">IndexSearcher</a> and associated version (lease)</p> |
| </section> |
| </article> |
| </div> |
| |
| <div class="hidden-sm col-md-2" role="complementary"> |
| <div class="sideaffix"> |
| <div class="contribution"> |
| <ul class="nav"> |
| <li> |
| <a href="https://github.com/apache/lucenenet/blob/docs/4.8.0-beta00008/src/Lucene.Net/Search/package.md/#L2" class="contribution-link">Improve this Doc</a> |
| </li> |
| </ul> |
| </div> |
| <nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix"> |
| <!-- <p><a class="back-to-top" href="#top">Back to top</a><p> --> |
| </nav> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| <footer> |
| <div class="grad-bottom"></div> |
| <div class="footer"> |
| <div class="container"> |
| <span class="pull-right"> |
| <a href="#top">Back to top</a> |
| </span> |
| Copyright © 2020 Licensed to the Apache Software Foundation (ASF) |
| |
| </div> |
| </div> |
| </footer> |
| </div> |
| |
| <script type="text/javascript" src="../../styles/docfx.vendor.js"></script> |
| <script type="text/javascript" src="../../styles/docfx.js"></script> |
| <script type="text/javascript" src="../../styles/main.js"></script> |
| </body> |
| </html> |
