| <!DOCTYPE html> |
| <!--[if IE]><![endif]--> |
| <html> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> |
| <title>Namespace Lucene.Net.Benchmarks.Quality |
| | Apache Lucene.NET 4.8.0-beta00013 Documentation </title> |
| <meta name="viewport" content="width=device-width"> |
| <meta name="title" content="Namespace Lucene.Net.Benchmarks.Quality |
| | Apache Lucene.NET 4.8.0-beta00013 Documentation "> |
| <meta name="generator" content="docfx 2.56.2.0"> |
| |
| <link rel="shortcut icon" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/favicon.ico"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.css"> |
| <meta property="docfx:navrel" content="toc.html"> |
| <meta property="docfx:tocrel" content="benchmark/toc.html"> |
| |
| <meta property="docfx:rel" content="https://lucenenet.apache.org/docs/4.8.0-beta00009/"> |
| |
| </head> |
| <body data-spy="scroll" data-target="#affix" data-offset="120"> |
| <span id="forkongithub"><a href="https://github.com/apache/lucenenet" target="_blank">Fork me on GitHub</a></span> |
| <div id="wrapper"> |
| <header> |
| |
| <nav id="autocollapse" class="navbar ng-scope" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| |
| <a class="navbar-brand" href="/"> |
| <img id="logo" class="svg" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/lucene-net-color.png" alt=""> |
| </a> |
| </div> |
| <div class="collapse navbar-collapse" id="navbar"> |
| <form class="navbar-form navbar-right" role="search" id="search"> |
| <div class="form-group"> |
| <input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off"> |
| </div> |
| </form> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="subnav navbar navbar-default"> |
| <div class="container hide-when-search"> |
| <ul class="level0 breadcrumb"> |
| <li> |
| <a href="https://lucenenet.apache.org/docs/4.8.0-beta00009/">API</a> |
| <span id="breadcrumb"> |
| <ul class="breadcrumb"> |
| <li></li> |
| </ul> |
| </span> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </header> |
| <div class="container body-content"> |
| |
| <div id="search-results"> |
| <div class="search-list"></div> |
| <div class="sr-items"> |
| <p><i class="glyphicon glyphicon-refresh index-loading"></i></p> |
| </div> |
| <ul id="pagination"></ul> |
| </div> |
| </div> |
| <div role="main" class="container body-content hide-when-search"> |
| |
| <div class="sidenav hide-when-search"> |
| <a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a> |
| <div class="sidetoggle collapse" id="sidetoggle"> |
| <div id="sidetoc"></div> |
| </div> |
| </div> |
| <div class="article row grid-right"> |
| <div class="col-md-10"> |
| <article class="content wrap" id="_content" data-uid="Lucene.Net.Benchmarks.Quality"> |
| |
| <h1 id="Lucene_Net_Benchmarks_Quality" data-uid="Lucene.Net.Benchmarks.Quality" class="text-break">Namespace Lucene.Net.Benchmarks.Quality |
| </h1> |
| <div class="markdown level0 summary"><!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <h2 id="search-quality-benchmarking">Search Quality Benchmarking.</h2> |
| <p>This package allows to benchmark search quality of a Lucene application.</p> |
| <p>In order to use this package you should provide:</p> |
| <ul> |
| <li><p>A <a href="{@docRoot}/../core/org/apache/lucene/search/IndexSearcher.html">IndexSearcher</a>.</p> |
| </li> |
| <li><p><a href="QualityQuery.html">Quality queries</a>.</p> |
| </li> |
| <li><p><a href="Judge.html">Judging object</a>.</p> |
| </li> |
| <li><p><a href="utils/SubmissionReport.html">Reporting object</a>.</p> |
| </li> |
| </ul> |
| <p>For benchmarking TREC collections with TREC QRels, take a look at the |
| <a href="trec/package-summary.html">trec package</a>.</p> |
| <p>Here is a sample code used to run the TREC 2006 queries 701-850 on the .Gov2 collection:</p> |
| <pre><code> File topicsFile = new File("topics-701-850.txt"); |
| File qrelsFile = new File("qrels-701-850.txt"); |
| IndexReader ir = DirectoryReader.open(directory): |
| IndexSearcher searcher = new IndexSearcher(ir); |
| |
| int maxResults = 1000; |
| String docNameField = "docname"; |
| |
| PrintWriter logger = new PrintWriter(System.out,true); |
| |
| // use trec utilities to read trec topics into quality queries |
| TrecTopicsReader qReader = new TrecTopicsReader(); |
| QualityQuery qqs[] = qReader.readQueries(new BufferedReader(new FileReader(topicsFile))); |
| |
| // prepare judge, with trec utilities that read from a QRels file |
| Judge judge = new TrecJudge(new BufferedReader(new FileReader(qrelsFile))); |
| |
| // validate topics & judgments match each other |
| judge.validateData(qqs, logger); |
| |
| // set the parsing of quality queries into Lucene queries. |
| QualityQueryParser qqParser = new SimpleQQParser("title", "body"); |
| |
| // run the benchmark |
| QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher, docNameField); |
| SubmissionReport submitLog = null; |
| QualityStats stats[] = qrun.execute(maxResults, judge, submitLog, logger); |
| |
| // print an avarage sum of the results |
| QualityStats avg = QualityStats.average(stats); |
| avg.log("SUMMARY",2,logger, " "); |
| </code></pre><p>Some immediate ways to modify this program to your needs are:</p> |
| <ul> |
| <li><p>To run on different formats of queries and judgements provide your own |
| <a href="Judge.html">Judge</a> and |
| <a href="QualityQuery.html">Quality queries</a>.</p> |
| </li> |
| <li><p>Create sophisticated Lucene queries by supplying a different |
| <a href="QualityQueryParser.html">Quality query parser</a>.</p> |
| </li> |
| </ul> |
| </div> |
| <div class="markdown level0 conceptual"></div> |
| <div class="markdown level0 remarks"></div> |
| <h3 id="classes">Classes |
| </h3> |
| <h4><a class="xref" href="Lucene.Net.Benchmarks.Quality.QualityBenchmark.html">QualityBenchmark</a></h4> |
| <section><p>Main entry point for running a quality benchmark. |
| <p> |
| There are two main configurations for running a quality benchmark: |
| <ul><li>Against existing judgements.</li><li>For submission (e.g. for a contest).</li></ul> |
| The first configuration requires a non null <a class="xref" href="Lucene.Net.Benchmarks.Quality.IJudge.html">IJudge</a>. |
| The second configuration requires a non null <a class="xref" href="Lucene.Net.Benchmarks.Quality.Utils.SubmissionReport.html">SubmissionReport</a>.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Benchmarks.Quality.QualityQuery.html">QualityQuery</a></h4> |
| <section><p>A QualityQuery has an ID and some name-value pairs. |
| <p> |
| The ID allows to map the quality query with its judgements. |
| <p> |
| The name-value pairs are used by a |
| <a class="xref" href="Lucene.Net.Benchmarks.Quality.IQualityQueryParser.html">IQualityQueryParser</a> |
| to create a Lucene <span class="xref">Lucene.Net.Search.Query</span>. |
| <p> |
| It is very likely that name-value-pairs would be mapped into fields in a Lucene query, |
| but it is up to the QualityQueryParser how to map - e.g. all values in a single field, |
| or each pair as its own field, etc., - and this of course must match the way the |
| searched index was constructed.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Benchmarks.Quality.QualityStats.html">QualityStats</a></h4> |
| <section><p>Results of quality benchmark run for a single query or for a set of queries.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Benchmarks.Quality.QualityStats.RecallPoint.html">QualityStats.RecallPoint</a></h4> |
| <section><p>A certain rank in which a relevant doc was found.</p> |
| </section> |
| <h3 id="interfaces">Interfaces |
| </h3> |
| <h4><a class="xref" href="Lucene.Net.Benchmarks.Quality.IJudge.html">IJudge</a></h4> |
| <section><p>Judge if a document is relevant for a quality query.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Benchmarks.Quality.IQualityQueryParser.html">IQualityQueryParser</a></h4> |
| <section><p>Parse a <a class="xref" href="Lucene.Net.Benchmarks.Quality.QualityQuery.html">QualityQuery</a> into a Lucene query.</p> |
| </section> |
| </article> |
| </div> |
| |
| <div class="hidden-sm col-md-2" role="complementary"> |
| <div class="sideaffix"> |
| <div class="contribution"> |
| <ul class="nav"> |
| <li> |
| <a href="https://github.com/apache/lucenenet/blob/docs/4.8.0-beta00013/src/Lucene.Net.Benchmark/Quality/package.md/#L2" class="contribution-link">Improve this Doc</a> |
| </li> |
| </ul> |
| </div> |
| <nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix"> |
| <!-- <p><a class="back-to-top" href="#top">Back to top</a><p> --> |
| </nav> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| <footer> |
| <div class="grad-bottom"></div> |
| <div class="footer"> |
| <div class="container"> |
| <span class="pull-right"> |
| <a href="#top">Back to top</a> |
| </span> |
| Copyright © 2020 The Apache Software Foundation, Licensed under the <a href='http://www.apache.org/licenses/LICENSE-2.0' target='_blank'>Apache License, Version 2.0</a><br> <small>Apache Lucene.Net, Lucene.Net, Apache, the Apache feather logo, and the Apache Lucene.Net project logo are trademarks of The Apache Software Foundation. <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</small> |
| |
| </div> |
| </div> |
| </footer> |
| </div> |
| |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.js"></script> |
| </body> |
| </html> |
