| <!DOCTYPE html> |
| <!--[if IE]><![endif]--> |
| <html> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> |
| <title>Namespace Lucene.Net.Facet.Taxonomy.WriterCache |
| | Apache Lucene.NET 4.8.0-beta00013 Documentation </title> |
| <meta name="viewport" content="width=device-width"> |
| <meta name="title" content="Namespace Lucene.Net.Facet.Taxonomy.WriterCache |
| | Apache Lucene.NET 4.8.0-beta00013 Documentation "> |
| <meta name="generator" content="docfx 2.56.2.0"> |
| |
| <link rel="shortcut icon" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/favicon.ico"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.css"> |
| <meta property="docfx:navrel" content="toc.html"> |
| <meta property="docfx:tocrel" content="facet/toc.html"> |
| |
| <meta property="docfx:rel" content="https://lucenenet.apache.org/docs/4.8.0-beta00009/"> |
| |
| </head> |
| <body data-spy="scroll" data-target="#affix" data-offset="120"> |
| <span id="forkongithub"><a href="https://github.com/apache/lucenenet" target="_blank">Fork me on GitHub</a></span> |
| <div id="wrapper"> |
| <header> |
| |
| <nav id="autocollapse" class="navbar ng-scope" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| |
| <a class="navbar-brand" href="/"> |
| <img id="logo" class="svg" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/lucene-net-color.png" alt=""> |
| </a> |
| </div> |
| <div class="collapse navbar-collapse" id="navbar"> |
| <form class="navbar-form navbar-right" role="search" id="search"> |
| <div class="form-group"> |
| <input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off"> |
| </div> |
| </form> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="subnav navbar navbar-default"> |
| <div class="container hide-when-search"> |
| <ul class="level0 breadcrumb"> |
| <li> |
| <a href="https://lucenenet.apache.org/docs/4.8.0-beta00009/">API</a> |
| <span id="breadcrumb"> |
| <ul class="breadcrumb"> |
| <li></li> |
| </ul> |
| </span> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </header> |
| <div class="container body-content"> |
| |
| <div id="search-results"> |
| <div class="search-list"></div> |
| <div class="sr-items"> |
| <p><i class="glyphicon glyphicon-refresh index-loading"></i></p> |
| </div> |
| <ul id="pagination"></ul> |
| </div> |
| </div> |
| <div role="main" class="container body-content hide-when-search"> |
| |
| <div class="sidenav hide-when-search"> |
| <a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a> |
| <div class="sidetoggle collapse" id="sidetoggle"> |
| <div id="sidetoc"></div> |
| </div> |
| </div> |
| <div class="article row grid-right"> |
| <div class="col-md-10"> |
| <article class="content wrap" id="_content" data-uid="Lucene.Net.Facet.Taxonomy.WriterCache"> |
| |
| <h1 id="Lucene_Net_Facet_Taxonomy_WriterCache" data-uid="Lucene.Net.Facet.Taxonomy.WriterCache" class="text-break">Namespace Lucene.Net.Facet.Taxonomy.WriterCache |
| </h1> |
| <div class="markdown level0 summary"><!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <p>Improves indexing time by caching a map of CategoryPath to their Ordinal. </p> |
| </div> |
| <div class="markdown level0 conceptual"></div> |
| <div class="markdown level0 remarks"></div> |
| <h3 id="classes">Classes |
| </h3> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.Cl2oTaxonomyWriterCache.html">Cl2oTaxonomyWriterCache</a></h4> |
| <section><p><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html">ITaxonomyWriterCache</a> using <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.html">CompactLabelToOrdinal</a>. Although |
| called cache, it maintains in memory all the mappings from category to |
| ordinal, relying on that <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.html">CompactLabelToOrdinal</a> is an efficient |
| mapping for this purpose.</p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></section> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CollisionMap.html">CollisionMap</a></h4> |
| <section><p>HashMap to store colliding labels. See <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.html">CompactLabelToOrdinal</a> for |
| details.</p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></section> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.html">CompactLabelToOrdinal</a></h4> |
| <section><p>This is a very efficient <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.LabelToOrdinal.html">LabelToOrdinal</a> implementation that uses a |
| <span class="xref">Lucene.Net.Facet.Taxonomy.WriterCache.CharBlockArray</span> to store all labels and a configurable number of <span class="xref">Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.HashArray</span>s to |
| reference the labels. |
| <p> |
| Since the <span class="xref">Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.HashArray</span>s don't handle collisions, a <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CollisionMap.html">CollisionMap</a> is used |
| to store the colliding labels. |
| </p> |
| <p> |
| This data structure grows by adding a new HashArray whenever the number of |
| collisions in the <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CollisionMap.html">CollisionMap</a> exceeds <span class="xref">Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.loadFactor</span> |
| <code>GetMaxOrdinal().</code> Growing also includes reinserting all colliding |
| labels into the <span class="xref">Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.HashArray</span>s to possibly reduce the number of collisions.</p> |
| <p>For setting the <span class="xref">Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.loadFactor</span> see |
| <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.CompactLabelToOrdinal.html#Lucene_Net_Facet_Taxonomy_WriterCache_CompactLabelToOrdinal__ctor_System_Int32_System_Single_System_Int32_">CompactLabelToOrdinal(Int32, Single, Int32)</a>. |
| </p> |
| <p> |
| This data structure has a much lower memory footprint (~30%) compared to a |
| Java HashMap<String, Integer>. It also only uses a small fraction of objects |
| a HashMap would use, thus limiting the GC overhead. Ingestion speed was also |
| ~50% faster compared to a HashMap for 3M unique labels.</p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div><p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.LabelToOrdinal.html">LabelToOrdinal</a></h4> |
| <section><p>Abstract class for storing Label->Ordinal mappings in a taxonomy. |
| <p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></section> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.LruTaxonomyWriterCache.html">LruTaxonomyWriterCache</a></h4> |
| <section><p>LRU <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html">ITaxonomyWriterCache</a> - good choice for huge taxonomies. |
| <p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></section> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.NameHashInt32CacheLru.html">NameHashInt32CacheLru</a></h4> |
| <section><p>An an LRU cache of mapping from name to int. |
| Used to cache Ordinals of category paths. |
| It uses as key, hash of the path instead of the path. |
| This way the cache takes less RAM, but correctness depends on |
| assuming no collisions. |
| <p> |
| NOTE: this was NameHashIntCacheLRU in Lucene |
| <p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></section> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.NameInt32CacheLru.html">NameInt32CacheLru</a></h4> |
| <section><p>An an LRU cache of mapping from name to int. |
| Used to cache Ordinals of category paths. |
| <p> |
| NOTE: This was NameIntCacheLRU in Lucene |
| <p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></section> |
| <h3 id="interfaces">Interfaces |
| </h3> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.INameInt32CacheLru.html">INameInt32CacheLru</a></h4> |
| <section><p>Public members of the <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.NameInt32CacheLru.html">NameInt32CacheLru</a> that are shared between instances.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html">ITaxonomyWriterCache</a></h4> |
| <section><p><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html">ITaxonomyWriterCache</a> is a relatively simple interface for a cache of |
| category->ordinal mappings, used in ITaxonomyWriter implementations (such as |
| <a class="xref" href="Lucene.Net.Facet.Taxonomy.Directory.DirectoryTaxonomyWriter.html">DirectoryTaxonomyWriter</a>). |
| <p> |
| It basically has <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html#Lucene_Net_Facet_Taxonomy_WriterCache_ITaxonomyWriterCache_Put_Lucene_Net_Facet_Taxonomy_FacetLabel_System_Int32_">Put(FacetLabel, Int32)</a> methods for adding a mapping, and <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html#Lucene_Net_Facet_Taxonomy_WriterCache_ITaxonomyWriterCache_Get_Lucene_Net_Facet_Taxonomy_FacetLabel_">Get(FacetLabel)</a> for looking a |
| mapping up the cache. The cache does <strong>not</strong> guarantee to hold everything |
| that has been put into it, and might in fact selectively delete some of the |
| mappings (e.g., the ones least recently used). This means that if <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html#Lucene_Net_Facet_Taxonomy_WriterCache_ITaxonomyWriterCache_Get_Lucene_Net_Facet_Taxonomy_FacetLabel_">Get(FacetLabel)</a> |
| returns a negative response, it does not necessarily mean that the category |
| doesn't exist - just that it is not in the cache. The caller can only infer |
| that the category doesn't exist if it knows the cache to be complete (because |
| all the categories were loaded into the cache, and since then no <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html#Lucene_Net_Facet_Taxonomy_WriterCache_ITaxonomyWriterCache_Put_Lucene_Net_Facet_Taxonomy_FacetLabel_System_Int32_">Put(FacetLabel, Int32)</a> |
| returned true). |
| </p> |
| <p> |
| However, if it does so, it should clear out large parts of the cache at once, |
| because the user will typically need to work hard to recover from every cache |
| cleanup (see <a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.ITaxonomyWriterCache.html#Lucene_Net_Facet_Taxonomy_WriterCache_ITaxonomyWriterCache_Put_Lucene_Net_Facet_Taxonomy_FacetLabel_System_Int32_">Put(FacetLabel, Int32)</a>'s return value). |
| </p> |
| <p> |
| <strong>NOTE:</strong> the cache may be accessed concurrently by multiple threads, |
| therefore cache implementations should take this into consideration.</p> |
| <div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div><p> |
| </section> |
| <h3 id="enums">Enums |
| </h3> |
| <h4><a class="xref" href="Lucene.Net.Facet.Taxonomy.WriterCache.LruTaxonomyWriterCache.LRUType.html">LruTaxonomyWriterCache.LRUType</a></h4> |
| <section><p>Determines cache type. |
| For guaranteed correctness - not relying on no-collisions in the hash |
| function, LRU_STRING should be used.</p> |
| </section> |
| </article> |
| </div> |
| |
| <div class="hidden-sm col-md-2" role="complementary"> |
| <div class="sideaffix"> |
| <div class="contribution"> |
| <ul class="nav"> |
| <li> |
| <a href="https://github.com/apache/lucenenet/blob/docs/4.8.0-beta00013/src/Lucene.Net.Facet/Taxonomy/WriterCache/package.md/#L2" class="contribution-link">Improve this Doc</a> |
| </li> |
| </ul> |
| </div> |
| <nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix"> |
| <!-- <p><a class="back-to-top" href="#top">Back to top</a><p> --> |
| </nav> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| <footer> |
| <div class="grad-bottom"></div> |
| <div class="footer"> |
| <div class="container"> |
| <span class="pull-right"> |
| <a href="#top">Back to top</a> |
| </span> |
| Copyright © 2020 The Apache Software Foundation, Licensed under the <a href='http://www.apache.org/licenses/LICENSE-2.0' target='_blank'>Apache License, Version 2.0</a><br> <small>Apache Lucene.Net, Lucene.Net, Apache, the Apache feather logo, and the Apache Lucene.Net project logo are trademarks of The Apache Software Foundation. <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</small> |
| |
| </div> |
| </div> |
| </footer> |
| </div> |
| |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.js"></script> |
| </body> |
| </html> |
