| <!DOCTYPE html> |
| <!--[if IE]><![endif]--> |
| <html> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> |
| <title>Namespace Lucene.Net.Analysis.Sinks |
| | Apache Lucene.NET 4.8.0-beta00010 Documentation </title> |
| <meta name="viewport" content="width=device-width"> |
| <meta name="title" content="Namespace Lucene.Net.Analysis.Sinks |
| | Apache Lucene.NET 4.8.0-beta00010 Documentation "> |
| <meta name="generator" content="docfx 2.56.0.0"> |
| |
| <link rel="shortcut icon" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/favicon.ico"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.css"> |
| <meta property="docfx:navrel" content="toc.html"> |
| <meta property="docfx:tocrel" content="analysis-common/toc.html"> |
| |
| <meta property="docfx:rel" content="https://lucenenet.apache.org/docs/4.8.0-beta00009/"> |
| |
| </head> |
| <body data-spy="scroll" data-target="#affix" data-offset="120"> |
| <div id="wrapper"> |
| <header> |
| |
| <nav id="autocollapse" class="navbar ng-scope" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| |
| <a class="navbar-brand" href="/"> |
| <img id="logo" class="svg" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/lucene-net-color.png" alt=""> |
| </a> |
| </div> |
| <div class="collapse navbar-collapse" id="navbar"> |
| <form class="navbar-form navbar-right" role="search" id="search"> |
| <div class="form-group"> |
| <input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off"> |
| </div> |
| </form> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="subnav navbar navbar-default"> |
| <div class="container hide-when-search"> |
| <ul class="level0 breadcrumb"> |
| <li> |
| <a href="https://lucenenet.apache.org/docs/4.8.0-beta00009/">API</a> |
| <span id="breadcrumb"> |
| <ul class="breadcrumb"> |
| <li></li> |
| </ul> |
| </span> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </header> |
| <div class="container body-content"> |
| |
| <div id="search-results"> |
| <div class="search-list"></div> |
| <div class="sr-items"> |
| <p><i class="glyphicon glyphicon-refresh index-loading"></i></p> |
| </div> |
| <ul id="pagination"></ul> |
| </div> |
| </div> |
| <div role="main" class="container body-content hide-when-search"> |
| |
| <div class="sidenav hide-when-search"> |
| <a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a> |
| <div class="sidetoggle collapse" id="sidetoggle"> |
| <div id="sidetoc"></div> |
| </div> |
| </div> |
| <div class="article row grid-right"> |
| <div class="col-md-10"> |
| <article class="content wrap" id="_content" data-uid="Lucene.Net.Analysis.Sinks"> |
| |
| <h1 id="Lucene_Net_Analysis_Sinks" data-uid="Lucene.Net.Analysis.Sinks" class="text-break">Namespace Lucene.Net.Analysis.Sinks |
| </h1> |
| <div class="markdown level0 summary"><!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <p><a class="xref" href="Lucene.Net.Analysis.Sinks.TeeSinkTokenFilter.html">TeeSinkTokenFilter</a> and implementations |
| of <a class="xref" href="Lucene.Net.Analysis.Sinks.TeeSinkTokenFilter.SinkFilter.html">TeeSinkTokenFilter.SinkFilter</a> that |
| might be useful.</p> |
| </div> |
| <div class="markdown level0 conceptual"></div> |
| <div class="markdown level0 remarks"></div> |
| <h3 id="classes">Classes |
| </h3> |
| <h4><a class="xref" href="Lucene.Net.Analysis.Sinks.DateRecognizerSinkFilter.html">DateRecognizerSinkFilter</a></h4> |
| <section><p>Attempts to parse the <a class="xref" href="http://localhost:8080/api/core/Lucene.Net.Analysis.TokenAttributes.CharTermAttribute.html#Lucene_Net_Analysis_TokenAttributes_CharTermAttribute_ToString">ToString()</a> as a Date using either the |
| <span class="xref">System.DateTime.TryParse(System.String,System.IFormatProvider,System.Globalization.DateTimeStyles,System.DateTime@)</span> or |
| <span class="xref">System.DateTime.TryParseExact(System.String,System.String[],System.IFormatProvider,System.Globalization.DateTimeStyles,System.DateTime@)</span> methods. |
| If a format is passed, <span class="xref">System.DateTime.TryParseExact(System.String,System.String[],System.IFormatProvider,System.Globalization.DateTimeStyles,System.DateTime@)</span> |
| will be used, and the format must strictly match one of the specified formats as specified in the MSDN documentation. |
| If the value is a Date, it will add it to the sink.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Analysis.Sinks.TeeSinkTokenFilter.html">TeeSinkTokenFilter</a></h4> |
| <section><p>This TokenFilter provides the ability to set aside attribute states |
| that have already been analyzed. This is useful in situations where multiple fields share |
| many common analysis steps and then go their separate ways. |
| <p> |
| It is also useful for doing things like entity extraction or proper noun analysis as |
| part of the analysis workflow and saving off those tokens for use in another field. |
| <p> |
| <pre><code>TeeSinkTokenFilter source1 = new TeeSinkTokenFilter(new WhitespaceTokenizer(version, reader1)); |
| TeeSinkTokenFilter.SinkTokenStream sink1 = source1.NewSinkTokenStream(); |
| TeeSinkTokenFilter.SinkTokenStream sink2 = source1.NewSinkTokenStream(); |
| |
| TeeSinkTokenFilter source2 = new TeeSinkTokenFilter(new WhitespaceTokenizer(version, reader2)); |
| source2.AddSinkTokenStream(sink1); |
| source2.AddSinkTokenStream(sink2); |
| |
| TokenStream final1 = new LowerCaseFilter(version, source1); |
| TokenStream final2 = source2; |
| TokenStream final3 = new EntityDetect(sink1); |
| TokenStream final4 = new URLDetect(sink2); |
| |
| d.Add(new TextField("f1", final1, Field.Store.NO)); |
| d.Add(new TextField("f2", final2, Field.Store.NO)); |
| d.Add(new TextField("f3", final3, Field.Store.NO)); |
| d.Add(new TextField("f4", final4, Field.Store.NO));</code></pre> |
| <p>In this example, <code>sink1</code> and <code>sink2</code> will both get tokens from both |
| <code>reader1</code> and <code>reader2</code> after whitespace tokenizer |
| and now we can further wrap any of these in extra analysis, and more "sources" can be inserted if desired. |
| It is important, that tees are consumed before sinks (in the above example, the field names must be |
| less the sink's field names). If you are not sure, which stream is consumed first, you can simply |
| add another sink and then pass all tokens to the sinks at once using <a class="xref" href="Lucene.Net.Analysis.Sinks.TeeSinkTokenFilter.html#Lucene_Net_Analysis_Sinks_TeeSinkTokenFilter_ConsumeAllTokens">ConsumeAllTokens()</a>. |
| This <span class="xref">Lucene.Net.Analysis.TokenFilter</span> is exhausted after this. In the above example, change |
| the example above to:</p> |
| <pre><code>... |
| TokenStream final1 = new LowerCaseFilter(version, source1.NewSinkTokenStream()); |
| TokenStream final2 = source2.NewSinkTokenStream(); |
| sink1.ConsumeAllTokens(); |
| sink2.ConsumeAllTokens(); |
| ...</code></pre> |
| <p>In this case, the fields can be added in any order, because the sources are not used anymore and all sinks are ready. |
| <p>Note, the EntityDetect and URLDetect TokenStreams are for the example and do not currently exist in Lucene. |
| </p></p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Analysis.Sinks.TeeSinkTokenFilter.SinkFilter.html">TeeSinkTokenFilter.SinkFilter</a></h4> |
| <section><p>A filter that decides which <span class="xref">Lucene.Net.Util.AttributeSource</span> states to store in the sink.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Analysis.Sinks.TeeSinkTokenFilter.SinkTokenStream.html">TeeSinkTokenFilter.SinkTokenStream</a></h4> |
| <section><p><span class="xref">Lucene.Net.Analysis.TokenStream</span> output from a tee with optional filtering.</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Analysis.Sinks.TokenRangeSinkFilter.html">TokenRangeSinkFilter</a></h4> |
| <section><p>Counts the tokens as they go by and saves to the internal list those between the range of lower and upper, exclusive of upper</p> |
| </section> |
| <h4><a class="xref" href="Lucene.Net.Analysis.Sinks.TokenTypeSinkFilter.html">TokenTypeSinkFilter</a></h4> |
| <section><p>Adds a token to the sink if it has a specific type.</p> |
| </section> |
| </article> |
| </div> |
| |
| <div class="hidden-sm col-md-2" role="complementary"> |
| <div class="sideaffix"> |
| <div class="contribution"> |
| <ul class="nav"> |
| <li> |
| <a href="https://github.com/apache/lucenenet/blob/docs/4.8.0-beta00010/src/Lucene.Net.Analysis.Common/Analysis/Sinks/package.md/#L2" class="contribution-link">Improve this Doc</a> |
| </li> |
| </ul> |
| </div> |
| <nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix"> |
| <!-- <p><a class="back-to-top" href="#top">Back to top</a><p> --> |
| </nav> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| <footer> |
| <div class="grad-bottom"></div> |
| <div class="footer"> |
| <div class="container"> |
| <span class="pull-right"> |
| <a href="#top">Back to top</a> |
| </span> |
| Copyright © 2020 Licensed to the Apache Software Foundation (ASF) |
| |
| </div> |
| </div> |
| </footer> |
| </div> |
| |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.js"></script> |
| </body> |
| </html> |
