| <!DOCTYPE html> |
| <!--[if IE]><![endif]--> |
| <html> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> |
| <title>Class StandardTokenizer |
| | Apache Lucene.NET 4.8.0-beta00013 Documentation </title> |
| <meta name="viewport" content="width=device-width"> |
| <meta name="title" content="Class StandardTokenizer |
| | Apache Lucene.NET 4.8.0-beta00013 Documentation "> |
| <meta name="generator" content="docfx 2.56.2.0"> |
| |
| <link rel="shortcut icon" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/favicon.ico"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.css"> |
| <link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.css"> |
| <meta property="docfx:navrel" content="toc.html"> |
| <meta property="docfx:tocrel" content="analysis-common/toc.html"> |
| |
| <meta property="docfx:rel" content="https://lucenenet.apache.org/docs/4.8.0-beta00009/"> |
| |
| </head> |
| <body data-spy="scroll" data-target="#affix" data-offset="120"> |
| <span id="forkongithub"><a href="https://github.com/apache/lucenenet" target="_blank">Fork me on GitHub</a></span> |
| <div id="wrapper"> |
| <header> |
| |
| <nav id="autocollapse" class="navbar ng-scope" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| |
| <a class="navbar-brand" href="/"> |
| <img id="logo" class="svg" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/lucene-net-color.png" alt=""> |
| </a> |
| </div> |
| <div class="collapse navbar-collapse" id="navbar"> |
| <form class="navbar-form navbar-right" role="search" id="search"> |
| <div class="form-group"> |
| <input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off"> |
| </div> |
| </form> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="subnav navbar navbar-default"> |
| <div class="container hide-when-search"> |
| <ul class="level0 breadcrumb"> |
| <li> |
| <a href="https://lucenenet.apache.org/docs/4.8.0-beta00009/">API</a> |
| <span id="breadcrumb"> |
| <ul class="breadcrumb"> |
| <li></li> |
| </ul> |
| </span> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </header> |
| <div class="container body-content"> |
| |
| <div id="search-results"> |
| <div class="search-list"></div> |
| <div class="sr-items"> |
| <p><i class="glyphicon glyphicon-refresh index-loading"></i></p> |
| </div> |
| <ul id="pagination"></ul> |
| </div> |
| </div> |
| <div role="main" class="container body-content hide-when-search"> |
| |
| <div class="sidenav hide-when-search"> |
| <a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a> |
| <div class="sidetoggle collapse" id="sidetoggle"> |
| <div id="sidetoc"></div> |
| </div> |
| </div> |
| <div class="article row grid-right"> |
| <div class="col-md-10"> |
| <article class="content wrap" id="_content" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer"> |
| |
| |
| <h1 id="Lucene_Net_Analysis_Standard_StandardTokenizer" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer" class="text-break">Class StandardTokenizer |
| </h1> |
| <div class="markdown level0 summary"><p>A grammar-based tokenizer constructed with JFlex. |
| <p> |
| As of Lucene version 3.1, this class implements the Word Break rules from the |
| Unicode Text Segmentation algorithm, as specified in |
| <a href="http://unicode.org/reports/tr29/">Unicode Standard Annex #29</a>. |
| <p> |
| </p> |
| <p>Many applications have specific tokenizer needs. If this tokenizer does |
| not suit your application, please consider copying this source code |
| directory to your project and maintaining your own grammar-based tokenizer.</p> |
| <p> |
| <p>You must specify the required <span class="xref">Lucene.Net.Util.LuceneVersion</span> |
| compatibility when creating <a class="xref" href="Lucene.Net.Analysis.Standard.StandardTokenizer.html">StandardTokenizer</a>: |
| <ul><li> As of 3.4, Hiragana and Han characters are no longer wrongly split |
| from their combining characters. If you use a previous version number, |
| you get the exact broken behavior for backwards compatibility.</li><li> As of 3.1, StandardTokenizer implements Unicode text segmentation. |
| If you use a previous version number, you get the exact behavior of |
| <a class="xref" href="Lucene.Net.Analysis.Standard.ClassicTokenizer.html">ClassicTokenizer</a> for backwards compatibility.</li></ul> |
| </p></p> |
| </div> |
| <div class="markdown level0 conceptual"></div> |
| <div class="inheritance"> |
| <h5>Inheritance</h5> |
| <div class="level0"><span class="xref">System.Object</span></div> |
| <div class="level1"><span class="xref">Lucene.Net.Util.AttributeSource</span></div> |
| <div class="level2"><span class="xref">Lucene.Net.Analysis.TokenStream</span></div> |
| <div class="level3"><span class="xref">Lucene.Net.Analysis.Tokenizer</span></div> |
| <div class="level4"><span class="xref">StandardTokenizer</span></div> |
| </div> |
| <div classs="implements"> |
| <h5>Implements</h5> |
| <div><span class="xref">System.IDisposable</span></div> |
| </div> |
| <div class="inheritedMembers"> |
| <h5>Inherited Members</h5> |
| <div> |
| <span class="xref">Lucene.Net.Analysis.Tokenizer.m_input</span> |
| </div> |
| <div> |
| <a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_CorrectOffset_System_Int32_">Tokenizer.CorrectOffset(Int32)</a> |
| </div> |
| <div> |
| <a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_SetReader_System_IO_TextReader_">Tokenizer.SetReader(TextReader)</a> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Analysis.TokenStream.Dispose()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeFactory()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeClassesEnumerator()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeImplsEnumerator()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.AddAttributeImpl(Lucene.Net.Util.Attribute)</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.AddAttribute<T>()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.HasAttributes</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.HasAttribute<T>()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.GetAttribute<T>()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.ClearAttributes()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.CaptureState()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.RestoreState(Lucene.Net.Util.AttributeSource.State)</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.GetHashCode()</span> |
| </div> |
| <div> |
| <a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_Equals_System_Object_">AttributeSource.Equals(Object)</a> |
| </div> |
| <div> |
| <a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_ReflectAsString_System_Boolean_">AttributeSource.ReflectAsString(Boolean)</a> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.ReflectWith(Lucene.Net.Util.IAttributeReflector)</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.CloneAttributes()</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.CopyTo(Lucene.Net.Util.AttributeSource)</span> |
| </div> |
| <div> |
| <span class="xref">Lucene.Net.Util.AttributeSource.ToString()</span> |
| </div> |
| <div> |
| <span class="xref">System.Object.Equals(System.Object, System.Object)</span> |
| </div> |
| <div> |
| <span class="xref">System.Object.GetType()</span> |
| </div> |
| <div> |
| <span class="xref">System.Object.MemberwiseClone()</span> |
| </div> |
| <div> |
| <span class="xref">System.Object.ReferenceEquals(System.Object, System.Object)</span> |
| </div> |
| </div> |
| <h6><strong>Namespace</strong>: <a class="xref" href="Lucene.Net.Analysis.Standard.html">Lucene.Net.Analysis.Standard</a></h6> |
| <h6><strong>Assembly</strong>: Lucene.Net.Analysis.Common.dll</h6> |
| <h5 id="Lucene_Net_Analysis_Standard_StandardTokenizer_syntax">Syntax</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public sealed class StandardTokenizer : Tokenizer, IDisposable</code></pre> |
| </div> |
| <h3 id="constructors">Constructors |
| </h3> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer__ctor_Lucene_Net_Util_LuceneVersion_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.%23ctor(Lucene.Net.Util.LuceneVersion%2CLucene.Net.Util.AttributeSource.AttributeFactory%2CSystem.IO.TextReader)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L146">View Source</a> |
| </span> |
| <a id="Lucene_Net_Analysis_Standard_StandardTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.#ctor*"></a> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer__ctor_Lucene_Net_Util_LuceneVersion_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.#ctor(Lucene.Net.Util.LuceneVersion,Lucene.Net.Util.AttributeSource.AttributeFactory,System.IO.TextReader)">StandardTokenizer(LuceneVersion, AttributeSource.AttributeFactory, TextReader)</h4> |
| <div class="markdown level1 summary"><p>Creates a new <a class="xref" href="Lucene.Net.Analysis.Standard.StandardTokenizer.html">StandardTokenizer</a> with a given <span class="xref">Lucene.Net.Util.AttributeSource.AttributeFactory</span> </p> |
| </div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public StandardTokenizer(LuceneVersion matchVersion, AttributeSource.AttributeFactory factory, TextReader input)</code></pre> |
| </div> |
| <h5 class="parameters">Parameters</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Name</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">Lucene.Net.Util.LuceneVersion</span></td> |
| <td><span class="parametername">matchVersion</span></td> |
| <td></td> |
| </tr> |
| <tr> |
| <td><span class="xref">Lucene.Net.Util.AttributeSource.AttributeFactory</span></td> |
| <td><span class="parametername">factory</span></td> |
| <td></td> |
| </tr> |
| <tr> |
| <td><span class="xref">System.IO.TextReader</span></td> |
| <td><span class="parametername">input</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer__ctor_Lucene_Net_Util_LuceneVersion_System_IO_TextReader_.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.%23ctor(Lucene.Net.Util.LuceneVersion%2CSystem.IO.TextReader)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L137">View Source</a> |
| </span> |
| <a id="Lucene_Net_Analysis_Standard_StandardTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.#ctor*"></a> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer__ctor_Lucene_Net_Util_LuceneVersion_System_IO_TextReader_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.#ctor(Lucene.Net.Util.LuceneVersion,System.IO.TextReader)">StandardTokenizer(LuceneVersion, TextReader)</h4> |
| <div class="markdown level1 summary"><p>Creates a new instance of the <a class="xref" href="Lucene.Net.Analysis.Standard.StandardTokenizer.html">StandardTokenizer</a>. Attaches |
| the <code data-dev-comment-type="paramref" class="paramref">input</code> to the newly created JFlex-generated (then ported to .NET) scanner.</p> |
| </div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public StandardTokenizer(LuceneVersion matchVersion, TextReader input)</code></pre> |
| </div> |
| <h5 class="parameters">Parameters</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Name</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">Lucene.Net.Util.LuceneVersion</span></td> |
| <td><span class="parametername">matchVersion</span></td> |
| <td><p>Lucene compatibility version - See <a class="xref" href="Lucene.Net.Analysis.Standard.StandardTokenizer.html">StandardTokenizer</a> </p> |
| </td> |
| </tr> |
| <tr> |
| <td><span class="xref">System.IO.TextReader</span></td> |
| <td><span class="parametername">input</span></td> |
| <td><p>The input reader</p> |
| <p>See <a href="http://issues.apache.org/jira/browse/LUCENE-1068">http://issues.apache.org/jira/browse/LUCENE-1068</a> </p> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| <h3 id="fields">Fields |
| </h3> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_ACRONYM.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.ACRONYM%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L65">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_ACRONYM" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.ACRONYM">ACRONYM</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">[Obsolete("(3.1)")] |
| public const int ACRONYM = 2</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_ACRONYM_DEP.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.ACRONYM_DEP%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L80">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_ACRONYM_DEP" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.ACRONYM_DEP">ACRONYM_DEP</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">[Obsolete("(3.1)")] |
| public const int ACRONYM_DEP = 8</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_ALPHANUM.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.ALPHANUM%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L59">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_ALPHANUM" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.ALPHANUM">ALPHANUM</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int ALPHANUM = 0</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_APOSTROPHE.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.APOSTROPHE%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L62">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_APOSTROPHE" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.APOSTROPHE">APOSTROPHE</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">[Obsolete("(3.1)")] |
| public const int APOSTROPHE = 1</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_CJ.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.CJ%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L76">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_CJ" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.CJ">CJ</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">[Obsolete("(3.1)")] |
| public const int CJ = 7</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_COMPANY.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.COMPANY%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L68">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_COMPANY" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.COMPANY">COMPANY</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">[Obsolete("(3.1)")] |
| public const int COMPANY = 3</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_EMAIL.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.EMAIL%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L69">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_EMAIL" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.EMAIL">EMAIL</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int EMAIL = 4</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_HANGUL.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.HANGUL%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L86">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_HANGUL" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.HANGUL">HANGUL</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int HANGUL = 13</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_HIRAGANA.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.HIRAGANA%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L84">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_HIRAGANA" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.HIRAGANA">HIRAGANA</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int HIRAGANA = 11</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_HOST.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.HOST%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L72">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_HOST" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.HOST">HOST</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">[Obsolete("(3.1)")] |
| public const int HOST = 5</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_IDEOGRAPHIC.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.IDEOGRAPHIC%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L83">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_IDEOGRAPHIC" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.IDEOGRAPHIC">IDEOGRAPHIC</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int IDEOGRAPHIC = 10</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_KATAKANA.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.KATAKANA%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L85">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_KATAKANA" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.KATAKANA">KATAKANA</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int KATAKANA = 12</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_NUM.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.NUM%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L73">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_NUM" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.NUM">NUM</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int NUM = 6</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_SOUTHEAST_ASIAN.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.SOUTHEAST_ASIAN%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L82">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_SOUTHEAST_ASIAN" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.SOUTHEAST_ASIAN">SOUTHEAST_ASIAN</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public const int SOUTHEAST_ASIAN = 9</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_TOKEN_TYPES.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.TOKEN_TYPES%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L90">View Source</a> |
| </span> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_TOKEN_TYPES" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.TOKEN_TYPES">TOKEN_TYPES</h4> |
| <div class="markdown level1 summary"><p>String token types that correspond to token type int constants </p> |
| </div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public static readonly string[] TOKEN_TYPES</code></pre> |
| </div> |
| <h5 class="fieldValue">Field Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.String</span>[]</td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <h3 id="properties">Properties |
| </h3> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_MaxTokenLength.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.MaxTokenLength%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L115">View Source</a> |
| </span> |
| <a id="Lucene_Net_Analysis_Standard_StandardTokenizer_MaxTokenLength_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.MaxTokenLength*"></a> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_MaxTokenLength" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.MaxTokenLength">MaxTokenLength</h4> |
| <div class="markdown level1 summary"><p>Set the max allowed token length. Any token longer |
| than this is skipped. </p> |
| </div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public int MaxTokenLength { get; set; }</code></pre> |
| </div> |
| <h5 class="propertyValue">Property Value</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Int32</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <h3 id="methods">Methods |
| </h3> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_Dispose_System_Boolean_.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.Dispose(System.Boolean)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L250">View Source</a> |
| </span> |
| <a id="Lucene_Net_Analysis_Standard_StandardTokenizer_Dispose_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.Dispose*"></a> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_Dispose_System_Boolean_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.Dispose(System.Boolean)">Dispose(Boolean)</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">protected override void Dispose(bool disposing)</code></pre> |
| </div> |
| <h5 class="parameters">Parameters</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Name</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Boolean</span></td> |
| <td><span class="parametername">disposing</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <h5 class="overrides">Overrides</h5> |
| <div><a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_Dispose_System_Boolean_">Tokenizer.Dispose(Boolean)</a></div> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_End.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.End%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L240">View Source</a> |
| </span> |
| <a id="Lucene_Net_Analysis_Standard_StandardTokenizer_End_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.End*"></a> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_End" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.End">End()</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public override sealed void End()</code></pre> |
| </div> |
| <h5 class="overrides">Overrides</h5> |
| <div><span class="xref">Lucene.Net.Analysis.TokenStream.End()</span></div> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_IncrementToken.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.IncrementToken%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L195">View Source</a> |
| </span> |
| <a id="Lucene_Net_Analysis_Standard_StandardTokenizer_IncrementToken_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.IncrementToken*"></a> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_IncrementToken" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.IncrementToken">IncrementToken()</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public override sealed bool IncrementToken()</code></pre> |
| </div> |
| <h5 class="returns">Returns</h5> |
| <table class="table table-bordered table-striped table-condensed"> |
| <thead> |
| <tr> |
| <th>Type</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><span class="xref">System.Boolean</span></td> |
| <td></td> |
| </tr> |
| </tbody> |
| </table> |
| <h5 class="overrides">Overrides</h5> |
| <div><span class="xref">Lucene.Net.Analysis.TokenStream.IncrementToken()</span></div> |
| <span class="small pull-right mobile-hide"> |
| <span class="divider">|</span> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer_Reset.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer.Reset%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a> |
| </span> |
| <span class="small pull-right mobile-hide"> |
| <a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L259">View Source</a> |
| </span> |
| <a id="Lucene_Net_Analysis_Standard_StandardTokenizer_Reset_" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.Reset*"></a> |
| <h4 id="Lucene_Net_Analysis_Standard_StandardTokenizer_Reset" data-uid="Lucene.Net.Analysis.Standard.StandardTokenizer.Reset">Reset()</h4> |
| <div class="markdown level1 summary"></div> |
| <div class="markdown level1 conceptual"></div> |
| <h5 class="decalaration">Declaration</h5> |
| <div class="codewrapper"> |
| <pre><code class="lang-csharp hljs">public override void Reset()</code></pre> |
| </div> |
| <h5 class="overrides">Overrides</h5> |
| <div><span class="xref">Lucene.Net.Analysis.Tokenizer.Reset()</span></div> |
| <h3 id="implements">Implements</h3> |
| <div> |
| <span class="xref">System.IDisposable</span> |
| </div> |
| </article> |
| </div> |
| |
| <div class="hidden-sm col-md-2" role="complementary"> |
| <div class="sideaffix"> |
| <div class="contribution"> |
| <ul class="nav"> |
| <li> |
| <a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_StandardTokenizer.md&value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.StandardTokenizer%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A" class="contribution-link">Improve this Doc</a> |
| </li> |
| <li> |
| <a href="https://github.com/apache/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Common/Analysis/Standard/StandardTokenizer.cs/#L53" class="contribution-link">View Source</a> |
| </li> |
| </ul> |
| </div> |
| <nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix"> |
| <!-- <p><a class="back-to-top" href="#top">Back to top</a><p> --> |
| </nav> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| <footer> |
| <div class="grad-bottom"></div> |
| <div class="footer"> |
| <div class="container"> |
| <span class="pull-right"> |
| <a href="#top">Back to top</a> |
| </span> |
| Copyright © 2020 The Apache Software Foundation, Licensed under the <a href='http://www.apache.org/licenses/LICENSE-2.0' target='_blank'>Apache License, Version 2.0</a><br> <small>Apache Lucene.Net, Lucene.Net, Apache, the Apache feather logo, and the Apache Lucene.Net project logo are trademarks of The Apache Software Foundation. <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</small> |
| |
| </div> |
| </div> |
| </footer> |
| </div> |
| |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.js"></script> |
| <script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.js"></script> |
| </body> |
| </html> |
