blob: 6671a590eded46ce0fdccdc12acc0d36adfcae80 [file] [log] [blame]
<!DOCTYPE html>
<!--[if IE]><![endif]-->
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Class WikipediaTokenizer
| Apache Lucene.NET 4.8.0-beta00008 Documentation </title>
<meta name="viewport" content="width=device-width">
<meta name="title" content="Class WikipediaTokenizer
| Apache Lucene.NET 4.8.0-beta00008 Documentation ">
<meta name="generator" content="docfx 2.50.0.0">
<link rel="shortcut icon" href="../../logo/favicon.ico">
<link rel="stylesheet" href="../../styles/docfx.vendor.css">
<link rel="stylesheet" href="../../styles/docfx.css">
<link rel="stylesheet" href="../../styles/main.css">
<meta property="docfx:navrel" content="../../toc.html">
<meta property="docfx:tocrel" content="../toc.html">
<meta property="docfx:rel" content="../../">
</head>
<body data-spy="scroll" data-target="#affix" data-offset="120">
<div id="wrapper">
<header>
<nav id="autocollapse" class="navbar ng-scope" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="../../index.html">
<img id="logo" class="svg" src="../../logo/lucene-net-color.png" alt="">
</a>
</div>
<div class="collapse navbar-collapse" id="navbar">
<form class="navbar-form navbar-right" role="search" id="search">
<div class="form-group">
<input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off">
</div>
</form>
</div>
</div>
</nav>
<div class="subnav navbar navbar-default">
<div class="container hide-when-search" id="breadcrumb">
<ul class="breadcrumb">
<li></li>
</ul>
</div>
</div>
</header>
<div class="container body-content">
<div id="search-results">
<div class="search-list"></div>
<div class="sr-items">
<p><i class="glyphicon glyphicon-refresh index-loading"></i></p>
</div>
<ul id="pagination"></ul>
</div>
</div>
<div role="main" class="container body-content hide-when-search">
<div class="sidenav hide-when-search">
<a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a>
<div class="sidetoggle collapse" id="sidetoggle">
<div id="sidetoc"></div>
</div>
</div>
<div class="article row grid-right">
<div class="col-md-10">
<article class="content wrap" id="_content" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer">
<h1 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer" class="text-break">Class WikipediaTokenizer
</h1>
<div class="markdown level0 summary"><p>Extension of <a class="xref" href="Lucene.Net.Analysis.Standard.StandardTokenizer.html">StandardTokenizer</a> that is aware of Wikipedia syntax. It is based off of the
Wikipedia tutorial available at <a href="http://en.wikipedia.org/wiki/Wikipedia:Tutorial">http://en.wikipedia.org/wiki/Wikipedia:Tutorial</a>, but it may not be complete.
<p>
<div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></div>
<div class="markdown level0 conceptual"></div>
<div class="inheritance">
<h5>Inheritance</h5>
<div class="level0"><span class="xref">System.Object</span></div>
<div class="level1"><a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html">AttributeSource</a></div>
<div class="level2"><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.TokenStream.html">TokenStream</a></div>
<div class="level3"><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.Tokenizer.html">Tokenizer</a></div>
<div class="level4"><span class="xref">WikipediaTokenizer</span></div>
</div>
<div classs="implements">
<h5>Implements</h5>
<div><span class="xref">System.IDisposable</span></div>
</div>
<div class="inheritedMembers">
<h5>Inherited Members</h5>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_m_input">Tokenizer.m_input</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_CorrectOffset_System_Int32_">Tokenizer.CorrectOffset(Int32)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_SetReader_System_IO_TextReader_">Tokenizer.SetReader(TextReader)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.TokenStream.html#Lucene_Net_Analysis_TokenStream_Dispose">TokenStream.Dispose()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_GetAttributeFactory">AttributeSource.GetAttributeFactory()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_GetAttributeClassesEnumerator">AttributeSource.GetAttributeClassesEnumerator()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_GetAttributeImplsEnumerator">AttributeSource.GetAttributeImplsEnumerator()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_AddAttributeImpl_Lucene_Net_Util_Attribute_">AttributeSource.AddAttributeImpl(Attribute)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_AddAttribute__1">AttributeSource.AddAttribute&lt;T&gt;()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_HasAttributes">AttributeSource.HasAttributes</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_HasAttribute__1">AttributeSource.HasAttribute&lt;T&gt;()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_GetAttribute__1">AttributeSource.GetAttribute&lt;T&gt;()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_ClearAttributes">AttributeSource.ClearAttributes()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_CaptureState">AttributeSource.CaptureState()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_RestoreState_Lucene_Net_Util_AttributeSource_State_">AttributeSource.RestoreState(AttributeSource.State)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_GetHashCode">AttributeSource.GetHashCode()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_Equals_System_Object_">AttributeSource.Equals(Object)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_ReflectAsString_System_Boolean_">AttributeSource.ReflectAsString(Boolean)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_ReflectWith_Lucene_Net_Util_IAttributeReflector_">AttributeSource.ReflectWith(IAttributeReflector)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_CloneAttributes">AttributeSource.CloneAttributes()</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_CopyTo_Lucene_Net_Util_AttributeSource_">AttributeSource.CopyTo(AttributeSource)</a>
</div>
<div>
<a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_ToString">AttributeSource.ToString()</a>
</div>
<div>
<span class="xref">System.Object.Equals(System.Object, System.Object)</span>
</div>
<div>
<span class="xref">System.Object.GetType()</span>
</div>
<div>
<span class="xref">System.Object.MemberwiseClone()</span>
</div>
<div>
<span class="xref">System.Object.ReferenceEquals(System.Object, System.Object)</span>
</div>
</div>
<h6><strong>Namespace</strong>: <a class="xref" href="Lucene.Net.Analysis.Wikipedia.html">Lucene.Net.Analysis.Wikipedia</a></h6>
<h6><strong>Assembly</strong>: Lucene.Net.Analysis.Common.dll</h6>
<h5 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_syntax">Syntax</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public sealed class WikipediaTokenizer : Tokenizer, IDisposable</code></pre>
</div>
<h3 id="constructors">Constructors
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_System_Int32_System_Collections_Generic_ICollection_System_String__.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.%23ctor(Lucene.Net.Util.AttributeSource.AttributeFactory%2CSystem.IO.TextReader%2CSystem.Int32%2CSystem.Collections.Generic.ICollection%7BSystem.String%7D)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L154">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_System_Int32_System_Collections_Generic_ICollection_System_String__" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.#ctor(Lucene.Net.Util.AttributeSource.AttributeFactory,System.IO.TextReader,System.Int32,System.Collections.Generic.ICollection{System.String})">WikipediaTokenizer(AttributeSource.AttributeFactory, TextReader, Int32, ICollection&lt;String&gt;)</h4>
<div class="markdown level1 summary"><p>Creates a new instance of the <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html">WikipediaTokenizer</a>. Attaches the
<code data-dev-comment-type="paramref" class="paramref">input</code> to a the newly created JFlex scanner. Uses the given <a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.AttributeFactory.html">AttributeSource.AttributeFactory</a>.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public WikipediaTokenizer(AttributeSource.AttributeFactory factory, TextReader input, int tokenOutput, ICollection&lt;string&gt; untokenizedTypes)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.AttributeFactory.html">AttributeSource.AttributeFactory</a></td>
<td><span class="parametername">factory</span></td>
<td><p>The <a class="xref" href="../Lucene.Net/Lucene.Net.Util.AttributeSource.AttributeFactory.html">AttributeSource.AttributeFactory</a> </p>
</td>
</tr>
<tr>
<td><span class="xref">System.IO.TextReader</span></td>
<td><span class="parametername">input</span></td>
<td><p>The input </p>
</td>
</tr>
<tr>
<td><span class="xref">System.Int32</span></td>
<td><span class="parametername">tokenOutput</span></td>
<td><p>One of <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html#Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_TOKENS_ONLY">TOKENS_ONLY</a>, <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html#Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_UNTOKENIZED_ONLY">UNTOKENIZED_ONLY</a>, <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html#Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOTH">BOTH</a> </p>
</td>
</tr>
<tr>
<td><span class="xref">System.Collections.Generic.ICollection</span>&lt;<span class="xref">System.String</span>&gt;</td>
<td><span class="parametername">untokenizedTypes</span></td>
<td><p>Untokenized types </p>
</td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_System_IO_TextReader_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.%23ctor(System.IO.TextReader)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L127">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_System_IO_TextReader_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.#ctor(System.IO.TextReader)">WikipediaTokenizer(TextReader)</h4>
<div class="markdown level1 summary"><p>Creates a new instance of the <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html">WikipediaTokenizer</a>. Attaches the
<code data-dev-comment-type="paramref" class="paramref">input</code> to a newly created JFlex scanner.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public WikipediaTokenizer(TextReader input)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.IO.TextReader</span></td>
<td><span class="parametername">input</span></td>
<td><p>The Input <span class="xref">System.IO.TextReader</span> </p>
</td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_System_IO_TextReader_System_Int32_System_Collections_Generic_ICollection_System_String__.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.%23ctor(System.IO.TextReader%2CSystem.Int32%2CSystem.Collections.Generic.ICollection%7BSystem.String%7D)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L139">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer__ctor_System_IO_TextReader_System_Int32_System_Collections_Generic_ICollection_System_String__" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.#ctor(System.IO.TextReader,System.Int32,System.Collections.Generic.ICollection{System.String})">WikipediaTokenizer(TextReader, Int32, ICollection&lt;String&gt;)</h4>
<div class="markdown level1 summary"><p>Creates a new instance of the <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html">WikipediaTokenizer</a>. Attaches the
<code data-dev-comment-type="paramref" class="paramref">input</code> to a the newly created JFlex scanner.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public WikipediaTokenizer(TextReader input, int tokenOutput, ICollection&lt;string&gt; untokenizedTypes)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.IO.TextReader</span></td>
<td><span class="parametername">input</span></td>
<td><p>The input </p>
</td>
</tr>
<tr>
<td><span class="xref">System.Int32</span></td>
<td><span class="parametername">tokenOutput</span></td>
<td><p>One of <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html#Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_TOKENS_ONLY">TOKENS_ONLY</a>, <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html#Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_UNTOKENIZED_ONLY">UNTOKENIZED_ONLY</a>, <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html#Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOTH">BOTH</a> </p>
</td>
</tr>
<tr>
<td><span class="xref">System.Collections.Generic.ICollection</span>&lt;<span class="xref">System.String</span>&gt;</td>
<td><span class="parametername">untokenizedTypes</span></td>
<td><p>Untokenized types </p>
</td>
</tr>
</tbody>
</table>
<h3 id="fields">Fields
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ACRONYM_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ACRONYM_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L49">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ACRONYM_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ACRONYM_ID">ACRONYM_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int ACRONYM_ID = 2</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ALPHANUM_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ALPHANUM_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L47">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ALPHANUM_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ALPHANUM_ID">ALPHANUM_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int ALPHANUM_ID = 0</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_APOSTROPHE_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.APOSTROPHE_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L48">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_APOSTROPHE_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.APOSTROPHE_ID">APOSTROPHE_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int APOSTROPHE_ID = 1</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L41">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD">BOLD</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string BOLD = &quot;b&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L59">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD_ID">BOLD_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int BOLD_ID = 12</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD_ITALICS.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD_ITALICS%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L43">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD_ITALICS" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD_ITALICS">BOLD_ITALICS</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string BOLD_ITALICS = &quot;bi&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD_ITALICS_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD_ITALICS_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L61">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOLD_ITALICS_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOLD_ITALICS_ID">BOLD_ITALICS_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int BOLD_ITALICS_ID = 14</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOTH.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOTH%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L100">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_BOTH" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.BOTH">BOTH</h4>
<div class="markdown level1 summary"><p>Output the both the untokenized token and the splits</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int BOTH = 2</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CATEGORY.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CATEGORY%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L40">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CATEGORY" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CATEGORY">CATEGORY</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string CATEGORY = &quot;c&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CATEGORY_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CATEGORY_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L58">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CATEGORY_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CATEGORY_ID">CATEGORY_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int CATEGORY_ID = 11</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CITATION.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CITATION%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L39">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CITATION" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CITATION">CITATION</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string CITATION = &quot;ci&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CITATION_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CITATION_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L57">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CITATION_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CITATION_ID">CITATION_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int CITATION_ID = 10</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CJ_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CJ_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L54">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_CJ_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.CJ_ID">CJ_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int CJ_ID = 7</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_COMPANY_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.COMPANY_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L50">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_COMPANY_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.COMPANY_ID">COMPANY_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int COMPANY_ID = 3</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EMAIL_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EMAIL_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L51">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EMAIL_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EMAIL_ID">EMAIL_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int EMAIL_ID = 4</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L36">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK">EXTERNAL_LINK</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string EXTERNAL_LINK = &quot;el&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L56">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK_ID">EXTERNAL_LINK_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int EXTERNAL_LINK_ID = 9</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK_URL.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK_URL%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L38">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK_URL" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK_URL">EXTERNAL_LINK_URL</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string EXTERNAL_LINK_URL = &quot;elu&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK_URL_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK_URL_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L64">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_EXTERNAL_LINK_URL_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.EXTERNAL_LINK_URL_ID">EXTERNAL_LINK_URL_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int EXTERNAL_LINK_URL_ID = 17</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_HEADING.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.HEADING%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L44">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_HEADING" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.HEADING">HEADING</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string HEADING = &quot;h&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_HEADING_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.HEADING_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L62">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_HEADING_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.HEADING_ID">HEADING_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int HEADING_ID = 15</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_HOST_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.HOST_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L52">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_HOST_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.HOST_ID">HOST_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int HOST_ID = 5</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_INTERNAL_LINK.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.INTERNAL_LINK%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L35">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_INTERNAL_LINK" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.INTERNAL_LINK">INTERNAL_LINK</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string INTERNAL_LINK = &quot;il&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_INTERNAL_LINK_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.INTERNAL_LINK_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L55">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_INTERNAL_LINK_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.INTERNAL_LINK_ID">INTERNAL_LINK_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int INTERNAL_LINK_ID = 8</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ITALICS.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ITALICS%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L42">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ITALICS" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ITALICS">ITALICS</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string ITALICS = &quot;i&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ITALICS_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ITALICS_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L60">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_ITALICS_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.ITALICS_ID">ITALICS_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int ITALICS_ID = 13</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_NUM_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.NUM_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L53">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_NUM_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.NUM_ID">NUM_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int NUM_ID = 6</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_SUB_HEADING.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.SUB_HEADING%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L45">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_SUB_HEADING" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.SUB_HEADING">SUB_HEADING</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const string SUB_HEADING = &quot;sh&quot;</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_SUB_HEADING_ID.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.SUB_HEADING_ID%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L63">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_SUB_HEADING_ID" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.SUB_HEADING_ID">SUB_HEADING_ID</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int SUB_HEADING_ID = 16</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_TOKEN_TYPES.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.TOKEN_TYPES%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L68">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_TOKEN_TYPES" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.TOKEN_TYPES">TOKEN_TYPES</h4>
<div class="markdown level1 summary"><p>String token types that correspond to token type int constants </p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public static readonly string[] TOKEN_TYPES</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span>[]</td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_TOKENS_ONLY.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.TOKENS_ONLY%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L92">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_TOKENS_ONLY" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.TOKENS_ONLY">TOKENS_ONLY</h4>
<div class="markdown level1 summary"><p>Only output tokens</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int TOKENS_ONLY = 0</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_UNTOKENIZED_ONLY.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.UNTOKENIZED_ONLY%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L96">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_UNTOKENIZED_ONLY" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.UNTOKENIZED_ONLY">UNTOKENIZED_ONLY</h4>
<div class="markdown level1 summary"><p>Only output untokenized tokens, which are tokens that would normally be split into several tokens</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int UNTOKENIZED_ONLY = 1</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_UNTOKENIZED_TOKEN_FLAG.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.UNTOKENIZED_TOKEN_FLAG%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L104">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_UNTOKENIZED_TOKEN_FLAG" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.UNTOKENIZED_TOKEN_FLAG">UNTOKENIZED_TOKEN_FLAG</h4>
<div class="markdown level1 summary"><p>This flag is used to indicate that the produced &quot;Token&quot; would, if <a class="xref" href="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.html#Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_TOKENS_ONLY">TOKENS_ONLY</a> was used, produce multiple tokens.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int UNTOKENIZED_TOKEN_FLAG = 1</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<h3 id="methods">Methods
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_Dispose_System_Boolean_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.Dispose(System.Boolean)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L318">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_Dispose_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.Dispose*"></a>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_Dispose_System_Boolean_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.Dispose(System.Boolean)">Dispose(Boolean)</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">protected override void Dispose(bool disposing)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td><span class="parametername">disposing</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="overrides">Overrides</h5>
<div><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_Dispose_System_Boolean_">Tokenizer.Dispose(Boolean)</a></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_End.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.End%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L339">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_End_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.End*"></a>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_End" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.End">End()</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override void End()</code></pre>
</div>
<h5 class="overrides">Overrides</h5>
<div><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.TokenStream.html#Lucene_Net_Analysis_TokenStream_End">TokenStream.End()</a></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_IncrementToken.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.IncrementToken%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L180">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_IncrementToken_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.IncrementToken*"></a>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_IncrementToken" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.IncrementToken">IncrementToken()</h4>
<div class="markdown level1 summary"><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.TokenStream.html#Lucene_Net_Analysis_TokenStream_IncrementToken">IncrementToken()</a>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override sealed bool IncrementToken()</code></pre>
</div>
<h5 class="returns">Returns</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="overrides">Overrides</h5>
<div><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.TokenStream.html#Lucene_Net_Analysis_TokenStream_IncrementToken">TokenStream.IncrementToken()</a></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_Reset.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.Reset%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L330">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_Reset_" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.Reset*"></a>
<h4 id="Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer_Reset" data-uid="Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer.Reset">Reset()</h4>
<div class="markdown level1 summary"><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.TokenStream.html#Lucene_Net_Analysis_TokenStream_Reset">Reset()</a>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override void Reset()</code></pre>
</div>
<h5 class="overrides">Overrides</h5>
<div><a class="xref" href="../Lucene.Net/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_Reset">Tokenizer.Reset()</a></div>
<h3 id="implements">Implements</h3>
<div>
<span class="xref">System.IDisposable</span>
</div>
</article>
</div>
<div class="hidden-sm col-md-2" role="complementary">
<div class="sideaffix">
<div class="contribution">
<ul class="nav">
<li>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00008/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Wikipedia_WikipediaTokenizer.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A" class="contribution-link">Improve this Doc</a>
</li>
<li>
<a href="https://github.com/Shazwazza/lucenenet/blob/docs-may/src/Lucene.Net.Analysis.Common/Analysis/Wikipedia/WikipediaTokenizer.cs/#L33" class="contribution-link">View Source</a>
</li>
</ul>
</div>
<nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix">
<!-- <p><a class="back-to-top" href="#top">Back to top</a><p> -->
</nav>
</div>
</div>
</div>
</div>
<footer>
<div class="grad-bottom"></div>
<div class="footer">
<div class="container">
<span class="pull-right">
<a href="#top">Back to top</a>
</span>
Copyright © 2020 Licensed to the Apache Software Foundation (ASF)
</div>
</div>
</footer>
</div>
<script type="text/javascript" src="../../styles/docfx.vendor.js"></script>
<script type="text/javascript" src="../../styles/docfx.js"></script>
<script type="text/javascript" src="../../styles/main.js"></script>
</body>
</html>