blob: c28e389145d058af02da95e0a052d564de746d0e [file] [log] [blame]
<!DOCTYPE html>
<!--[if IE]><![endif]-->
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Class UAX29URLEmailTokenizer
| Apache Lucene.NET 4.8.0-beta00010 Documentation </title>
<meta name="viewport" content="width=device-width">
<meta name="title" content="Class UAX29URLEmailTokenizer
| Apache Lucene.NET 4.8.0-beta00010 Documentation ">
<meta name="generator" content="docfx 2.56.0.0">
<link rel="shortcut icon" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/favicon.ico">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.css">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.css">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.css">
<meta property="docfx:navrel" content="toc.html">
<meta property="docfx:tocrel" content="analysis-common/toc.html">
<meta property="docfx:rel" content="https://lucenenet.apache.org/docs/4.8.0-beta00009/">
</head>
<body data-spy="scroll" data-target="#affix" data-offset="120">
<div id="wrapper">
<header>
<nav id="autocollapse" class="navbar ng-scope" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/">
<img id="logo" class="svg" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/lucene-net-color.png" alt="">
</a>
</div>
<div class="collapse navbar-collapse" id="navbar">
<form class="navbar-form navbar-right" role="search" id="search">
<div class="form-group">
<input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off">
</div>
</form>
</div>
</div>
</nav>
<div class="subnav navbar navbar-default">
<div class="container hide-when-search">
<ul class="level0 breadcrumb">
<li>
<a href="https://lucenenet.apache.org/docs/4.8.0-beta00009/">API</a>
<span id="breadcrumb">
<ul class="breadcrumb">
<li></li>
</ul>
</span>
</li>
</ul>
</div>
</div>
</header>
<div class="container body-content">
<div id="search-results">
<div class="search-list"></div>
<div class="sr-items">
<p><i class="glyphicon glyphicon-refresh index-loading"></i></p>
</div>
<ul id="pagination"></ul>
</div>
</div>
<div role="main" class="container body-content hide-when-search">
<div class="sidenav hide-when-search">
<a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a>
<div class="sidetoggle collapse" id="sidetoggle">
<div id="sidetoc"></div>
</div>
</div>
<div class="article row grid-right">
<div class="col-md-10">
<article class="content wrap" id="_content" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer">
<h1 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer" class="text-break">Class UAX29URLEmailTokenizer
</h1>
<div class="markdown level0 summary"><p>This class implements Word Break rules from the Unicode Text Segmentation
algorithm, as specified in `
<a href="http://unicode.org/reports/tr29/">Unicode Standard Annex #29</a>
URLs and email addresses are also tokenized according to the relevant RFCs.
<p>
Tokens produced are of the following types:
<ul><li>&lt;ALPHANUM&gt;: A sequence of alphabetic and numeric characters</li><li>&lt;NUM&gt;: A number</li><li>&lt;URL&gt;: A URL</li><li>&lt;EMAIL&gt;: An email address</li><li>&lt;SOUTHEAST_ASIAN&gt;: A sequence of characters from South and Southeast
Asian languages, including Thai, Lao, Myanmar, and Khmer</li><li>&lt;IDEOGRAPHIC&gt;: A single CJKV ideographic character</li><li>&lt;HIRAGANA&gt;: A single hiragana character</li></ul>
<p>You must specify the required <span class="xref">Lucene.Net.Util.LuceneVersion</span>
compatibility when creating <a class="xref" href="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.html">UAX29URLEmailTokenizer</a>:
<ul><li> As of 3.4, Hiragana and Han characters are no longer wrongly split
from their combining characters. If you use a previous version number,
you get the exact broken behavior for backwards compatibility.</li></ul>
</p></p>
</div>
<div class="markdown level0 conceptual"></div>
<div class="inheritance">
<h5>Inheritance</h5>
<div class="level0"><span class="xref">System.Object</span></div>
<div class="level1"><span class="xref">Lucene.Net.Util.AttributeSource</span></div>
<div class="level2"><span class="xref">Lucene.Net.Analysis.TokenStream</span></div>
<div class="level3"><span class="xref">Lucene.Net.Analysis.Tokenizer</span></div>
<div class="level4"><span class="xref">UAX29URLEmailTokenizer</span></div>
</div>
<div classs="implements">
<h5>Implements</h5>
<div><span class="xref">System.IDisposable</span></div>
</div>
<div class="inheritedMembers">
<h5>Inherited Members</h5>
<div>
<span class="xref">Lucene.Net.Analysis.Tokenizer.m_input</span>
</div>
<div>
<a class="xref" href="http://localhost:8080/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_CorrectOffset_System_Int32_">Tokenizer.CorrectOffset(Int32)</a>
</div>
<div>
<a class="xref" href="http://localhost:8080/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_SetReader_System_IO_TextReader_">Tokenizer.SetReader(TextReader)</a>
</div>
<div>
<span class="xref">Lucene.Net.Analysis.TokenStream.Dispose()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeFactory()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeClassesEnumerator()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeImplsEnumerator()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.AddAttributeImpl(Lucene.Net.Util.Attribute)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.AddAttribute&lt;T&gt;()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.HasAttributes</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.HasAttribute&lt;T&gt;()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttribute&lt;T&gt;()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.ClearAttributes()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.CaptureState()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.RestoreState(Lucene.Net.Util.AttributeSource.State)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetHashCode()</span>
</div>
<div>
<a class="xref" href="http://localhost:8080/api/core/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_Equals_System_Object_">AttributeSource.Equals(Object)</a>
</div>
<div>
<a class="xref" href="http://localhost:8080/api/core/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_ReflectAsString_System_Boolean_">AttributeSource.ReflectAsString(Boolean)</a>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.ReflectWith(Lucene.Net.Util.IAttributeReflector)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.CloneAttributes()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.CopyTo(Lucene.Net.Util.AttributeSource)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.ToString()</span>
</div>
<div>
<span class="xref">System.Object.Equals(System.Object, System.Object)</span>
</div>
<div>
<span class="xref">System.Object.GetType()</span>
</div>
<div>
<span class="xref">System.Object.MemberwiseClone()</span>
</div>
<div>
<span class="xref">System.Object.ReferenceEquals(System.Object, System.Object)</span>
</div>
</div>
<h6><strong>Namespace</strong>: <a class="xref" href="Lucene.Net.Analysis.Standard.html">Lucene.Net.Analysis.Standard</a></h6>
<h6><strong>Assembly</strong>: Lucene.Net.Analysis.Common.dll</h6>
<h5 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_syntax">Syntax</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public sealed class UAX29URLEmailTokenizer : Tokenizer, IDisposable</code></pre>
</div>
<h3 id="constructors">Constructors
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer__ctor_Lucene_Net_Util_LuceneVersion_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.%23ctor(Lucene.Net.Util.LuceneVersion%2CLucene.Net.Util.AttributeSource.AttributeFactory%2CSystem.IO.TextReader)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L121">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer__ctor_Lucene_Net_Util_LuceneVersion_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.#ctor(Lucene.Net.Util.LuceneVersion,Lucene.Net.Util.AttributeSource.AttributeFactory,System.IO.TextReader)">UAX29URLEmailTokenizer(LuceneVersion, AttributeSource.AttributeFactory, TextReader)</h4>
<div class="markdown level1 summary"><p>Creates a new <a class="xref" href="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.html">UAX29URLEmailTokenizer</a> with a given <span class="xref">Lucene.Net.Util.AttributeSource.AttributeFactory</span></p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public UAX29URLEmailTokenizer(LuceneVersion matchVersion, AttributeSource.AttributeFactory factory, TextReader input)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">Lucene.Net.Util.LuceneVersion</span></td>
<td><span class="parametername">matchVersion</span></td>
<td></td>
</tr>
<tr>
<td><span class="xref">Lucene.Net.Util.AttributeSource.AttributeFactory</span></td>
<td><span class="parametername">factory</span></td>
<td></td>
</tr>
<tr>
<td><span class="xref">System.IO.TextReader</span></td>
<td><span class="parametername">input</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer__ctor_Lucene_Net_Util_LuceneVersion_System_IO_TextReader_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.%23ctor(Lucene.Net.Util.LuceneVersion%2CSystem.IO.TextReader)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L112">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer__ctor_Lucene_Net_Util_LuceneVersion_System_IO_TextReader_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.#ctor(Lucene.Net.Util.LuceneVersion,System.IO.TextReader)">UAX29URLEmailTokenizer(LuceneVersion, TextReader)</h4>
<div class="markdown level1 summary"><p>Creates a new instance of the <a class="xref" href="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.html">UAX29URLEmailTokenizer</a>. Attaches
the <code data-dev-comment-type="paramref" class="paramref">input</code> to the newly created JFlex scanner.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public UAX29URLEmailTokenizer(LuceneVersion matchVersion, TextReader input)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">Lucene.Net.Util.LuceneVersion</span></td>
<td><span class="parametername">matchVersion</span></td>
<td><p>Lucene compatibility version </p>
</td>
</tr>
<tr>
<td><span class="xref">System.IO.TextReader</span></td>
<td><span class="parametername">input</span></td>
<td><p>The input reader </p>
</td>
</tr>
</tbody>
</table>
<h3 id="fields">Fields
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_ALPHANUM.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.ALPHANUM%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L61">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_ALPHANUM" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.ALPHANUM">ALPHANUM</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int ALPHANUM = 0</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_EMAIL.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.EMAIL%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L69">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_EMAIL" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.EMAIL">EMAIL</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int EMAIL = 8</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_HANGUL.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.HANGUL%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L67">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_HANGUL" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.HANGUL">HANGUL</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int HANGUL = 6</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_HIRAGANA.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.HIRAGANA%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L65">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_HIRAGANA" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.HIRAGANA">HIRAGANA</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int HIRAGANA = 4</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_IDEOGRAPHIC.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.IDEOGRAPHIC%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L64">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_IDEOGRAPHIC" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.IDEOGRAPHIC">IDEOGRAPHIC</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int IDEOGRAPHIC = 3</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_KATAKANA.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.KATAKANA%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L66">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_KATAKANA" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.KATAKANA">KATAKANA</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int KATAKANA = 5</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_NUM.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.NUM%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L62">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_NUM" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.NUM">NUM</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int NUM = 1</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_SOUTHEAST_ASIAN.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.SOUTHEAST_ASIAN%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L63">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_SOUTHEAST_ASIAN" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.SOUTHEAST_ASIAN">SOUTHEAST_ASIAN</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int SOUTHEAST_ASIAN = 2</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_TOKEN_TYPES.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.TOKEN_TYPES%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L73">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_TOKEN_TYPES" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.TOKEN_TYPES">TOKEN_TYPES</h4>
<div class="markdown level1 summary"><p>String token types that correspond to token type int constants </p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public static readonly string[] TOKEN_TYPES</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.String</span>[]</td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_URL.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.URL%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L68">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_URL" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.URL">URL</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int URL = 7</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<h3 id="properties">Properties
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_MaxTokenLength.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.MaxTokenLength%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L93">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_MaxTokenLength_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.MaxTokenLength*"></a>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_MaxTokenLength" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.MaxTokenLength">MaxTokenLength</h4>
<div class="markdown level1 summary"><p>Set the max allowed token length. Any token longer
than this is skipped. </p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public int MaxTokenLength { get; set; }</code></pre>
</div>
<h5 class="propertyValue">Property Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<h3 id="methods">Methods
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_Dispose_System_Boolean_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.Dispose(System.Boolean)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L216">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_Dispose_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.Dispose*"></a>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_Dispose_System_Boolean_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.Dispose(System.Boolean)">Dispose(Boolean)</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">protected override void Dispose(bool disposing)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td><span class="parametername">disposing</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="overrides">Overrides</h5>
<div><a class="xref" href="http://localhost:8080/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_Dispose_System_Boolean_">Tokenizer.Dispose(Boolean)</a></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_End.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.End%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L206">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_End_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.End*"></a>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_End" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.End">End()</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override sealed void End()</code></pre>
</div>
<h5 class="overrides">Overrides</h5>
<div><span class="xref">Lucene.Net.Analysis.TokenStream.End()</span></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_IncrementToken.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.IncrementToken%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L174">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_IncrementToken_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.IncrementToken*"></a>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_IncrementToken" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.IncrementToken">IncrementToken()</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override sealed bool IncrementToken()</code></pre>
</div>
<h5 class="returns">Returns</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="overrides">Overrides</h5>
<div><span class="xref">Lucene.Net.Analysis.TokenStream.IncrementToken()</span></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_Reset.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.Reset%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L225">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_Reset_" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.Reset*"></a>
<h4 id="Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer_Reset" data-uid="Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer.Reset">Reset()</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override void Reset()</code></pre>
</div>
<h5 class="overrides">Overrides</h5>
<div><span class="xref">Lucene.Net.Analysis.Tokenizer.Reset()</span></div>
<h3 id="implements">Implements</h3>
<div>
<span class="xref">System.IDisposable</span>
</div>
</article>
</div>
<div class="hidden-sm col-md-2" role="complementary">
<div class="sideaffix">
<div class="contribution">
<ul class="nav">
<li>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Standard_UAX29URLEmailTokenizer.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Standard.UAX29URLEmailTokenizer%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A" class="contribution-link">Improve this Doc</a>
</li>
<li>
<a href="https://github.com/apache/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net.Analysis.Common/Analysis/Standard/UAX29URLEmailTokenizer.cs/#L55" class="contribution-link">View Source</a>
</li>
</ul>
</div>
<nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix">
<!-- <p><a class="back-to-top" href="#top">Back to top</a><p> -->
</nav>
</div>
</div>
</div>
</div>
<footer>
<div class="grad-bottom"></div>
<div class="footer">
<div class="container">
<span class="pull-right">
<a href="#top">Back to top</a>
</span>
Copyright © 2020 Licensed to the Apache Software Foundation (ASF)
</div>
</div>
</footer>
</div>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.js"></script>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.js"></script>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.js"></script>
</body>
</html>