blob: c0914c8c657d5c58698c5102b0d4940111426424 [file] [log] [blame]
<!DOCTYPE html>
<!--[if IE]><![endif]-->
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Class JapaneseTokenizer
| Apache Lucene.NET 4.8.0-beta00013 Documentation </title>
<meta name="viewport" content="width=device-width">
<meta name="title" content="Class JapaneseTokenizer
| Apache Lucene.NET 4.8.0-beta00013 Documentation ">
<meta name="generator" content="docfx 2.56.2.0">
<link rel="shortcut icon" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/favicon.ico">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.css">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.css">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.css">
<meta property="docfx:navrel" content="toc.html">
<meta property="docfx:tocrel" content="analysis-kuromoji/toc.html">
<meta property="docfx:rel" content="https://lucenenet.apache.org/docs/4.8.0-beta00009/">
</head>
<body data-spy="scroll" data-target="#affix" data-offset="120">
<span id="forkongithub"><a href="https://github.com/apache/lucenenet" target="_blank">Fork me on GitHub</a></span>
<div id="wrapper">
<header>
<nav id="autocollapse" class="navbar ng-scope" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/">
<img id="logo" class="svg" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/lucene-net-color.png" alt="">
</a>
</div>
<div class="collapse navbar-collapse" id="navbar">
<form class="navbar-form navbar-right" role="search" id="search">
<div class="form-group">
<input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off">
</div>
</form>
</div>
</div>
</nav>
<div class="subnav navbar navbar-default">
<div class="container hide-when-search">
<ul class="level0 breadcrumb">
<li>
<a href="https://lucenenet.apache.org/docs/4.8.0-beta00009/">API</a>
<span id="breadcrumb">
<ul class="breadcrumb">
<li></li>
</ul>
</span>
</li>
</ul>
</div>
</div>
</header>
<div class="container body-content">
<div id="search-results">
<div class="search-list"></div>
<div class="sr-items">
<p><i class="glyphicon glyphicon-refresh index-loading"></i></p>
</div>
<ul id="pagination"></ul>
</div>
</div>
<div role="main" class="container body-content hide-when-search">
<div class="sidenav hide-when-search">
<a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a>
<div class="sidetoggle collapse" id="sidetoggle">
<div id="sidetoc"></div>
</div>
</div>
<div class="article row grid-right">
<div class="col-md-10">
<article class="content wrap" id="_content" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer">
<h1 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer" class="text-break">Class JapaneseTokenizer
</h1>
<div class="markdown level0 summary"><p>Tokenizer for Japanese that uses morphological analysis.</p>
</div>
<div class="markdown level0 conceptual"></div>
<div class="inheritance">
<h5>Inheritance</h5>
<div class="level0"><span class="xref">System.Object</span></div>
<div class="level1"><span class="xref">Lucene.Net.Util.AttributeSource</span></div>
<div class="level2"><span class="xref">Lucene.Net.Analysis.TokenStream</span></div>
<div class="level3"><span class="xref">Lucene.Net.Analysis.Tokenizer</span></div>
<div class="level4"><span class="xref">JapaneseTokenizer</span></div>
</div>
<div classs="implements">
<h5>Implements</h5>
<div><span class="xref">System.IDisposable</span></div>
</div>
<div class="inheritedMembers">
<h5>Inherited Members</h5>
<div>
<span class="xref">Lucene.Net.Analysis.Tokenizer.m_input</span>
</div>
<div>
<a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_CorrectOffset_System_Int32_">Tokenizer.CorrectOffset(Int32)</a>
</div>
<div>
<a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_SetReader_System_IO_TextReader_">Tokenizer.SetReader(TextReader)</a>
</div>
<div>
<span class="xref">Lucene.Net.Analysis.TokenStream.Dispose()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeFactory()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeClassesEnumerator()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttributeImplsEnumerator()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.AddAttributeImpl(Lucene.Net.Util.Attribute)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.AddAttribute&lt;T&gt;()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.HasAttributes</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.HasAttribute&lt;T&gt;()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetAttribute&lt;T&gt;()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.ClearAttributes()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.CaptureState()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.RestoreState(Lucene.Net.Util.AttributeSource.State)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.GetHashCode()</span>
</div>
<div>
<a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_Equals_System_Object_">AttributeSource.Equals(Object)</a>
</div>
<div>
<a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Util.AttributeSource.html#Lucene_Net_Util_AttributeSource_ReflectAsString_System_Boolean_">AttributeSource.ReflectAsString(Boolean)</a>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.ReflectWith(Lucene.Net.Util.IAttributeReflector)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.CloneAttributes()</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.CopyTo(Lucene.Net.Util.AttributeSource)</span>
</div>
<div>
<span class="xref">Lucene.Net.Util.AttributeSource.ToString()</span>
</div>
<div>
<span class="xref">System.Object.Equals(System.Object, System.Object)</span>
</div>
<div>
<span class="xref">System.Object.GetType()</span>
</div>
<div>
<span class="xref">System.Object.MemberwiseClone()</span>
</div>
<div>
<span class="xref">System.Object.ReferenceEquals(System.Object, System.Object)</span>
</div>
</div>
<h6><strong>Namespace</strong>: <a class="xref" href="Lucene.Net.Analysis.Ja.html">Lucene.Net.Analysis.Ja</a></h6>
<h6><strong>Assembly</strong>: Lucene.Net.Analysis.Kuromoji.dll</h6>
<h5 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_syntax">Syntax</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public sealed class JapaneseTokenizer : Tokenizer, IDisposable</code></pre>
</div>
<h5 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_remarks"><strong>Remarks</strong></h5>
<div class="markdown level0 remarks"><p>This tokenizer sets a number of additional attributes:
<ul><li><a class="xref" href="Lucene.Net.Analysis.Ja.TokenAttributes.IBaseFormAttribute.html">IBaseFormAttribute</a> containing base form for inflected adjectives and verbs.</li><li><a class="xref" href="Lucene.Net.Analysis.Ja.TokenAttributes.IPartOfSpeechAttribute.html">IPartOfSpeechAttribute</a> containing part-of-speech.</li><li><a class="xref" href="Lucene.Net.Analysis.Ja.TokenAttributes.IReadingAttribute.html">IReadingAttribute</a> containing reading and pronunciation.</li><li><a class="xref" href="Lucene.Net.Analysis.Ja.TokenAttributes.IInflectionAttribute.html">IInflectionAttribute</a> containing additional part-of-speech information for inflected forms.</li></ul>
<p>
This tokenizer uses a rolling Viterbi search to find the
least cost segmentation (path) of the incoming characters.
For tokens that appear to be compound (&gt; length 2 for all
Kanji, or &gt; length 7 for non-Kanji), we see if there is a
2nd best segmentation of that token after applying
penalties to the long tokens. If so, and the Mode is
<a class="xref" href="Lucene.Net.Analysis.Ja.JapaneseTokenizerMode.html#Lucene_Net_Analysis_Ja_JapaneseTokenizerMode_SEARCH">SEARCH</a>, we output the alternate segmentation
as well.</p>
</div>
<h3 id="constructors">Constructors
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer__ctor_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_Lucene_Net_Analysis_Ja_Dict_UserDictionary_System_Boolean_Lucene_Net_Analysis_Ja_JapaneseTokenizerMode_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.%23ctor(Lucene.Net.Util.AttributeSource.AttributeFactory%2CSystem.IO.TextReader%2CLucene.Net.Analysis.Ja.Dict.UserDictionary%2CSystem.Boolean%2CLucene.Net.Analysis.Ja.JapaneseTokenizerMode)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L159">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Ja_JapaneseTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer__ctor_Lucene_Net_Util_AttributeSource_AttributeFactory_System_IO_TextReader_Lucene_Net_Analysis_Ja_Dict_UserDictionary_System_Boolean_Lucene_Net_Analysis_Ja_JapaneseTokenizerMode_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.#ctor(Lucene.Net.Util.AttributeSource.AttributeFactory,System.IO.TextReader,Lucene.Net.Analysis.Ja.Dict.UserDictionary,System.Boolean,Lucene.Net.Analysis.Ja.JapaneseTokenizerMode)">JapaneseTokenizer(AttributeSource.AttributeFactory, TextReader, UserDictionary, Boolean, JapaneseTokenizerMode)</h4>
<div class="markdown level1 summary"><p>Create a new JapaneseTokenizer.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public JapaneseTokenizer(AttributeSource.AttributeFactory factory, TextReader input, UserDictionary userDictionary, bool discardPunctuation, JapaneseTokenizerMode mode)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">Lucene.Net.Util.AttributeSource.AttributeFactory</span></td>
<td><span class="parametername">factory</span></td>
<td><p>The AttributeFactory to use.</p>
</td>
</tr>
<tr>
<td><span class="xref">System.IO.TextReader</span></td>
<td><span class="parametername">input</span></td>
<td><p>TextReader containing text.</p>
</td>
</tr>
<tr>
<td><a class="xref" href="Lucene.Net.Analysis.Ja.Dict.UserDictionary.html">UserDictionary</a></td>
<td><span class="parametername">userDictionary</span></td>
<td><p>Optional: if non-null, user dictionary.</p>
</td>
</tr>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td><span class="parametername">discardPunctuation</span></td>
<td><p><code>true</code> if punctuation tokens should be dropped from the output.</p>
</td>
</tr>
<tr>
<td><a class="xref" href="Lucene.Net.Analysis.Ja.JapaneseTokenizerMode.html">JapaneseTokenizerMode</a></td>
<td><span class="parametername">mode</span></td>
<td><p>Tokenization mode.</p>
</td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer__ctor_System_IO_TextReader_Lucene_Net_Analysis_Ja_Dict_UserDictionary_System_Boolean_Lucene_Net_Analysis_Ja_JapaneseTokenizerMode_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.%23ctor(System.IO.TextReader%2CLucene.Net.Analysis.Ja.Dict.UserDictionary%2CSystem.Boolean%2CLucene.Net.Analysis.Ja.JapaneseTokenizerMode)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L146">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Ja_JapaneseTokenizer__ctor_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer__ctor_System_IO_TextReader_Lucene_Net_Analysis_Ja_Dict_UserDictionary_System_Boolean_Lucene_Net_Analysis_Ja_JapaneseTokenizerMode_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.#ctor(System.IO.TextReader,Lucene.Net.Analysis.Ja.Dict.UserDictionary,System.Boolean,Lucene.Net.Analysis.Ja.JapaneseTokenizerMode)">JapaneseTokenizer(TextReader, UserDictionary, Boolean, JapaneseTokenizerMode)</h4>
<div class="markdown level1 summary"><p>Create a new JapaneseTokenizer.
<p>
Uses the default AttributeFactory.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public JapaneseTokenizer(TextReader input, UserDictionary userDictionary, bool discardPunctuation, JapaneseTokenizerMode mode)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.IO.TextReader</span></td>
<td><span class="parametername">input</span></td>
<td><p>TextReader containing text.</p>
</td>
</tr>
<tr>
<td><a class="xref" href="Lucene.Net.Analysis.Ja.Dict.UserDictionary.html">UserDictionary</a></td>
<td><span class="parametername">userDictionary</span></td>
<td><p>Optional: if non-null, user dictionary.</p>
</td>
</tr>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td><span class="parametername">discardPunctuation</span></td>
<td><p><code>true</code> if punctuation tokens should be dropped from the output.</p>
</td>
</tr>
<tr>
<td><a class="xref" href="Lucene.Net.Analysis.Ja.JapaneseTokenizerMode.html">JapaneseTokenizerMode</a></td>
<td><span class="parametername">mode</span></td>
<td><p>Tokenization mode.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="fields">Fields
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer_DEFAULT_MODE.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.DEFAULT_MODE%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L63">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_DEFAULT_MODE" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.DEFAULT_MODE">DEFAULT_MODE</h4>
<div class="markdown level1 summary"><p>Default tokenization mode. Currently this is <a class="xref" href="Lucene.Net.Analysis.Ja.JapaneseTokenizerMode.html#Lucene_Net_Analysis_Ja_JapaneseTokenizerMode_SEARCH">SEARCH</a>.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public static readonly JapaneseTokenizerMode DEFAULT_MODE</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a class="xref" href="Lucene.Net.Analysis.Ja.JapaneseTokenizerMode.html">JapaneseTokenizerMode</a></td>
<td></td>
</tr>
</tbody>
</table>
<h3 id="properties">Properties
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer_GraphvizFormatter.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.GraphvizFormatter%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L226">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_GraphvizFormatter_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.GraphvizFormatter*"></a>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_GraphvizFormatter" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.GraphvizFormatter">GraphvizFormatter</h4>
<div class="markdown level1 summary"><p>Expert: set this to produce graphviz (dot) output of
the Viterbi lattice</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public GraphvizFormatter GraphvizFormatter { get; set; }</code></pre>
</div>
<h5 class="propertyValue">Property Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a class="xref" href="Lucene.Net.Analysis.Ja.GraphvizFormatter.html">GraphvizFormatter</a></td>
<td></td>
</tr>
</tbody>
</table>
<h3 id="methods">Methods
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer_Dispose_System_Boolean_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.Dispose(System.Boolean)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L232">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_Dispose_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.Dispose*"></a>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_Dispose_System_Boolean_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.Dispose(System.Boolean)">Dispose(Boolean)</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">protected override void Dispose(bool disposing)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td><span class="parametername">disposing</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="overrides">Overrides</h5>
<div><a class="xref" href="https://lucenenet.apache.org/docs/4.8.0-beta00013/api/core/Lucene.Net.Analysis.Tokenizer.html#Lucene_Net_Analysis_Tokenizer_Dispose_System_Boolean_">Tokenizer.Dispose(Boolean)</a></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer_End.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.End%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L262">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_End_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.End*"></a>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_End" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.End">End()</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override void End()</code></pre>
</div>
<h5 class="overrides">Overrides</h5>
<div><span class="xref">Lucene.Net.Analysis.TokenStream.End()</span></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer_IncrementToken.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.IncrementToken%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L365">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_IncrementToken_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.IncrementToken*"></a>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_IncrementToken" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.IncrementToken">IncrementToken()</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override bool IncrementToken()</code></pre>
</div>
<h5 class="returns">Returns</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="overrides">Overrides</h5>
<div><span class="xref">Lucene.Net.Analysis.TokenStream.IncrementToken()</span></div>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer_Reset.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer.Reset%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L241">View Source</a>
</span>
<a id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_Reset_" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.Reset*"></a>
<h4 id="Lucene_Net_Analysis_Ja_JapaneseTokenizer_Reset" data-uid="Lucene.Net.Analysis.Ja.JapaneseTokenizer.Reset">Reset()</h4>
<div class="markdown level1 summary"></div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public override void Reset()</code></pre>
</div>
<h5 class="overrides">Overrides</h5>
<div><span class="xref">Lucene.Net.Analysis.Tokenizer.Reset()</span></div>
<h3 id="implements">Implements</h3>
<div>
<span class="xref">System.IDisposable</span>
</div>
</article>
</div>
<div class="hidden-sm col-md-2" role="complementary">
<div class="sideaffix">
<div class="contribution">
<ul class="nav">
<li>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00013/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_Ja_JapaneseTokenizer.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.Ja.JapaneseTokenizer%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A" class="contribution-link">Improve this Doc</a>
</li>
<li>
<a href="https://github.com/apache/lucenenet/blob/fix/apidocs-layout/src/Lucene.Net.Analysis.Kuromoji/JapaneseTokenizer.cs/#L56" class="contribution-link">View Source</a>
</li>
</ul>
</div>
<nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix">
<!-- <p><a class="back-to-top" href="#top">Back to top</a><p> -->
</nav>
</div>
</div>
</div>
</div>
<footer>
<div class="grad-bottom"></div>
<div class="footer">
<div class="container">
<span class="pull-right">
<a href="#top">Back to top</a>
</span>
Copyright © 2020 The Apache Software Foundation, Licensed under the <a href='http://www.apache.org/licenses/LICENSE-2.0' target='_blank'>Apache License, Version 2.0</a><br> <small>Apache Lucene.Net, Lucene.Net, Apache, the Apache feather logo, and the Apache Lucene.Net project logo are trademarks of The Apache Software Foundation. <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</small>
</div>
</div>
</footer>
</div>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.js"></script>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.js"></script>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.js"></script>
</body>
</html>