blob: c128ecbf73bec68374df6194b5ce4d3c3af393c8 [file] [log] [blame]
<!DOCTYPE html>
<!--[if IE]><![endif]-->
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Class TokenStreamToAutomaton
| Apache Lucene.NET 4.8.0-beta00010 Documentation </title>
<meta name="viewport" content="width=device-width">
<meta name="title" content="Class TokenStreamToAutomaton
| Apache Lucene.NET 4.8.0-beta00010 Documentation ">
<meta name="generator" content="docfx 2.56.0.0">
<link rel="shortcut icon" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/favicon.ico">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.css">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.css">
<link rel="stylesheet" href="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.css">
<meta property="docfx:navrel" content="toc.html">
<meta property="docfx:tocrel" content="core/toc.html">
<meta property="docfx:rel" content="https://lucenenet.apache.org/docs/4.8.0-beta00009/">
</head>
<body data-spy="scroll" data-target="#affix" data-offset="120">
<div id="wrapper">
<header>
<nav id="autocollapse" class="navbar ng-scope" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/">
<img id="logo" class="svg" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/logo/lucene-net-color.png" alt="">
</a>
</div>
<div class="collapse navbar-collapse" id="navbar">
<form class="navbar-form navbar-right" role="search" id="search">
<div class="form-group">
<input type="text" class="form-control" id="search-query" placeholder="Search" autocomplete="off">
</div>
</form>
</div>
</div>
</nav>
<div class="subnav navbar navbar-default">
<div class="container hide-when-search">
<ul class="level0 breadcrumb">
<li>
<a href="https://lucenenet.apache.org/docs/4.8.0-beta00009/">API</a>
<span id="breadcrumb">
<ul class="breadcrumb">
<li></li>
</ul>
</span>
</li>
</ul>
</div>
</div>
</header>
<div class="container body-content">
<div id="search-results">
<div class="search-list"></div>
<div class="sr-items">
<p><i class="glyphicon glyphicon-refresh index-loading"></i></p>
</div>
<ul id="pagination"></ul>
</div>
</div>
<div role="main" class="container body-content hide-when-search">
<div class="sidenav hide-when-search">
<a class="btn toc-toggle collapse" data-toggle="collapse" href="#sidetoggle" aria-expanded="false" aria-controls="sidetoggle">Show / Hide Table of Contents</a>
<div class="sidetoggle collapse" id="sidetoggle">
<div id="sidetoc"></div>
</div>
</div>
<div class="article row grid-right">
<div class="col-md-10">
<article class="content wrap" id="_content" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton">
<h1 id="Lucene_Net_Analysis_TokenStreamToAutomaton" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton" class="text-break">Class TokenStreamToAutomaton
</h1>
<div class="markdown level0 summary"><p>Consumes a <a class="xref" href="Lucene.Net.Analysis.TokenStream.html">TokenStream</a> and creates an <a class="xref" href="Lucene.Net.Util.Automaton.Automaton.html">Automaton</a>
where the transition labels are UTF8 bytes (or Unicode
code points if unicodeArcs is true) from the <a class="xref" href="Lucene.Net.Analysis.TokenAttributes.ITermToBytesRefAttribute.html">ITermToBytesRefAttribute</a>.
Between tokens we insert
<a class="xref" href="Lucene.Net.Analysis.TokenStreamToAutomaton.html#Lucene_Net_Analysis_TokenStreamToAutomaton_POS_SEP">POS_SEP</a> and for holes we insert <a class="xref" href="Lucene.Net.Analysis.TokenStreamToAutomaton.html#Lucene_Net_Analysis_TokenStreamToAutomaton_HOLE">HOLE</a>.</p>
<div class="lucene-block lucene-experimental">This is a Lucene.NET EXPERIMENTAL API, use at your own risk</div></div>
<div class="markdown level0 conceptual"></div>
<div class="inheritance">
<h5>Inheritance</h5>
<div class="level0"><span class="xref">System.Object</span></div>
<div class="level1"><span class="xref">TokenStreamToAutomaton</span></div>
</div>
<div class="inheritedMembers">
<h5>Inherited Members</h5>
<div>
<span class="xref">System.Object.Equals(System.Object)</span>
</div>
<div>
<span class="xref">System.Object.Equals(System.Object, System.Object)</span>
</div>
<div>
<span class="xref">System.Object.GetHashCode()</span>
</div>
<div>
<span class="xref">System.Object.GetType()</span>
</div>
<div>
<span class="xref">System.Object.MemberwiseClone()</span>
</div>
<div>
<span class="xref">System.Object.ReferenceEquals(System.Object, System.Object)</span>
</div>
<div>
<span class="xref">System.Object.ToString()</span>
</div>
</div>
<h6><strong>Namespace</strong>: <a class="xref" href="Lucene.Net.Analysis.html">Lucene.Net.Analysis</a></h6>
<h6><strong>Assembly</strong>: Lucene.Net.dll</h6>
<h5 id="Lucene_Net_Analysis_TokenStreamToAutomaton_syntax">Syntax</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public class TokenStreamToAutomaton</code></pre>
</div>
<h3 id="constructors">Constructors
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton__ctor.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton.%23ctor%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L50">View Source</a>
</span>
<a id="Lucene_Net_Analysis_TokenStreamToAutomaton__ctor_" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.#ctor*"></a>
<h4 id="Lucene_Net_Analysis_TokenStreamToAutomaton__ctor" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.#ctor">TokenStreamToAutomaton()</h4>
<div class="markdown level1 summary"><p>Sole constructor. </p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public TokenStreamToAutomaton()</code></pre>
</div>
<h3 id="fields">Fields
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton_HOLE.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton.HOLE%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L120">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_TokenStreamToAutomaton_HOLE" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.HOLE">HOLE</h4>
<div class="markdown level1 summary"><p>We add this arc to represent a hole. </p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int HOLE = 30</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton_POS_SEP.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton.POS_SEP%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L116">View Source</a>
</span>
<h4 id="Lucene_Net_Analysis_TokenStreamToAutomaton_POS_SEP" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.POS_SEP">POS_SEP</h4>
<div class="markdown level1 summary"><p>We create transition between two adjacent tokens. </p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public const int POS_SEP = 31</code></pre>
</div>
<h5 class="fieldValue">Field Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Int32</span></td>
<td></td>
</tr>
</tbody>
</table>
<h3 id="properties">Properties
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton_PreservePositionIncrements.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton.PreservePositionIncrements%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L57">View Source</a>
</span>
<a id="Lucene_Net_Analysis_TokenStreamToAutomaton_PreservePositionIncrements_" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.PreservePositionIncrements*"></a>
<h4 id="Lucene_Net_Analysis_TokenStreamToAutomaton_PreservePositionIncrements" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.PreservePositionIncrements">PreservePositionIncrements</h4>
<div class="markdown level1 summary"><p>Whether to generate holes in the automaton for missing positions, <code>true</code> by default. </p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public virtual bool PreservePositionIncrements { get; set; }</code></pre>
</div>
<h5 class="propertyValue">Property Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton_UnicodeArcs.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton.UnicodeArcs%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L67">View Source</a>
</span>
<a id="Lucene_Net_Analysis_TokenStreamToAutomaton_UnicodeArcs_" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.UnicodeArcs*"></a>
<h4 id="Lucene_Net_Analysis_TokenStreamToAutomaton_UnicodeArcs" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.UnicodeArcs">UnicodeArcs</h4>
<div class="markdown level1 summary"><p>Whether to make transition labels Unicode code points instead of UTF8 bytes,
<code>false</code> by default</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public virtual bool UnicodeArcs { get; set; }</code></pre>
</div>
<h5 class="propertyValue">Property Value</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><span class="xref">System.Boolean</span></td>
<td></td>
</tr>
</tbody>
</table>
<h3 id="methods">Methods
</h3>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton_ChangeToken_Lucene_Net_Util_BytesRef_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton.ChangeToken(Lucene.Net.Util.BytesRef)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L109">View Source</a>
</span>
<a id="Lucene_Net_Analysis_TokenStreamToAutomaton_ChangeToken_" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.ChangeToken*"></a>
<h4 id="Lucene_Net_Analysis_TokenStreamToAutomaton_ChangeToken_Lucene_Net_Util_BytesRef_" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.ChangeToken(Lucene.Net.Util.BytesRef)">ChangeToken(BytesRef)</h4>
<div class="markdown level1 summary"><p>Subclass &amp; implement this if you need to change the
token (such as escaping certain bytes) before it&apos;s
turned into a graph.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">protected virtual BytesRef ChangeToken(BytesRef in)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a class="xref" href="Lucene.Net.Util.BytesRef.html">BytesRef</a></td>
<td><span class="parametername">in</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="returns">Returns</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a class="xref" href="Lucene.Net.Util.BytesRef.html">BytesRef</a></td>
<td></td>
</tr>
</tbody>
</table>
<span class="small pull-right mobile-hide">
<span class="divider">|</span>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton_ToAutomaton_Lucene_Net_Analysis_TokenStream_.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton.ToAutomaton(Lucene.Net.Analysis.TokenStream)%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A">Improve this Doc</a>
</span>
<span class="small pull-right mobile-hide">
<a href="https://github.com/NightOwl888/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L128">View Source</a>
</span>
<a id="Lucene_Net_Analysis_TokenStreamToAutomaton_ToAutomaton_" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.ToAutomaton*"></a>
<h4 id="Lucene_Net_Analysis_TokenStreamToAutomaton_ToAutomaton_Lucene_Net_Analysis_TokenStream_" data-uid="Lucene.Net.Analysis.TokenStreamToAutomaton.ToAutomaton(Lucene.Net.Analysis.TokenStream)">ToAutomaton(TokenStream)</h4>
<div class="markdown level1 summary"><p>Pulls the graph (including <a class="xref" href="Lucene.Net.Analysis.TokenAttributes.IPositionLengthAttribute.html">IPositionLengthAttribute</a>
from the provided <a class="xref" href="Lucene.Net.Analysis.TokenStream.html">TokenStream</a>, and creates the corresponding
automaton where arcs are bytes (or Unicode code points
if unicodeArcs = true) from each term.</p>
</div>
<div class="markdown level1 conceptual"></div>
<h5 class="decalaration">Declaration</h5>
<div class="codewrapper">
<pre><code class="lang-csharp hljs">public virtual Automaton ToAutomaton(TokenStream in)</code></pre>
</div>
<h5 class="parameters">Parameters</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a class="xref" href="Lucene.Net.Analysis.TokenStream.html">TokenStream</a></td>
<td><span class="parametername">in</span></td>
<td></td>
</tr>
</tbody>
</table>
<h5 class="returns">Returns</h5>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a class="xref" href="Lucene.Net.Util.Automaton.Automaton.html">Automaton</a></td>
<td></td>
</tr>
</tbody>
</table>
</article>
</div>
<div class="hidden-sm col-md-2" role="complementary">
<div class="sideaffix">
<div class="contribution">
<ul class="nav">
<li>
<a href="https://github.com/apache/lucenenet/new/docs/4.8.0-beta00010/websites/apidocs/apiSpec/new?filename=Lucene_Net_Analysis_TokenStreamToAutomaton.md&amp;value=---%0Auid%3A%20Lucene.Net.Analysis.TokenStreamToAutomaton%0Asummary%3A%20'*You%20can%20override%20summary%20for%20the%20API%20here%20using%20*MARKDOWN*%20syntax'%0A---%0A%0A*Please%20type%20below%20more%20information%20about%20this%20API%3A*%0A%0A" class="contribution-link">Improve this Doc</a>
</li>
<li>
<a href="https://github.com/apache/lucenenet/blob/release/Lucene.Net_4_8_0_beta00010/src/Lucene.Net/Analysis/TokenStreamToAutomaton.cs/#L41" class="contribution-link">View Source</a>
</li>
</ul>
</div>
<nav class="bs-docs-sidebar hidden-print hidden-xs hidden-sm affix" id="affix">
<!-- <p><a class="back-to-top" href="#top">Back to top</a><p> -->
</nav>
</div>
</div>
</div>
</div>
<footer>
<div class="grad-bottom"></div>
<div class="footer">
<div class="container">
<span class="pull-right">
<a href="#top">Back to top</a>
</span>
Copyright © 2020 Licensed to the Apache Software Foundation (ASF)
</div>
</div>
</footer>
</div>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.vendor.js"></script>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/docfx.js"></script>
<script type="text/javascript" src="https://lucenenet.apache.org/docs/4.8.0-beta00009/styles/main.js"></script>
</body>
</html>