| <!DOCTYPE HTML> |
| <html lang="de"> |
| <head> |
| <!-- Generated by javadoc (17) --> |
| <title>Uses of Interface opennlp.tools.tokenize.Tokenizer (Apache OpenNLP Tools 2.3.3 API)</title> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> |
| <meta name="description" content="use: package: opennlp.tools.tokenize, interface: Tokenizer"> |
| <meta name="generator" content="javadoc/ClassUseWriter"> |
| <link rel="stylesheet" type="text/css" href="../../../../stylesheet.css" title="Style"> |
| <link rel="stylesheet" type="text/css" href="../../../../script-dir/jquery-ui.min.css" title="Style"> |
| <link rel="stylesheet" type="text/css" href="../../../../jquery-ui.overrides.css" title="Style"> |
| <script type="text/javascript" src="../../../../script.js"></script> |
| <script type="text/javascript" src="../../../../script-dir/jquery-3.6.1.min.js"></script> |
| <script type="text/javascript" src="../../../../script-dir/jquery-ui.min.js"></script> |
| </head> |
| <body class="class-use-page"> |
| <script type="text/javascript">var pathtoroot = "../../../../"; |
| loadScripts(document, 'script');</script> |
| <noscript> |
| <div>JavaScript is disabled on your browser.</div> |
| </noscript> |
| <div class="flex-box"> |
| <header role="banner" class="flex-header"> |
| <nav role="navigation"> |
| <!-- ========= START OF TOP NAVBAR ======= --> |
| <div class="top-nav" id="navbar-top"> |
| <div class="skip-nav"><a href="#skip-navbar-top" title="Skip navigation links">Skip navigation links</a></div> |
| <ul id="navbar-top-firstrow" class="nav-list" title="Navigation"> |
| <li><a href="../../../../index.html">Overview</a></li> |
| <li><a href="../package-summary.html">Package</a></li> |
| <li><a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Class</a></li> |
| <li class="nav-bar-cell1-rev">Use</li> |
| <li><a href="../package-tree.html">Tree</a></li> |
| <li><a href="../../../../deprecated-list.html">Deprecated</a></li> |
| <li><a href="../../../../index-all.html">Index</a></li> |
| <li><a href="../../../../help-doc.html#use">Help</a></li> |
| </ul> |
| </div> |
| <div class="sub-nav"> |
| <div class="nav-list-search"><label for="search-input">SEARCH:</label> |
| <input type="text" id="search-input" value="search" disabled="disabled"> |
| <input type="reset" id="reset-button" value="reset" disabled="disabled"> |
| </div> |
| </div> |
| <!-- ========= END OF TOP NAVBAR ========= --> |
| <span class="skip-nav" id="skip-navbar-top"></span></nav> |
| </header> |
| <div class="flex-content"> |
| <main role="main"> |
| <div class="header"> |
| <h1 title="Uses of Interface opennlp.tools.tokenize.Tokenizer" class="title">Uses of Interface<br>opennlp.tools.tokenize.Tokenizer</h1> |
| </div> |
| <div class="caption"><span>Packages that use <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a></span></div> |
| <div class="summary-table two-column-summary"> |
| <div class="table-header col-first">Package</div> |
| <div class="table-header col-last">Description</div> |
| <div class="col-first even-row-color"><a href="#opennlp.tools.cmdline.parser">opennlp.tools.cmdline.parser</a></div> |
| <div class="col-last even-row-color"> </div> |
| <div class="col-first odd-row-color"><a href="#opennlp.tools.formats.brat">opennlp.tools.formats.brat</a></div> |
| <div class="col-last odd-row-color"> |
| <div class="block">Experimental package related to the corpus format used by the "brat rapid annotation tool" (brat).</div> |
| </div> |
| <div class="col-first even-row-color"><a href="#opennlp.tools.formats.muc">opennlp.tools.formats.muc</a></div> |
| <div class="col-last even-row-color"> |
| <div class="block">Experimental package related to the <code>MUC</code> corpus format.</div> |
| </div> |
| <div class="col-first odd-row-color"><a href="#opennlp.tools.tokenize">opennlp.tools.tokenize</a></div> |
| <div class="col-last odd-row-color"> |
| <div class="block">Contains classes related to finding token or words in a string.</div> |
| </div> |
| <div class="col-first even-row-color"><a href="#opennlp.tools.util.featuregen">opennlp.tools.util.featuregen</a></div> |
| <div class="col-last even-row-color"> |
| <div class="block">This package contains classes for generating sequence features.</div> |
| </div> |
| </div> |
| <section class="class-uses"> |
| <ul class="block-list"> |
| <li> |
| <section class="detail" id="opennlp.tools.cmdline.parser"> |
| <h2>Uses of <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> in <a href="../../cmdline/parser/package-summary.html">opennlp.tools.cmdline.parser</a></h2> |
| <div class="caption"><span>Methods in <a href="../../cmdline/parser/package-summary.html">opennlp.tools.cmdline.parser</a> with parameters of type <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a></span></div> |
| <div class="summary-table three-column-summary"> |
| <div class="table-header col-first">Modifier and Type</div> |
| <div class="table-header col-second">Method</div> |
| <div class="table-header col-last">Description</div> |
| <div class="col-first even-row-color"><code>static <a href="../../parser/Parse.html" title="class in opennlp.tools.parser">Parse</a>[]</code></div> |
| <div class="col-second even-row-color"><span class="type-name-label">ParserTool.</span><code><a href="../../cmdline/parser/ParserTool.html#parseLine(java.lang.String,opennlp.tools.parser.Parser,opennlp.tools.tokenize.Tokenizer,int)" class="member-name-link">parseLine</a><wbr>(<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a> line, |
| <a href="../../parser/Parser.html" title="interface in opennlp.tools.parser">Parser</a> parser, |
| <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| int numParses)</code></div> |
| <div class="col-last even-row-color"> </div> |
| </div> |
| </section> |
| </li> |
| <li> |
| <section class="detail" id="opennlp.tools.formats.brat"> |
| <h2>Uses of <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> in <a href="../../formats/brat/package-summary.html">opennlp.tools.formats.brat</a></h2> |
| <div class="caption"><span>Constructors in <a href="../../formats/brat/package-summary.html">opennlp.tools.formats.brat</a> with parameters of type <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a></span></div> |
| <div class="summary-table three-column-summary"> |
| <div class="table-header col-first">Modifier</div> |
| <div class="table-header col-second">Constructor</div> |
| <div class="table-header col-last">Description</div> |
| <div class="col-first even-row-color"><code> </code></div> |
| <div class="col-second even-row-color"><code><a href="../../formats/brat/BratDocumentParser.html#%3Cinit%3E(opennlp.tools.sentdetect.SentenceDetector,opennlp.tools.tokenize.Tokenizer)" class="member-name-link">BratDocumentParser</a><wbr>(<a href="../../sentdetect/SentenceDetector.html" title="interface in opennlp.tools.sentdetect">SentenceDetector</a> sentenceDetector, |
| <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer)</code></div> |
| <div class="col-last even-row-color"> </div> |
| <div class="col-first odd-row-color"><code> </code></div> |
| <div class="col-second odd-row-color"><code><a href="../../formats/brat/BratDocumentParser.html#%3Cinit%3E(opennlp.tools.sentdetect.SentenceDetector,opennlp.tools.tokenize.Tokenizer,java.util.Set)" class="member-name-link">BratDocumentParser</a><wbr>(<a href="../../sentdetect/SentenceDetector.html" title="interface in opennlp.tools.sentdetect">SentenceDetector</a> sentenceDetector, |
| <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/Set.html" title="class or interface in java.util" class="external-link">Set</a><<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>> nameTypes)</code></div> |
| <div class="col-last odd-row-color"> </div> |
| <div class="col-first even-row-color"><code> </code></div> |
| <div class="col-second even-row-color"><code><a href="../../formats/brat/BratNameSampleStream.html#%3Cinit%3E(opennlp.tools.sentdetect.SentenceDetector,opennlp.tools.tokenize.Tokenizer,opennlp.tools.util.ObjectStream)" class="member-name-link">BratNameSampleStream</a><wbr>(<a href="../../sentdetect/SentenceDetector.html" title="interface in opennlp.tools.sentdetect">SentenceDetector</a> sentDetector, |
| <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| <a href="../../util/ObjectStream.html" title="interface in opennlp.tools.util">ObjectStream</a><<a href="../../formats/brat/BratDocument.html" title="class in opennlp.tools.formats.brat">BratDocument</a>> samples)</code></div> |
| <div class="col-last even-row-color"> |
| <div class="block">Creates a new <a href="../../formats/brat/BratNameSampleStream.html" title="class in opennlp.tools.formats.brat"><code>BratNameSampleStream</code></a>.</div> |
| </div> |
| <div class="col-first odd-row-color"><code> </code></div> |
| <div class="col-second odd-row-color"><code><a href="../../formats/brat/BratNameSampleStream.html#%3Cinit%3E(opennlp.tools.sentdetect.SentenceDetector,opennlp.tools.tokenize.Tokenizer,opennlp.tools.util.ObjectStream,java.util.Set)" class="member-name-link">BratNameSampleStream</a><wbr>(<a href="../../sentdetect/SentenceDetector.html" title="interface in opennlp.tools.sentdetect">SentenceDetector</a> sentDetector, |
| <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| <a href="../../util/ObjectStream.html" title="interface in opennlp.tools.util">ObjectStream</a><<a href="../../formats/brat/BratDocument.html" title="class in opennlp.tools.formats.brat">BratDocument</a>> samples, |
| <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/Set.html" title="class or interface in java.util" class="external-link">Set</a><<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>> nameTypes)</code></div> |
| <div class="col-last odd-row-color"> |
| <div class="block">Creates a new <a href="../../formats/brat/BratNameSampleStream.html" title="class in opennlp.tools.formats.brat"><code>BratNameSampleStream</code></a>.</div> |
| </div> |
| </div> |
| </section> |
| </li> |
| <li> |
| <section class="detail" id="opennlp.tools.formats.muc"> |
| <h2>Uses of <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> in <a href="../../formats/muc/package-summary.html">opennlp.tools.formats.muc</a></h2> |
| <div class="caption"><span>Constructors in <a href="../../formats/muc/package-summary.html">opennlp.tools.formats.muc</a> with parameters of type <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a></span></div> |
| <div class="summary-table three-column-summary"> |
| <div class="table-header col-first">Modifier</div> |
| <div class="table-header col-second">Constructor</div> |
| <div class="table-header col-last">Description</div> |
| <div class="col-first even-row-color"><code> </code></div> |
| <div class="col-second even-row-color"><code><a href="../../formats/muc/MucNameContentHandler.html#%3Cinit%3E(opennlp.tools.tokenize.Tokenizer,java.util.List)" class="member-name-link">MucNameContentHandler</a><wbr>(<a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| <a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/List.html" title="class or interface in java.util" class="external-link">List</a><<a href="../../namefind/NameSample.html" title="class in opennlp.tools.namefind">NameSample</a>> storedSamples)</code></div> |
| <div class="col-last even-row-color"> |
| <div class="block">Initializes a <a href="../../formats/muc/MucNameContentHandler.html" title="class in opennlp.tools.formats.muc"><code>MucNameContentHandler</code></a>.</div> |
| </div> |
| <div class="col-first odd-row-color"><code>protected </code></div> |
| <div class="col-second odd-row-color"><code><a href="../../formats/muc/MucNameSampleStream.html#%3Cinit%3E(opennlp.tools.tokenize.Tokenizer,opennlp.tools.util.ObjectStream)" class="member-name-link">MucNameSampleStream</a><wbr>(<a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| <a href="../../util/ObjectStream.html" title="interface in opennlp.tools.util">ObjectStream</a><<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>> samples)</code></div> |
| <div class="col-last odd-row-color"> |
| <div class="block">Initializes a <a href="../../formats/muc/MucNameSampleStream.html" title="class in opennlp.tools.formats.muc"><code>MucNameSampleStream</code></a>.</div> |
| </div> |
| </div> |
| </section> |
| </li> |
| <li> |
| <section class="detail" id="opennlp.tools.tokenize"> |
| <h2>Uses of <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> in <a href="../package-summary.html">opennlp.tools.tokenize</a></h2> |
| <div class="caption"><span>Classes in <a href="../package-summary.html">opennlp.tools.tokenize</a> that implement <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a></span></div> |
| <div class="summary-table three-column-summary"> |
| <div class="table-header col-first">Modifier and Type</div> |
| <div class="table-header col-second">Class</div> |
| <div class="table-header col-last">Description</div> |
| <div class="col-first even-row-color"><code>class </code></div> |
| <div class="col-second even-row-color"><code><a href="../SimpleTokenizer.html" class="type-name-link" title="class in opennlp.tools.tokenize">SimpleTokenizer</a></code></div> |
| <div class="col-last even-row-color"> |
| <div class="block">A basic <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize"><code>Tokenizer</code></a> implementation which performs tokenization |
| using character classes.</div> |
| </div> |
| <div class="col-first odd-row-color"><code>class </code></div> |
| <div class="col-second odd-row-color"><code><a href="../TokenizerME.html" class="type-name-link" title="class in opennlp.tools.tokenize">TokenizerME</a></code></div> |
| <div class="col-last odd-row-color"> |
| <div class="block">A <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize"><code>Tokenizer</code></a> for converting raw text into separated tokens.</div> |
| </div> |
| <div class="col-first even-row-color"><code>class </code></div> |
| <div class="col-second even-row-color"><code><a href="../WhitespaceTokenizer.html" class="type-name-link" title="class in opennlp.tools.tokenize">WhitespaceTokenizer</a></code></div> |
| <div class="col-last even-row-color"> |
| <div class="block">A basic <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize"><code>Tokenizer</code></a> implementation which performs tokenization |
| using white spaces.</div> |
| </div> |
| <div class="col-first odd-row-color"><code>class </code></div> |
| <div class="col-second odd-row-color"><code><a href="../WordpieceTokenizer.html" class="type-name-link" title="class in opennlp.tools.tokenize">WordpieceTokenizer</a></code></div> |
| <div class="col-last odd-row-color"> |
| <div class="block">A <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize"><code>Tokenizer</code></a> implementation which performs tokenization |
| using word pieces.</div> |
| </div> |
| </div> |
| <div class="caption"><span>Constructors in <a href="../package-summary.html">opennlp.tools.tokenize</a> with parameters of type <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a></span></div> |
| <div class="summary-table three-column-summary"> |
| <div class="table-header col-first">Modifier</div> |
| <div class="table-header col-second">Constructor</div> |
| <div class="table-header col-last">Description</div> |
| <div class="col-first even-row-color"><code> </code></div> |
| <div class="col-second even-row-color"><code><a href="../TokenizerEvaluator.html#%3Cinit%3E(opennlp.tools.tokenize.Tokenizer,opennlp.tools.tokenize.TokenizerEvaluationMonitor...)" class="member-name-link">TokenizerEvaluator</a><wbr>(<a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| <a href="../TokenizerEvaluationMonitor.html" title="interface in opennlp.tools.tokenize">TokenizerEvaluationMonitor</a>... listeners)</code></div> |
| <div class="col-last even-row-color"> |
| <div class="block">Initializes an instance to evaluate a <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize"><code>Tokenizer</code></a>.</div> |
| </div> |
| <div class="col-first odd-row-color"><code> </code></div> |
| <div class="col-second odd-row-color"><code><a href="../TokenizerStream.html#%3Cinit%3E(opennlp.tools.tokenize.Tokenizer,opennlp.tools.util.ObjectStream)" class="member-name-link">TokenizerStream</a><wbr>(<a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> tokenizer, |
| <a href="../../util/ObjectStream.html" title="interface in opennlp.tools.util">ObjectStream</a><<a href="https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html" title="class or interface in java.lang" class="external-link">String</a>> input)</code></div> |
| <div class="col-last odd-row-color"> |
| <div class="block">Initializes a <a href="../TokenizerStream.html" title="class in opennlp.tools.tokenize"><code>instance</code></a>.</div> |
| </div> |
| </div> |
| </section> |
| </li> |
| <li> |
| <section class="detail" id="opennlp.tools.util.featuregen"> |
| <h2>Uses of <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> in <a href="../../util/featuregen/package-summary.html">opennlp.tools.util.featuregen</a></h2> |
| <div class="caption"><span>Constructors in <a href="../../util/featuregen/package-summary.html">opennlp.tools.util.featuregen</a> with parameters of type <a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a></span></div> |
| <div class="summary-table three-column-summary"> |
| <div class="table-header col-first">Modifier</div> |
| <div class="table-header col-second">Constructor</div> |
| <div class="table-header col-last">Description</div> |
| <div class="col-first even-row-color"><code> </code></div> |
| <div class="col-second even-row-color"><code><a href="../../util/featuregen/TokenPatternFeatureGenerator.html#%3Cinit%3E(opennlp.tools.tokenize.Tokenizer)" class="member-name-link">TokenPatternFeatureGenerator</a><wbr>(<a href="../Tokenizer.html" title="interface in opennlp.tools.tokenize">Tokenizer</a> supportTokenizer)</code></div> |
| <div class="col-last even-row-color"> |
| <div class="block">Initializes a <a href="../../util/featuregen/TokenPatternFeatureGenerator.html" title="class in opennlp.tools.util.featuregen"><code>TokenPatternFeatureGenerator</code></a> instance.</div> |
| </div> |
| </div> |
| </section> |
| </li> |
| </ul> |
| </section> |
| </main> |
| <footer role="contentinfo"> |
| <hr> |
| <p class="legal-copy"><small>Copyright © 2023 <a href="https://www.apache.org/">The Apache Software Foundation</a>. All rights reserved.</small></p> |
| </footer> |
| </div> |
| </div> |
| </body> |
| </html> |
