WIP.
diff --git a/_includes/left-side-menu.html b/_includes/left-side-menu.html
index 93907e6..40eec54 100644
--- a/_includes/left-side-menu.html
+++ b/_includes/left-side-menu.html
@@ -39,6 +39,13 @@
{% endif %}
</li>
<li>
+ {% if page.id == "semantic" %}
+ <a class="active" href="/semantic.html">Semantic model</a>
+ {% else %}
+ <a href="/semantic.html">Semantic model</a>
+ {% endif %}
+ </li>
+ <li>
{% if page.id == "built-components" %}
<a class="active" href="/custom-components.html">Custom components</a>
{% else %}
@@ -46,27 +53,6 @@
{% endif %}
</li>
<li>
- {% if page.id == "installation" %}
- <a class="active" href="/installation.html">Installation</a>
- {% else %}
- <a href="/installation.html">Installation</a>
- {% endif %}
- </li>
- <li>
- {% if page.id == "first_example" %}
- <a class="active" href="/first-example.html">First Example</a>
- {% else %}
- <a href="/first-example.html">First Example</a>
- {% endif %}
- </li>
- <li>
- {% if page.id == "data_model" %}
- <a class="active" href="/data-model.html">Data Model</a>
- {% else %}
- <a href="/data-model.html">Data Model</a>
- {% endif %}
- </li>
- <li>
{% if page.id == "intent_matching" %}
<a class="active" href="/intent-matching.html">Intent Matching</a>
{% else %}
@@ -80,6 +66,13 @@
<a href="/short-term-memory.html">Short-Term Memory</a>
{% endif %}
</li>
+ <li>
+ {% if page.id == "installation" %}
+ <a class="active" href="/installation.html">Installation</a>
+ {% else %}
+ <a href="/installation.html">Installation</a>
+ {% endif %}
+ </li>
<li class="side-nav-title">Examples</li>
<li>
{% if page.id == "calculator" %}
diff --git a/api-review.html b/api-review.html
index 2138b8f..82407ad 100644
--- a/api-review.html
+++ b/api-review.html
@@ -76,29 +76,29 @@
</ul>
<p>
- Base client methods:
+ <b>Base client methods:</b>
</p>
<ul>
<li>
- <code>ask</code> passes user text input to the model and receives back triggered callback method execution result or
+ <code>ask()</code> passes user text input to the model and receives back triggered callback method execution result or
rejection exception if there isn't any triggered intents.
</li>
<li>
- <code>debugAsk</code> passes user text input to the model and receives back callback and its parameters or
+ <code>debugAsk()</code> passes user text input to the model and receives back callback and its parameters or
rejection exception if there isn't any triggered intents.
Main difference from <code>ask</code> that triggered intent callback method is not called.
This method and this parameter can be useful for tests scenarios.
</li>
<li>
- <code>clearStm</code> clears STM state. Memory is cleared wholly or with some predicate.
+ <code>clearStm()</code> clears STM state. Memory is cleared wholly or with some predicate.
Loot at <a href="short-term-memory.html">Conversation</a> chapter to get more details.
</li>
<li>
- <code>clearDialog</code> clears dialog state. Dialog is cleared wholly or with some predicate.
+ <code>clearDialog()</code> clears dialog state. Dialog is cleared wholly or with some predicate.
Loot at <a href="short-term-memory.html">Conversation</a> chapter to get more details.
</li>
<li>
- <code>close</code> - Closes client. You can't call another client's methods after this method was closed.
+ <code>close()</code> - Closes client. You can't call another client's methods after this method was closed.
</li>
</ul>
</section>
@@ -142,20 +142,29 @@
<code>Token</code> represented as <a href="apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a>.
It is simple string, part of user input, which split according to some rules, for instance by spaces and some additional conditions, which depends on language and some expectations.
So user input "<b>Where is it?</b>" contains four tokens: "<b>Where</b>", "<b>is</b>", "<b>it</b>", "<b>?</b>".
+ Tokens data are input for searching <code>entities</code>.
</li>
<li>
<code>Entity</code> represented as <a href="apis/latest/org/apache/nlpcraft/NCEntity.html">NCEntity</a>.
- According to wikipedia, named entity is a real-world object, such as a person, location, organization, product, etc., that can be denoted with a proper name. It can be abstract or have a physical existence. Each entity can contain one or more tokens.
+ According to wikipedia, named entity is a real-world object, such as a person, location, organization,
+ product, etc., that can be denoted with a proper name. It can be abstract or have a physical existence.
+ Each entity can contain one or more tokens.
+ Entities data are input for searching <a href="intent-matching.html">Intent matching</a> conditions.
</li>
<li>
<code>Variant</code> represented as <a href="apis/latest/org/apache/nlpcraft/NCVariant.html">NCVariant</a>.
- List of entities. Potentially, each token can be recognized as different entities, so user input can be processed as set of variants. For example user input "Mercedes" can be processed as 2 variants, both of them contains single element list of entities: car brand or Spanish family name.
+ List of entities. Potentially, each token can be recognized as different entities,
+ so user input can be processed as set of variants.
+ For example user input "Mercedes" can be processed as 2 variants,
+ both of them contains single element list of entities: car brand or Spanish family name.
+ When words are not overlapped with different <code>entities</code> there is only one
+ <code>variant</code> detected.
</li>
</ul>
<p>
- Back to <code>Pipeline</code> represented as <a href="apis/latest/org/apache/nlpcraft/NCPipeline.html">pipeline</a>.
- Pipeline should be created based in following components:
+ Back to <code>pipeline</code> represented as <a href="apis/latest/org/apache/nlpcraft/NCPipeline.html">NCPipeline</a>.
+ <code>Pipeline</code> should be created based in following components:
</p>
<ul>
<li>
diff --git a/built-components.html b/built-components.html
index 055acb9..f162a8e 100644
--- a/built-components.html
+++ b/built-components.html
@@ -108,7 +108,7 @@
There are provided a number of built implementations, all of them for English language.
<ul>
<li>
- <a href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a> converts NLP tokens into entities with four mandatory properties:
+ <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCNLPEntityParser.html">NCNLPEntityParser</a> converts NLP tokens into entities with four mandatory properties:
<code>nlp:token:text</code>, <code>nlp:token:index</code>, <code>nlp:token:startCharIndex</code> and
<code>nlp:token:endCharIndex</code>. However, if any other properties were added into
processed tokens by <code>NCTokenEnricher</code> components, they will be copied also with names
@@ -119,7 +119,7 @@
<a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCOpenNLPTokenParser.html">NCOpenNLPTokenParser</a> is wrapper on <a href="https://opennlp.apache.org/">Apache OpenNLP</a> NER components.
Look at the supported <b>Name Finder</b> models <a href="https://opennlp.sourceforge.net/models-1.5/">here</a>.
For example for English language are accessible: <code>Location</code>, <code>Money</code>,
- <code>Person</code>, <code>Organizationon</code>, <code>Date</code>, <code>Time</code> and <code>Percentage</code>.
+ <code>Person</code>, <code>Organization</code>, <code>Date</code>, <code>Time</code> and <code>Percentage</code>.
</li>
<li>
<code>NCStanfordNLPEntityParser</code> is wrapper on <a href="https://nlp.stanford.edu/">Stanford NLP</a> NER components.
@@ -127,7 +127,8 @@
</li>
<li>
<a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a> is entity parser which is based on list of synonyms elements.
- This component is described with more details below in <a href="#semantic">Semantic enrichers</a> section.
+ It is very important component which allow to solve a wide range of tasks.
+ Separated chapter <a href="semantic.html">Semantic model</a> is dedicated to its detailed description.
</li>
</ul>
</li>
@@ -162,49 +163,6 @@
</ul>
</section>
- <section id="semantic">
- <h2 class="section-title">Semantic entity parser <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
-
- <p>
- Semantic entity parser <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a>
- is the implementation of <a href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
- which deserves a special mention.
- It describes list of <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
- which are represented <a href="https://en.wikipedia.org/wiki/Named-entity_recognition">Named entities</a>.
- </p>
-
- <p>
- Each <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
- is presented by <code>id</code>, <code>groups</code>, <code>synonyms</code>, <code>values</code> and <code>properties</code>.
- <p>
-
- <section id="element-synonyms">
- <h3>Element synonyms</h3>
- <p>TODO...</p>
- </section>
-
- <section id="element-values">
- <h3>Element values</h3>
- <p>TODO...</p>
- </section>
-
- <section id="element-groups">
- <h3>Element values</h3>
- <p>TODO...</p>
- </section>
-
- <section id="element-macros">
- <h3>Element macros</h3>
- <p>TODO...</p>
- </section>
-
- <section id="external-declaration">
- <h3>External JSON/YAML Declaration</h3>
- <p>TODO...</p>
- </section>
-
- </section>
-
<section id="examples">
<h2 class="section-title">Examples <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
diff --git a/custom-components.html b/custom-components.html
index 9e0030e..529acbd 100644
--- a/custom-components.html
+++ b/custom-components.html
@@ -90,10 +90,8 @@
You have to implement <a href="apis/latest/org/apache/nlpcraft/NCTokenEnricher.html">NCTokenEnricher</a> trait.
</p>
<p>
- <a href="apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a> are used in
- <a href="intent-matching.html">Intent matching</a>. NlpCraft provides a numeric of built token enricher
- implementations for English language.
- You may want to create your own or extends existing. Look at the following example:
+ Tokens enricher is component which allows to add additional properties to prepared tokens.
+ These tokens properties are used later when entities detection.
</p>
<pre class="brush: scala, highlight: [25, 26]">
import org.apache.nlpcraft.*
@@ -139,6 +137,27 @@
<p>
You have to implement <a href="apis/latest/org/apache/nlpcraft/NCTokenValidator.html">NCTokenValidator</a> trait.
</p>
+
+ <p>
+ There are tokens are inspected and exception can be thrown from user code to break user input processing.
+ </p>
+
+ <pre class="brush: scala, highlight: [3]">
+ new NCTokenValidator:
+ override def validate(req: NCRequest, cfg: NCModelConfig, toks: List[NCToken]): Unit =
+ if toks.exists(_.contains("restrictionFlag"))
+ then throw new NCException("Sentence cannot be processed.")
+ </pre>
+
+ <ul>
+ <li>
+ There is anonymous instance of <a href="apis/latest/org/apache/nlpcraft/NCTokenValidator.html">NCTokenValidator</a>
+ created.
+ </li>
+ <li>
+ <code>Lines 3</code> defines the rule when exception should be thrown and sentence processing should be stopped.
+ </li>
+ </ul>
</section>
<section id="entity-parser">
@@ -146,6 +165,37 @@
<p>
You have to implement <a href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a> trait.
</p>
+
+ <p>
+ Most important component which finds user specific data.
+ These defined entities are input for <a href="intent-matching.html">Intent matching</a> conditions.
+ If built <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a>
+ is not enough, you can implement your own NER searching here.
+ There is point for potential integrations with neural networks or any other solutions which
+ help you find and mark your domain specific named entities.
+ </p>
+
+ <pre class="brush: scala, highlight: [5]">
+ import org.apache.nlpcraft.*
+
+ class CommentsEntityParser extends NCEntityParser :
+ def parse(req: NCRequest, cfg: NCModelConfig, toks: List[NCToken]): List[NCEntity] =
+ if req.getText.trim.startsWith("--") then
+ List(
+ new NCPropertyMapAdapter with NCEntity :
+ override def getTokens: List[NCToken] = toks
+ override def getRequestId: String = req.getRequestId
+ override def getId: String = "comment"
+ )
+ else
+ List.empty
+ </pre>
+ <ul>
+ <li>
+ In given example whole input sentence is marked as single element <code>comment</code> if
+ condition defined on <code>line 5</code> is <code>true</code>.
+ </li>
+ </ul>
</section>
<section id="entity-enricher">
@@ -153,13 +203,85 @@
<p>
You have to implement <a href="apis/latest/org/apache/nlpcraft/NCEntityEnricher.html">NCEntityEnricher</a> trait.
</p>
+ <p>
+ Entity enricher is component which allows to add additional properties to prepared entities.
+ Can be useful for extending existing entity enrichers functionality.
+ </p>
+
+ <pre class="brush: scala, highlight: [4, 10, 11]">
+ import org.apache.nlpcraft.*
+
+ object CityPopulationEntityEnricher:
+ val citiesPopulation: Map[String, Int] = someExternalService.getCitiesPopulation()
+
+ import CityPopulationEntityEnricher.*
+
+ class CityPopulationEntityEnricher extends NCEntityEnricher :
+ def enrich(req: NCRequest, cfg: NCModelConfig, ents: List[NCEntity]): Unit =
+ ents.
+ filter(_.getId == "city").
+ foreach(e => e.put("city:population", citiesPopulation(e("city:name"))))
+ </pre>
+
+ <ul>
+ <li>
+ <code>Line 4</code> defines getting cities population data from some external service.
+ </li>
+ <li>
+ <code>Line 10</code> filters entities by <code>ID</code>.
+ </li>
+ <li>
+ <code>Line 11</code> enriches entities by new <code>city:population</code> property.
+ </li>
+ </ul>
</section>
<section id="entity-mapper">
- <h2 class="section-title">Entity enricher<a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+ <h2 class="section-title">Entity mapper<a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
<p>
You have to implement <a href="apis/latest/org/apache/nlpcraft/NCEntityMapper.html">NCEntityMapper</a> trait.
</p>
+
+ <p>
+ Entity mapper is component which allows to map one set of entities into another after the entities
+ were parsed and enriched. Can be useful for building complex parsers based on existing.
+ </p>
+
+ <pre class="brush: scala, highlight: [4, 10, 12, 13, 14]">
+ import org.apache.nlpcraft.*
+
+ object CityPopulationEntityMapper:
+ val citiesPopulation: Map[String, Int] = externalService.getCitiesPopulation()
+
+ import CityPopulationEntityMapper.*
+
+ class CityPopulationEntityMapper extends NCEntityMapper :
+ def map(req: NCRequest, cfg: NCModelConfig, ents: List[NCEntity]): List[NCEntity] =
+ val cities = ents.filter(_.getId == "city")
+
+ ents.filterNot(_.getId == "city") ++
+ cities ++
+ cities.filter(city => citiesPopulation(city("city:name")) > 1000000).
+ map(city =>
+ new NCPropertyMapAdapter with NCEntity :
+ override def getTokens: List[NCToken] = city.getTokens
+ override def getRequestId: String = req.getRequestId
+ override def getId: String = "big-city"
+ )
+ </pre>
+ <ul>
+ <li>
+ <code>Line 4</code> defines getting cities population data from some external service.
+ </li>
+ <li>
+ <code>Line 10</code> filters entities by <code>ID</code>.
+ </li>
+ <li>
+ <code>Line 12, 13 and 14</code> define component result entities set.
+ It contains previously defined <code>city</code> elements, new elements <code>big-city</code> and
+ another not city elements.
+ </li>
+ </ul>
</section>
<section id="entity-validator">
@@ -167,6 +289,27 @@
<p>
You have to implement <a href="apis/latest/org/apache/nlpcraft/NCEntityValidator.html">NCEntityValidator</a> trait.
</p>
+ <p>
+ Entities validator is user defined component, where prepared entities are inspected and exceptions
+ can be thrown from user code to break user input processing.
+ </p>
+
+ <pre class="brush: scala, highlight: [3]">
+ new NCEntityValidator :
+ override def validate(req: NCRequest, cfg: NCModelConfig, ents: List[NCEntity]): Unit =
+ if ents.exists(_.getId == "restrictedID")
+ then throw new NCException("Sentence cannot be processed.")
+ </pre>
+
+ <ul>
+ <li>
+ There is anonymous instance of <a href="apis/latest/org/apache/nlpcraft/NCEntityValidator.html">NCEntityValidator</a>
+ created.
+ </li>
+ <li>
+ <code>Lines 3</code> defines the rule when exception should be thrown and sentence processing should be stopped.
+ </li>
+ </ul>
</section>
<section id="variant-filter">
@@ -174,6 +317,27 @@
<p>
You have to implement <a href="apis/latest/org/apache/nlpcraft/NCVariantFilter.html">NCVariantFilter</a> trait.
</p>
+
+ <p>
+ Component which allows filtering detected variants, rejecting undesirable.
+ </p>
+
+ <pre class="brush: scala, highlight: [3]">
+ new NCVariantFilter :
+ def filter(req: NCRequest, cfg: NCModelConfig, vars: List[NCVariant]): List[NCVariant] =
+ vars.filter(_.getEntities.exists(_.getId == "requiredID"))
+ </pre>
+
+ <ul>
+ <li>
+ There is anonymous instance of <a href="apis/latest/org/apache/nlpcraft/NCVariantFilter.html">NCVariantFilter</a>
+ created.
+ </li>
+ <li>
+ <code>Lines 3</code> defines variant's filter,
+ it passed only variants which contain <code>requiredID</code> elements.
+ </li>
+ </ul>
</section>
</div>
diff --git a/examples/calculator.html b/examples/calculator.html
index 16d7583..081f55c 100644
--- a/examples/calculator.html
+++ b/examples/calculator.html
@@ -252,7 +252,6 @@
<li><a href="#overview">Overview</a></li>
<li><a href="#new_project">New Project</a></li>
<li><a href="#model">Data Model</a></li>
- <li><a href="#code">Model Class</a></li>
<li><a href="#testing">Testing</a></li>
{% include quick-links.html %}
</ul>
diff --git a/installation.html b/installation.html
index 2131048..27d1576 100644
--- a/installation.html
+++ b/installation.html
@@ -22,32 +22,21 @@
-->
<div id="installation" class="col-md-8 second-column">
- <section id="tldr">
- <h2 class="section-title">TL;DR; <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
- <p>
- Watch this quick video (6:17) on how to download, install and verify your installation:
- </p>
- <div>
- <iframe
- width="514"
- height="289"
- src="https://www.youtube.com/embed/s4zEVJVhW68?modestbranding=1"
- title="NLPCraft - Download, Install & Verify"
- frameborder="0"
- allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
- allowfullscreen>
- </iframe>
- </div>
- </section>
<section id="install">
<h2 class="section-title">Installation <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
<p>
Apache NLPCraft requires <b>no special installation</b> operations:
</p>
<ul>
- <li>If you <a href="/download.html">downloaded</a> binary ZIP archive - <a target="asf" href="https://www.apache.org/dyn/closer.cgi#verify">verify</a> it and unzip it</li>
- <li>If you <a href="/download.html">cloned</a> GitHub repository or downloaded source archive - use <code>mvn clean package -P examples</code> to build it</li>
- <li>If you <a href="/download.html">used</a> Maven/Gradle/SBT - you know what you are doing...</li>
+ <li>If you <a href="/download.html">downloaded</a> binary ZIP archive -
+ <a target="asf" href="https://www.apache.org/dyn/closer.cgi#verify">verify</a> it and unzip it.
+ </li>
+ <li>If you <a href="/download.html">cloned</a> GitHub repository or downloaded source archive -
+ <code>sbt clean compile</code> to build it.
+ </li>
+ <li>Also you can use <code>maven</code> command <code>mvn -B clean package -P stanford-core,examples</code>
+ and get same result.
+ </li>
</ul>
<p>
You are done 👍
@@ -57,68 +46,55 @@
<b>Minimal Java and Scala versions</b>
</p>
<p>
+ Starting with version <b>1.0.0</b> NLPCraft requires JDK 11+ and Scala 3.1.3.<br/>
Starting with version <b>0.9.0</b> NLPCraft requires JDK 11+ and Scala 2.13.<br/>
Starting with version <b>0.7.0</b> NLPCraft requires JDK 11+ and Scala 2.12.<br/>
Prior versions supported JDK 8 and Scala 2.12.
</p>
</div>
</section>
- <section id="clean">
- <h2 class="section-title">Reset Database <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+ <section id="new_project">
+ <h2 class="section-title">Create New Project <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
<p>
- For releases marked with <i class="fas fa-fw fa-bomb"></i> you need to reset the existing database. If you are
- using the Apache Ignite storage (default configuration for NLPCraft) you can accomplish this by
- removing <code>${USER_HOME}/.nlpcraft/ignite</code> folder.
+ You can create new Scala projects in many ways - we'll use SBT
+ to accomplish this task. Make sure that <code>build.sbt</code> file has the following content:
</p>
+ <pre class="brush: js, highlight: []">
+ ThisBuild / version := "0.1.0-SNAPSHOT"
+ ThisBuild / scalaVersion := "3.1.3"
+ lazy val root = (project in file("."))
+ .settings(
+ name := "Your project",
+ version := "{{site.latest_version}}",
+ libraryDependencies += "org.apache.nlpcraft" % "nlpcraft" % "{{site.latest_version}}",
+ libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.14" % "test"
+ )
+ </pre>
+
+ <ul>
+ <li>
+ Now you can use <code>NlpCraft</code> library in <code>Your project</code>.
+ </li>
+ <li>
+ You can use <code>Sbt</code> or <code>Maven</code> systems depending on your requirements via links
+ to <a href="https://search.maven.org/">Maven Central</a>.
+ </li>
+ <li>
+ You also can build <code>NlpCraft</code> library soource code and use references on jar files via <code>Ant</code>
+ or any another way.
+ </li>
+ </ul>
+
<p>
- If you are using other storage engines (e.g. Postgres or MySQL) you need to follow their specific
- procedures to recreate the database using SQL scripts from <code>sql</code> sub-folder in NLPCraft
- installation directory.
+ Go to Examples section to start work with concrete example.
</p>
</section>
- <section id="config">
- <h2 class="section-title">External Configuration <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
- <p>
- Due to <a target="asf" href="https://apache.org/legal/release-policy.html">Apache release policy</a> the official Apache release cannot include content that is based on
- <a target=_ href="https://creativecommons.org/licenses/by/4.0/">Commons Attribution 4.0 International</a> (CC BY 4.0) license
- or any binary artifacts regardless of their respective licences. Due to these restrictions the official Apache NLPCraft release
- does not include certain configuration files that are necessary for its operation. These files will be auto-downloaded upon first star of NLPCraft (hence the internet
- connection is required for the first start).
- On subsequent starts the NLPCraft will check for the updates and will re-download these files only if they were changed.
- </p>
- <p>
- By default, this external configuration is hosted on GitHub (in <code>external_config</code> branch of NLPCraft project) from where it will
- be automatically downloaded on the first start: <a target="github" href="https://github.com/apache/incubator-nlpcraft/raw/external_config/external">https://github.com/apache/incubator-nlpcraft/raw/external_config/external</a>.
- You can change this location, e.g. move the external configuration behind your firewall or onto your local file system,
- as well as change other parameters related to auto-download of the external <a href="/server-and-probe.html#config">configuration</a>
- in <code>*.conf</code> file (see <code>nlpcraft.extConfig</code> property).
- </p>
- </section>
- <section id="ignite">
- <h2 class="section-title">Apache Ignite 2.x <span class="amp">&</span> Java 11+ <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
- <p>
- Starting with NLPCraft 0.7.0, NLPCraft requires JDK 11 as its minimum JDK version.
- NLPCraft uses <a target=_ href="https://ignite.apache.org/">Apache Ignite 2.x</a> project as its
- underlying in-memory distributed storage and cluster computing framework. When running Apache Ignite 2.x
- on Java 11 you need to follow these
- <a target=_ href="https://apacheignite.readme.io/docs/getting-started#section-running-ignite-with-java-9-10-11">instructions</a> which
- are automatically followed when you use <a href="/tools/script.html"><code>nlpcraft.{sh|cmd}</code></a> script.
- </p>
- <div class="bq info">
- <p>
- <b>NOTE:</b> It is NOT recommended to change the version of Apache Ignite inside NLPCraft project.
- </p>
- </div>
- <br/>
- </section>
-</div>
+ </div>
<div class="col-md-2 third-column">
<ul class="side-nav">
<li class="side-nav-title">On This Page</li>
<li><a href="#install">Installation</a></li>
- <li><a href="#clean">Reset Database</a></li>
- <li><a href="#config">External Config</a></li>
- <li><a href="#ignite">Apache Ignite</a></li>
+ <li><a href="#new_project">New Project</a></li>
{% include quick-links.html %}
</ul>
</div>
diff --git a/semantic.html b/semantic.html
new file mode 100644
index 0000000..bfde0c5
--- /dev/null
+++ b/semantic.html
@@ -0,0 +1,627 @@
+---
+active_crumb: Docs
+layout: documentation
+id: semantic
+---
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<div class="col-md-8 second-column">
+ <section id="overview">
+ <h2 class="section-title">Semantic model<a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+
+ <p>
+ Semantic entity parser <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a>
+ is the implementation of <a href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
+ which deserves a special mention.
+ It defines list of <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
+ which are represent <a href="https://en.wikipedia.org/wiki/Named-entity_recognition">Named entities</a>.
+ We will name this list as <code>Semantic Model</code>.
+
+ Let's talk a little bit more about <a href="https://en.wikipedia.org/wiki/Named-entity_recognition">Named entities</a>.
+ </p>
+ </section>
+
+ <section id="ne">
+ <h2 class="section-title">Named Entities<a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+ <p>
+ Named entity, also known as a semantic element or a token, is one of the main a components defined by the NLPCraft data model.
+ A named entity is one or more individual words that have a consistent semantic meaning and typically denote a
+ real-world object, such as persons, locations, number, date and time, organizations, products, etc. Such
+ object can be abstract or have a physical existence.
+ </p>
+ <p>
+ For example, in the following sentence: TODO: PIC
+ </p>
+ <figure>
+ <img alt="named entities" class="img-fluid" src="/images/named-entities.png">
+ <figcaption><b>Fig 2.</b> Named Entities</figcaption>
+ </figure>
+ <p>
+ the following named entities can be detected:
+ </p>
+ <table class="gradient-table">
+ <thead>
+ <tr>
+ <th>Words</th>
+ <th>Type</th>
+ <th>Normalized Value</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><b>Top 20</b></td>
+ <td><code>user:element:1</code></td>
+ <td>top 20</td>
+ </tr>
+ <tr>
+ <td><b>best pages</b></td>
+ <td><code>user:element:2</code></td>
+ <td>best pages</td>
+ </tr>
+ <tr>
+ <td><b>California USA</b></td>
+ <td><code>stanford:city</code></td>
+ <td>USA, California</td>
+ </tr>
+ <tr>
+ <td><b>last 3 months</b></td>
+ <td><code>stanford:date</code></td>
+ <td>1/1/2021 - 4/1/2021</td>
+ </tr>
+ </tbody>
+ </table>
+ <p>
+ In most cases named entities will have associated <em>normalized value</em>. It is especially important for named entities that have many
+ notational forms such as time and date, currency, geographical locations, etc. For example, <code>New York</code>,
+ <code>New York City</code> and <code>NYC</code> all refer to the same "New York City, NY USA" location which is a standard normalized form.
+ </p>
+ <p>
+ The process of detecting named entities is called Named Entity Recognition (NER). There are many ways of how a certain named entity can be detected: through list of synonyms, by name, rule-based or by using
+ statistical techniques like neural networks with large corpus of predefined data. NLPCraft natively supports synonym-based
+ named entities definition as well as the ability to compose new named entities through powerful <a href="/intent-matching.html">Intent Definition Language</a> (IDL)
+ combining other named entities including named entities from
+ such OpenNLP, or Stanford CoreNLP, look at the <a href="built-components.html">Built component</a> chapter.
+ </p>
+ <p>
+ Named entities allow you to abstract from basic linguistic forms like nouns and verbs to deal with the higher level semantic
+ abstractions like geographical location or time when you are trying to understand the meaning of the sentence.
+ One of the main goals of named entities is to act as an input ingredients for <a href="/intent-matching.html">intent matching</a>.
+ </p>
+ <div class="bq info">
+ <p>
+ <b>😀 User Input → Named Entities → Parsing Variants → Intent Matcher → Winning Intent 🚀</b>
+ </p>
+ <p>
+ User input is parsed into the list of named entities. That list is then further transformed into one or more
+ parsing variants where each variant represents a particular order and combination of detected named entities.
+ Finally, the list of variants act as an input to intent matching where each variant is matched against every intent
+ in the process of detecting the best matching intent for the original user input.
+ </p>
+ </div>
+ </section>
+
+ <section id="elements">
+ <h2 class="section-title">Semantic elements<a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+ <p>
+ <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a> represents
+ NER element for its detection un the user input.
+ <p>
+
+ <div class="bq info">
+ <p>
+ <b>Semantic Element <span class="amp">&</span> Named Entity <span class="amp">&</span> Token</b>
+ </p>
+ <p>
+ Terms 'semantic element', 'named entity' and 'token' are used throughout this documentation relatively interchangeably:
+ </p>
+ <dl>
+ <dt>Semantic Element</dt>
+ <dd>
+ Denotes a named entity <em>declared</em> in NLPCraft model.
+ </dd>
+ <dt>Token</dt>
+ <dd>
+ Denotes a semantic element that was <em>detected</em> by NLPCraft in the user input.
+ </dd>
+ <dt>Named Entity</dt>
+ <dd>
+ Denotes a classic term, i.e. one or more individual words that have a
+ consistent semantic meaning and typically define a real-world object.
+ </dd>
+ </dl>
+ </div>
+
+ <p>
+ Each <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
+ is presented by <code>id</code>, <code>groups</code>, <code>synonyms</code>, <code>values</code> and <code>properties</code>.
+ <p>
+ <span id="synonyms" class="section-sub-title">Synonyms <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ NLPCraft uses fully deterministic named entity recognition and is not based on statistical approaches that
+ would require pre-existing marked up data sets and extensive training. For each semantic element you can either provide a
+ set of synonyms to match on or specify a piece of code that would be responsible for detecting that named
+ entity (discussed below). A synonym can have one or more individual words. Note that element's ID is its
+ implicit synonym so that even if no additional synonyms are defined at least one synonym always exists. Note
+ also that synonym matching is performed on <em>normalized</em> and <em>stemmatized</em> forms of both
+ a synonym and user input.
+ </p>
+
+ <p>
+ Here's an example of a simple semantic element definition in JSON:
+ </p>
+ <pre class="brush: js, highlight: [6,7,8,9,10,11,12]">
+ ...
+ "elements": [
+ {
+ "id": "transport.vehicle",
+ "description": "Transportation vehicle",
+ "synonyms": [
+ "car",
+ "truck",
+ "light duty truck"
+ "heavy duty truck"
+ "sedan",
+ "coupe"
+ ]
+ }
+ ]
+ ...
+ </pre>
+ <p>
+ While adding multi-word synonyms looks somewhat
+ trivial - in real models, the naive approach can lead to thousands and even tens of thousands of
+ possible synonyms due to words, grammar, and linguistic permutations - which quickly becomes untenable if
+ performed manually.
+ </p>
+ <p>
+ NLPCraft provides an effective tool for a compact synonyms representation. Instead of listing all possible
+ multi-word synonyms one by one you can use combination of following techniques:
+ </p>
+ <ul>
+ <li><a href="#macros">Macros</a></li>
+ <li><a href="#regex">Regular expressions</a></li>
+ <li><a href="#option-groups">Option Groups</a></li>
+ </ul>
+ <p>
+ Each whitespace separated string in the synonym can be either a regular word (like in the above transportation example
+ where it will be matched on using its normalized and stemmatized form) or one of the above expression.
+ </p>
+ <p>
+ Note that this synonyms definition is also used in the following
+ <a href="/apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a> methods:
+ </p>
+ <ul>
+ <li><code>getSynonyms()</code> - gets synonyms to match on.</li>
+ <li><code>getValues()</code> - get values to match on (see <a href="#values">below</a>).</li>
+ </ul>
+ <span id="values" class="section-sub-title">Element Values <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Semantic element can have an optional set of special synonyms called <em>values</em> or "proper nouns" for this element.
+ Unlike basic synonyms, each value is a pair of a name and a set of standard synonyms by which that value,
+ and ultimately its element, can be recognized in the user input. Note that the value name itself acts as an
+ implicit synonym even when no additional synonyms added for that value.
+ </p>
+ <p>
+ When a semantic element is recognized it is made available to the model's matching logic as an instance of
+ the <a href="/apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a> interface.
+ This interface has a method
+ <a href="/apis/latest/org/apache/nlpcraft/NCToken.html">getValue()</a> which
+ returns the name of the value, if any, by which
+ that semantic element was recognized. That value name can be further used in intent matching.
+ </p>
+ <p>
+ To understand the importance of the values consider the following changes to our transportation
+ example model:
+ </p>
+ <pre class="brush: js, highlight: [19,20,21,22,23,24,25,26,27,28,29,30]">
+ ...
+ "macros": [
+ {
+ "name": "<TRUCK_TYPE>",
+ "macro": "{light duty|heavy duty|half ton|1/2 ton|3/4 ton|one ton|super duty}"
+ }
+ ]
+ "elements": [
+ {
+ "id": "transport.vehicle",
+ "description": "Transportation vehicle",
+ "synonyms": [
+ "car",
+ "{<TRUCK_TYPE>|_} {pickup|_} truck"
+ "sedan",
+ "coupe"
+ ],
+ "values": [
+ {
+ "value": "mercedes",
+ "synonyms": ["mercedes-ben{z|s}", "mb", "ben{z|s}"]
+ },
+ {
+ "value": "bmw",
+ "synonyms": ["{bimmer|bimer|beemer}", "bayerische motoren werke"]
+ }
+ {
+ "value": "chevrolet",
+ "synonyms": ["chevy"]
+ }
+ ]
+ }
+ ]
+ ...
+ </pre>
+ <p>
+ With that setup <code>transport.vehicle</code> element will be recognized by any of the following input string:
+ </p>
+ <ul>
+ <li><code>car</code></li>
+ <li><code>benz</code> (with value <code>mercedes</code>)</li>
+ <li><code>3/4 ton pickup truck</code></li>
+ <li><code>light duty truck</code></li>
+ <li><code>chevy</code> (with value <code>chevrolet</code>)</li>
+ <li><code>bimmer</code> (with value <code>bmw</code>)</li>
+ <li><code>transport.vehicle</code></li>
+ </ul>
+ <span id="groups" class="section-sub-title">Element Groups <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Each semantic element always belongs to one or more groups. Semantic element provides its groups via
+ <a href="/apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">getGroups()</a> method.
+ By default, if element group is not specified, the element ID will act as its default group ID.
+ Group membership is a quick and easy way to organise similar semantic elements together and use this
+ categorization in <a href="/intent-matching.html">IDL</a> intents.
+ </p>
+ <p>
+ Note that the proper grouping of the elements is also necessary for the correct operation of
+ Short-Term-Memory (STM) in the conversational context. Consider a
+ <a href="/apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a> that
+ represents a previously found semantic element that is stored in the conversation. Such token
+ will be overridden in the conversation by the more <b>recent token</b>
+ from the <b>same group</b> - a critical rule of maintaining the proper conversational context.
+ See
+ <a href="/apis/latest/org/apache/nlpcraft/NCConversation.html">NCConversation</a>
+ for mode details.
+ </p>
+ </section>
+
+ <section id="macros">
+ <h2 class="section-title">Macros <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+ <p>
+ Listing all possible multi-word synonyms for a given element can be a time-consuming task. Macros
+ together with option groups allow for significant simplification of this task.
+ Macros allow you to give a name to an often used set of words or option groups and reuse it without
+ repeating those words or option groups again and again. A model provides a list of macros via
+ <a href="/apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">macros</a> method.
+ Each macro has a name in a form of <code><X></code> where <code>X</code>
+ is any string, and a string value. Note that macros can be nested (but not recursive), i.e. macro value can include
+ references to other macros. When macro name <code>X</code> is encountered in the synonym it gets recursively
+ replaced with its value.
+ </p>
+ <p>
+ Here's a code snippet of macro definitions using JSON definition:
+ </p>
+ <pre class="brush: js">
+ "macros": [
+ {
+ "name": "<A>",
+ "macro": "aaa"
+ },
+ {
+ "name": "<B>",
+ "macro": "<A> bbb"
+ },
+ {
+ "name": "<C>",
+ "macro": "<A> bbb {z|w}"
+ }
+ ]
+ </pre>
+ <span id="option-groups" class="section-sub-title">Option Groups <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Option groups are similar to wildcard patterns that operates on a single word base. One line of
+ option group expands into one or more individual synonyms. Option groups is the key mechanism for shortened
+ synonyms notation. The following examples demonstrate how to use option groups.
+ </p>
+ <p>
+ Consider the following macros defined below (note that macros <code><B></code> and <code><C></code>
+ are nested):
+ </p>
+ <table class="gradient-table">
+ <thead>
+ <tr>
+ <th>Name</th>
+ <th>Value</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code><A></code></td>
+ <td><code>aaa</code></td>
+ </tr>
+ <tr>
+ <td><code><B></code></td>
+ <td><code><A> bbb</code></td>
+ </tr>
+ <tr>
+ <td><code><C></code></td>
+ <td><code><A> bbb {z|w}</code></td>
+ </tr>
+ </tbody>
+ </table>
+ <p>
+ Then the following option group expansions will occur in these examples:
+ </p>
+ <table class="gradient-table">
+ <thead>
+ <tr>
+ <th>Synonym</th>
+ <th>Synonym Expansions</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code><A> {b|_} c</code></td>
+ <td>
+ <code>"aaa b c"</code><br>
+ <code>"aaa c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code><A> {b|a}[1,2] c</code></td>
+ <td>
+ <code>"aaa b c"</code><br>
+ <code>"aaa b b c"</code><br>
+ <code>"aaa a c"</code><br>
+ <code>"aaa a a c"</code><br>
+ <code>"aaa c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td>
+ <code><B> {b|_} c</code><br>
+ or<br>
+ <code><B> {b}[0,1] c</code>
+ </td>
+ <td>
+ <code>"aaa bbb b c"</code><br>
+ <code>"aaa bbb c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>{b|\{\_\}}</code></td>
+ <td>
+ <code>"b"</code><br>
+ <code>"b {_}"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>a {b|_}. c</code></td>
+ <td>
+ <code>"a b. c"</code><br>
+ <code>"a . c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>a .{b, |_}. c</code></td>
+ <td>
+ <code>"a .b, . c"</code><br>
+ <code>"a .. c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>
+ {% raw %}a {{b|c}|_}.{% endraw %}</code></td>
+ <td>
+ <code>"a ."</code><br>
+ <code>"a b."</code><br>
+ <code>"a c."</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>a {% raw %}{{{<C>}}|{_}}{% endraw %} c</code></td>
+ <td>
+ <code>"a aaa bbb z c"</code><br>
+ <code>"a aaa bbb w c"</code><br>
+ <code>"a c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>{% raw %}{{{a}}} {b||_|{{_}}||_}{% endraw %}</code></td>
+ <td>
+ <code>"a b"</code><br>
+ <code>"a"</code>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+ <p>
+ Specifically:
+ </p>
+ <ul>
+ <li><code>{A|B}</code> denotes either <code>A</code> or <code>B</code>.</li>
+ <li>
+ <code>{A|B|_}</code> denotes either <code>A</code> or <code>B</code> or nothing.
+ <ul>
+ <li>Symbol <code>_</code> cam appear anywhere in the list of options, i.e. <code>{A|B|_}</code> is equal to <code>{A|_|B}</code>.</li>
+ </ul>
+ </li>
+ <li>
+ <code>{C}[x,y]</code> denotes an option group with quantifier, i.e. group <code>C</code> appearing from <code>x</code> to <code>y</code> times inclusive.
+ <ul>
+ <li>For example, <code>{C}[1,3]</code> is the same as <code>{C|C C|C C C}</code> notation.</li>
+ <li>Note that <code>{C|_}</code> is equal to <code>{C}[0,1]</code></li>
+ </ul>
+ </li>
+ <li>Excessive curly brackets are ignored, when safe to do so.</li>
+ <li>Macros cannot be recursive but can be nested.</li>
+ <li>Option groups can be nested.</li>
+ <li>
+ <code>'\'</code> (backslash) can be used to escape <code>'{'</code>, <code>'}'</code>, <code>'|'</code> and
+ <code>'_'</code> special symbols used by the option groups.
+ </li>
+ <li>Excessive whitespaces are trimmed when expanding option groups.</li>
+ </ul>
+ <p>
+ We can rewrite our transportation semantic element in a more efficient way using macros and option groups.
+ Even though the actual length of definition hasn't changed much it now auto-generates many dozens of synonyms
+ we would have to write out manually otherwise:
+ </p>
+ <pre class="brush: js, highlight: [4,5,14]">
+ ...
+ "macros": [
+ {
+ "name": "<TRUCK_TYPE>",
+ "macro": "{ {light|super|heavy|medium} duty|half ton|1/2 ton|3/4 ton|one ton}"
+ }
+ ]
+ "elements": [
+ {
+ "id": "transport.vehicle",
+ "description": "Transportation vehicle",
+ "synonyms": [
+ "car",
+ "{<TRUCK_TYPE>|_} {pickup|_} truck"
+ "sedan",
+ "coupe"
+ ]
+ }
+ ]
+ ...
+ </pre>
+ <span id="regex" class="section-sub-title">Regular Expressions <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Any individual synonym word that starts and ends with <code>//</code> (two forward slashes) is
+ considered to be Java regular expression as defined in <code>java.util.regex.Pattern</code>. Note that
+ regular expression can only span a single word, i.e. only individual words from the user input will be
+ matched against given regular expression and no whitespaces are allowed within regular expression. Note
+ also that option group special symbols <code>{</code>, <code>}</code>,
+ <code>|</code> and <code>_</code> have to be escaped in the regular expression using <code>\</code>
+ (backslash).
+ </p>
+ <p>
+ For example, the following synonym:
+ </p>
+ <pre class="brush: js">
+ "synonyms": [
+ "{foo|//[bar].+//}}"
+ ]
+ </pre>
+ <p>
+ will match word <code>foo</code> or any other strings that start with <code>bar</code> as long as
+ this string doesn't contain whitespaces.
+ </p>
+ <div class="bq info">
+ <b>Regular Expressions Performance</b>
+ <p>
+ It's important to note that regular expressions can significantly affect the performance of the
+ NLPCraft processing if used uncontrolled. Use it with caution and test the performance
+ of your model to ensure it meets your requirements.
+ </p>
+ </div>
+ </section>
+
+ <section id="usage">
+ <h2 class="section-title">Usage examples<a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+
+ <p>
+ The following examples shows how to build model programmatically.
+ </p>
+
+ <pre class="brush: scala, highlight: [3, 5, 10]">
+ val mdl = new NCModelAdapter(
+ NCModelConfig("test.id", "Test Model", "1.0"),
+ new NCPipelineBuilder().withSemantic(
+ "en",
+ Map(
+ "<OF>" -> "{of|for|per}",
+ "<CUR>" -> "current|present|now|local}",
+ "<TIME>" -> "{time <OF> day|day time|date|time|moment|datetime|hour|o'clock|clock|date time|date and time|time and date}",
+ )
+ List(
+ new NCSemanticElement():
+ override def getId: String = "time"
+ override def getSynonyms: Set[String] = Set("{<CUR>|_} <TIME>", "what <TIME> {is it now|now|is it|_}" )
+ )
+ ).build
+ )
+ </pre>
+ <ul>
+ <li>
+ <code>Line 5</code> shows <code>macro</code> parameter usage.
+ </li>
+ <li>
+ <code>Line 10</code> shows <code>macro</code> list of <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a> parameter usage.
+ </li>
+ <li>
+ Note that usage <code>withSemantic()</code> method which represented on <code>line 3</code> i optional.
+ You can add <a href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCNLPEntityParser.html">NCNLPEntityParser</a>
+ as usual <a href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
+ when you define your <a href="apis/latest/org/apache/nlpcraft/NCPipeline.html">NCPipeline</a>.
+ </li>
+ </ul>
+
+ <p>
+ The following examples is based on YAML semantic elements representation.
+ </p>
+
+ <pre class="brush: js, highlight: []">
+ macros:
+ "<OF>": "{of|for|per}"
+ "<CUR>": "{current|present|now|local}"
+ "<TIME>": "{time <OF> day|day time|date|time|moment|datetime|hour|o'clock|clock|date time|date and time|time and date}"
+ elements:
+ - id: "x:time"
+ description: "Date and/or time token indicator."
+ synonyms:
+ - "{<CUR>|_} <TIME>"
+ - "what <TIME> {is it now|now|is it|_}"
+ </pre>
+
+ <ul>
+ <li>
+ Same macros and the same element as in previous example are defined here in
+ <code>time_model.yaml</code> YAML file.
+ </li>
+ </ul>
+ <pre class="brush: scala, highlight: [3]">
+ val mdl = new NCModelAdapter(
+ NCModelConfig("test.id", "Test Model", "1.0"),
+ new NCPipelineBuilder().withSemantic("en", "time_model.yaml").build
+ )
+ </pre>
+ <ul>
+ <li>
+ <code>Line 3</code> makes semantic model which defined in <code>time_model.yaml</code> YAML file.
+ </li>
+ </ul>
+ </section>
+</div>
+<div class="col-md-2 third-column">
+ <ul class="side-nav">
+ <li class="side-nav-title">On This Page</li>
+ <li><a href="#overview">Overview</a></li>
+ <li><a href="#ne">Named Entities</a></li>
+ <li><a href="#elements">Semantic elements</a></li>
+ <li><a href="#macros">Macros</a></li>
+ <li><a href="#usage">Usage examples</a></li>
+ {% include quick-links.html %}
+ </ul>
+</div>
+
+
+
+
