docs.html - incubator-nlpcraft-website - Git at Google

 ---
 active_crumb: Docs
 layout: documentation
 id: overview
 ---

 <!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
 -->

 <div class="col-md-8 second-column">
     <section id="overview">
         <h2 class="section-title">Overview <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
         <p>
             Apache NLPCraft is a JVM-based <a target=_blank href="https://www.apache.org/licenses/">open source</a> library
             for adding a natural language interface to modern applications.  It enables people to interact with your products using voice or text. NLPCraft can connect with
             any private or public data source, and has no hardware or software lock-ins. Its design is based on advanced
             <a href="/intent-matching.html">Intent Definition Language</a> (IDL) for defining non-trivial intents and a fully deterministic intent matching
             algorithm for the input utterances. You can build intents for NLPCraft using any JVM-based languages like Java, Scala, Kotlin, Groovy, etc. NLPCraft
             exposes REST APIs for integration with end-user applications.
         </p>
         <p>
             One of the key features of NLPCraft is its use of <a href="/intent-matching.html">IDL</a> coupled with deterministic intent matching that are tailor made for
             <em>domain-specific</em> natural language interface. This design doesn't force developers to use direct deep learning
             approach with time consuming corpora development and model training - resulting in much a
             <em>simpler <span class="amp">&</span> faster</em> implementation.
         </p>
         <p>
             Another key aspect of NLPCraft is its initial focus on processing English language. Although it may sound
             counterintuitive, this narrower initial focus enables NLPCraft to deliver unprecedented ease of use combined with
             unparalleled comprehension capabilities for English input out-of-the-box. It avoids academic, watered down functionality or overly
             complicated configuration and usage - following on project's <em>"built for engineers by engineers"</em> ethos.
             English language is spoken by more
             than a billion people on this planet and is de facto standard global language of the business and commerce.
         </p>
         <p>
             So, how does it work in a nutshell?
         </p>
         <p>
             When using NLPCraft you will be dealing with three main components:
         </p>
         <ul>
             <li><a href="#data-model">Data model</a></li>
             <li><a href="#data-probe">Data probe</a></li>
             <li><a href="#server">REST Server</a></li>
         </ul>
         <figure>
             <img class="img-fluid" src="/images/homepage-fig1.1.png" alt="">
             <figcaption><b>Fig 1.</b> NLPCraft Architecture</figcaption>
         </figure>
     </section>
     <section id="data-model">
         <h2 class="section-title">Data Model <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
         <p>
             NLPCraft employs a <em>model-as-a-code</em> approach where everything you do in NLPCraft is part of your source code. Data model is simply an implementation of
             <a target="javadoc" href="/apis/latest/org/apache/nlpcraft/model/NCModel.html">NCModel</a> Java interface that
             can be developed using any JVM programming language like Java, Scala, Kotlin or Groovy.
             Data model defines named entities, various configuration properties as well as intents to interpret user input. Model-as-a-code natively supports
             any software lifecycle tools and frameworks in Java ecosystem.
         </p>
         <p>
             Declarative portion of the model can be stored in a separate JSON or YAML file
             for simpler maintenance. There are no practical limitation on how complex or simple a model
             can be, or what other tools it can use. Data models use <a href="/intent-matching.html">intents</a> to match the user input.
         </p>
         <p>
             To use data model it has to be deployed into a data probe.
         </p>
     </section>
     <section id="data-probe">
         <h2 class="section-title">Data Probe <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
         <p>
             Data probe is a light-weight container designed to securely deploy and manage user data models.
             Each probe can deploy and manage multiple models and many probes can be connected to the REST server (or a cluster of REST servers).
             The main purpose of the data probe is to separate data model hosting from managing REST calls from the clients.
             While you would typically have just one REST server, you may have multiple data probes deployed
             in different geo-locations and configured differently.
         </p>
         <p>
             Data probes can be deployed and run anywhere as long as there is an ingress connectivity from the REST server, and are
             typically deployed in DMZ or close to your target data sources: on-premise, in the cloud, etc. Data
             probe uses strong 256-bit encryption and ingress only connectivity for communicating with the REST server.
         </p>
     </section>
     <section id="server">
         <h2 class="section-title">REST Server <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
         <p>
             REST server (or a cluster of REST servers behind a load balancer) provides URL endpoint for end-user applications
             to securely query data sources using natural language via data models deployed in data probes. Its main purpose is to
             accept REST-over-HTTP calls from end-user applications and route these requests to and from requested data probes.
         </p>
         <p>
             Unlike data probe that gets restarted every time the model is changed, i.e. during development, the
             REST server is a "fire-and-forget" component that can be launched once while various data probes can
             continuously reconnect to it. It can typically run as a Docker image locally on premise or in the cloud.
         </p>
         <p>
             Learn more about <a href="data-model.html">data model</a>,
             <a href="server-and-probe.html#probe">data probe</a> and <a href="server-and-probe.html#server">REST server</a>.
         </p>
     </section>
     <section id="in-depth">
         <h2 class="section-title">In-Depth Look <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
         <p>
             Watch this full video (34:42) of the presentation from
             <a target=_ href="https://www.apachecon.com/acasia2021/">ApacheCon Asia 2021</a> conference to get in-depth understanding of
             the reasons why NLPCraft project was developed and what are the key principles that underlying it:
         </p>
         <div>
             <iframe
                     width="514"
                     height="289"
                     src="https://www.youtube.com/embed/O7iK0AXvcJ8?modestbranding=1"
                     title="NLPCraft - Breaking Years Of Dogma In NLP"
                     frameborder="0"
                     allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
                     allowfullscreen>
             </iframe>
         </div>
     </section>
 </div>
 <div class="col-md-2 third-column">
     <ul class="side-nav">
         <li class="side-nav-title">On This Page</li>
         <li><a href="#overview">Overview</a></li>
         <li><a href="#data-model">Data Model</a></li>
         <li><a href="#data-probe">Data Probe</a></li>
         <li><a href="#server">REST Server</a></li>
         {% include quick-links.html %}
     </ul>
 </div>
	---
	active_crumb: Docs
	layout: documentation
	id: overview
	---

	<!--
	Licensed to the Apache Software Foundation (ASF) under one or more
	contributor license agreements. See the NOTICE file distributed with
	this work for additional information regarding copyright ownership.
	The ASF licenses this file to You under the Apache License, Version 2.0
	(the "License"); you may not use this file except in compliance with
	the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing, software
	distributed under the License is distributed on an "AS IS" BASIS,
	WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	See the License for the specific language governing permissions and
	limitations under the License.
	-->

	<div class="col-md-8 second-column">
	<section id="overview">
	<h2 class="section-title">Overview <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
	<p>
	Apache NLPCraft is a JVM-based <a target=_blank href="https://www.apache.org/licenses/">open source</a> library
	for adding a natural language interface to modern applications. It enables people to interact with your products using voice or text. NLPCraft can connect with
	any private or public data source, and has no hardware or software lock-ins. Its design is based on advanced
	<a href="/intent-matching.html">Intent Definition Language</a> (IDL) for defining non-trivial intents and a fully deterministic intent matching
	algorithm for the input utterances. You can build intents for NLPCraft using any JVM-based languages like Java, Scala, Kotlin, Groovy, etc. NLPCraft
	exposes REST APIs for integration with end-user applications.
	</p>
	<p>
	One of the key features of NLPCraft is its use of <a href="/intent-matching.html">IDL</a> coupled with deterministic intent matching that are tailor made for
	<em>domain-specific</em> natural language interface. This design doesn't force developers to use direct deep learning
	approach with time consuming corpora development and model training - resulting in much a
	<em>simpler <span class="amp">&</span> faster</em> implementation.
	</p>
	<p>
	Another key aspect of NLPCraft is its initial focus on processing English language. Although it may sound
	counterintuitive, this narrower initial focus enables NLPCraft to deliver unprecedented ease of use combined with
	unparalleled comprehension capabilities for English input out-of-the-box. It avoids academic, watered down functionality or overly
	complicated configuration and usage - following on project's <em>"built for engineers by engineers"</em> ethos.
	English language is spoken by more
	than a billion people on this planet and is de facto standard global language of the business and commerce.
	</p>
	<p>
	So, how does it work in a nutshell?
	</p>
	<p>
	When using NLPCraft you will be dealing with three main components:
	</p>
	<ul>
	<li><a href="#data-model">Data model</a></li>
	<li><a href="#data-probe">Data probe</a></li>
	<li><a href="#server">REST Server</a></li>
	</ul>
	<figure>
	<img class="img-fluid" src="/images/homepage-fig1.1.png" alt="">
	<figcaption><b>Fig 1.</b> NLPCraft Architecture</figcaption>
	</figure>
	</section>
	<section id="data-model">
	<h2 class="section-title">Data Model <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
	<p>
	NLPCraft employs a <em>model-as-a-code</em> approach where everything you do in NLPCraft is part of your source code. Data model is simply an implementation of
	<a target="javadoc" href="/apis/latest/org/apache/nlpcraft/model/NCModel.html">NCModel</a> Java interface that
	can be developed using any JVM programming language like Java, Scala, Kotlin or Groovy.
	Data model defines named entities, various configuration properties as well as intents to interpret user input. Model-as-a-code natively supports
	any software lifecycle tools and frameworks in Java ecosystem.
	</p>
	<p>
	Declarative portion of the model can be stored in a separate JSON or YAML file
	for simpler maintenance. There are no practical limitation on how complex or simple a model
	can be, or what other tools it can use. Data models use <a href="/intent-matching.html">intents</a> to match the user input.
	</p>
	<p>
	To use data model it has to be deployed into a data probe.
	</p>
	</section>
	<section id="data-probe">
	<h2 class="section-title">Data Probe <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
	<p>
	Data probe is a light-weight container designed to securely deploy and manage user data models.
	Each probe can deploy and manage multiple models and many probes can be connected to the REST server (or a cluster of REST servers).
	The main purpose of the data probe is to separate data model hosting from managing REST calls from the clients.
	While you would typically have just one REST server, you may have multiple data probes deployed
	in different geo-locations and configured differently.
	</p>
	<p>
	Data probes can be deployed and run anywhere as long as there is an ingress connectivity from the REST server, and are
	typically deployed in DMZ or close to your target data sources: on-premise, in the cloud, etc. Data
	probe uses strong 256-bit encryption and ingress only connectivity for communicating with the REST server.
	</p>
	</section>
	<section id="server">
	<h2 class="section-title">REST Server <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
	<p>
	REST server (or a cluster of REST servers behind a load balancer) provides URL endpoint for end-user applications
	to securely query data sources using natural language via data models deployed in data probes. Its main purpose is to
	accept REST-over-HTTP calls from end-user applications and route these requests to and from requested data probes.
	</p>
	<p>
	Unlike data probe that gets restarted every time the model is changed, i.e. during development, the
	REST server is a "fire-and-forget" component that can be launched once while various data probes can
	continuously reconnect to it. It can typically run as a Docker image locally on premise or in the cloud.
	</p>
	<p>
	Learn more about <a href="data-model.html">data model</a>,
	<a href="server-and-probe.html#probe">data probe</a> and <a href="server-and-probe.html#server">REST server</a>.
	</p>
	</section>
	<section id="in-depth">
	<h2 class="section-title">In-Depth Look <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
	<p>
	Watch this full video (34:42) of the presentation from
	<a target=_ href="https://www.apachecon.com/acasia2021/">ApacheCon Asia 2021</a> conference to get in-depth understanding of
	the reasons why NLPCraft project was developed and what are the key principles that underlying it:
	</p>
	<div>
	<iframe
	width="514"
	height="289"
	src="https://www.youtube.com/embed/O7iK0AXvcJ8?modestbranding=1"
	title="NLPCraft - Breaking Years Of Dogma In NLP"
	frameborder="0"
	allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
	allowfullscreen>
	</iframe>
	</div>
	</section>
	</div>
	<div class="col-md-2 third-column">
	<ul class="side-nav">
	<li class="side-nav-title">On This Page</li>
	<li><a href="#overview">Overview</a></li>
	<li><a href="#data-model">Data Model</a></li>
	<li><a href="#data-probe">Data Probe</a></li>
	<li><a href="#server">REST Server</a></li>
	{% include quick-links.html %}
	</ul>
	</div>