| --- |
| active_crumb: Server <span class="amp">&</span> Probe |
| layout: documentation |
| id: server_and_probe |
| --- |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| <div id="server-and-probes" class="col-md-8 second-column"> |
| <section> |
| <h2 class="section-title">Overview</h2> |
| <p> |
| As mentioned in <a href="/docs.html">overview</a> section the REST server and data probe are the two |
| runtime components that you need to run when using NLPCraft. |
| Data probes are used to deploy and host data model, while REST server (or a cluster of servers) is used |
| to accept client REST call and route them to the data models via data probes. |
| </p> |
| <div class="bq info"> |
| <b>REST Server vs. Data Probe</b> |
| <p> |
| It's important to remember why REST server is a separate component from a data probe. While a |
| typical deployment would have only one REST server (or a cluster of REST servers behind a single |
| load balancer), there are maybe multiple data probes hosting different data models deployed in |
| different physical locations, managed through different life cycles and requiring different |
| security and network configurations. |
| </p> |
| <p> |
| Moreover, REST server is a heavy and resource consuming component that is built around Apache |
| Ignite distributing in-memory computing capabilities - while the data probe is a lightweight |
| data model container. During the development and testing of data models, the developers need to |
| frequently redeploy data models by restarting the data probe. If the REST server and the data probe |
| would be one component - this process would be very inefficient. |
| </p> |
| </div> |
| <p> |
| Fro the rest of this section we assume that NLPCraft was <a href="/download.html">downloaded</a> and |
| <a href="/installation.html">installed</a> via ZIP archive. However, all instructions below are fully |
| applicable to any type of installation. |
| </p> |
| <p> |
| Once NLPCraft is downloaded as a ZIP archive and unzipped, the current directory will look like this: |
| </p> |
| <pre class="console"> |
| ├── LICENSE |
| ├── bin |
| ├── sql |
| ├── build |
| │ └── apache-nlpcraft-{{site.latest_version}} |
| │ ├── <b>nlpcraft.conf</b> |
| │ ├── ignite.xml |
| │ ├── log4j2.xml |
| │ └── <b>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</b> |
| ├── javadoc |
| ├── openapi |
| └── src |
| </pre> |
| <p> |
| Regardless of how NLPCraft was installed it comes with a single executable JAR file that includes all |
| necessary dependencies: <code>build/apache-nlpcraft-{{site.latest_version}}/<b>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</b></code>. |
| This JAR file includes binaries for data probe, data model APIs and the REST server. |
| </p> |
| </section> |
| <section id="server"> |
| <h2 class="section-title">REST Server</h2> |
| <p> |
| As mentioned above REST server (or a cluster of servers) is used to accept client REST calls and |
| route them to the data model via data probes. Note that both data probe and the REST server start the same way. |
| </p> |
| <p> |
| REST server can be stared in a <em>standard way</em> from either the command line or IDE such as Eclipse or IntelliJ IDEA: |
| </p> |
| <nav> |
| <div class="nav nav-tabs" role="tablist"> |
| <a class="nav-item nav-link active" data-toggle="tab" href="#nav-srv-jar" role="tab" aria-controls="nav-home" aria-selected="true">Executable JAR</a> |
| <a class="nav-item nav-link" data-toggle="tab" href="#nav-srv-cmd" role="tab" aria-controls="nav-home" aria-selected="true">Command Line</a> |
| <a class="nav-item nav-link" data-toggle="tab" href="#nav-srv-ide" role="tab" aria-controls="nav-home" aria-selected="true">IDE</a> |
| </div> |
| </nav> |
| <div class="tab-content"> |
| <div class="tab-pane fade show active" id="nav-srv-jar" role="tabpanel"> |
| <pre class="brush: plain"> |
| $ cd build |
| $ java -Xms1024m -jar apache-nlpcraft-{{site.latest_version}}-all-deps.jar -server |
| </pre> |
| </div> |
| <div class="tab-pane fade show" id="nav-srv-cmd" role="tabpanel"> |
| <pre class="brush: plain"> |
| $ cd build |
| $ java -Xms1024m -cp apache-nlpcraft-{{site.latest_version}}-all-deps.jar org.apache.nlpcraft.NCStart -server |
| </pre> |
| </div> |
| <div class="tab-pane fade show" id="nav-srv-ide" role="tabpanel"> |
| <p style="padding-top: 10px"> |
| Configure run configuration with the main class <code>org.apache.nlpcraft.NCStart</code>. |
| Note that <code>org.apache.nlpcraft.NCStart</code> class starts both the REST server and the data probe and is the |
| class that is configured as <code>Main-Class</code> in <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> |
| JAR file manifest. |
| </p> |
| </div> |
| </div> |
| Parameters: |
| <dl> |
| <dt> |
| <code>-server</code> |
| </dt> |
| <dd> |
| <em>Mandatory</em> parameter to indicate that you are starting the REST server. |
| </dd> |
| <dt><code>-config=path</code></dt> |
| <dd> |
| <em>Optional</em> parameter to provide configuration file path. |
| Server will automatically look for <code>nlpcraft.conf</code> configuration file in the same directory |
| as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file. If the configuration |
| file has different name or in different location use <code>-config=path</code> parameter |
| where <code>path</code> is an absolute path to the configuration file. Note that the server and the data |
| probe can use the same file for their configuration (just like the |
| default <code>nlpcraft.conf</code> contains configuration for both the server and the data probe). |
| </dd> |
| <dt><code>-igniteConfig=path</code></dt> |
| <dd> |
| <em>Optional</em> parameter to provide <a target=_ href="https://ignite.apache.org/">Apache Ignite</a> configuration file path. |
| Note that Apache Ignite is used as a cluster computing plane and a default distributed storage. |
| Server will automatically look for <code>ignite.xml</code> |
| configuration file in the same directory as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file. |
| If the configuration file has different name or in different location use <code>-igniteConfig=path</code> parameter |
| where <code>path</code> is an absolute path to the Ignite configuration file. |
| </dd> |
| </dl> |
| <div class="bq warn"> |
| <b>Java Memory</b> |
| <p> |
| Make sure to allocate enough memory for server JVM using <code>-Xms</code> JVM option, i.e. <code>-Xms1024m</code>. |
| Many 3rd party NLP engines like Stanford CoreNLP are very memory intensive and may require several GBs |
| of JVM heap allocated depending on the models used. Note that when server JVM has insufficient heap |
| memory the Apache Ignite may throw the following warning logs: |
| </p> |
| <pre class="brush: text, highlight: 2"> |
| Jul-22 13:27:56 [INFO ] ... |
| Jul-22 13:28:08 [WARN ] Possible too long JVM pause: 11364 milliseconds. |
| Jul-22 13:28:11 [INFO ] ... |
| </pre> |
| <p> |
| The abnormally long GC pauses (over 5s) can be caused by the excessive memory swapping performed by OS due to |
| insufficient JVM heap memory. |
| </p> |
| </div> |
| <h3 class="section-title"><i class="fab fa-docker"></i> Docker</h3> |
| <p> |
| You can also run REST server inside of Docker container: |
| </p> |
| <pre class="brush: plain"> |
| $ docker run -m 8G -p 8081:8081 -p 8201:8201 -p 8202:8202 nlpcraftserver/server:{{site.latest_version}} |
| </pre> |
| <p> |
| By default, the Docker image runs with a default configuration. |
| See <a href="#config">configuration</a> section on how to provide custom configuration via environment variables for the REST |
| server running inside of Docker container. |
| </p> |
| <p> |
| When the REST server successfully started you should see the log output similar to this: |
| </p> |
| <pre class="brush: plain, highlight: 19"> |
| _ ____ ______ ______ |
| / | / / /___ / ____/________ _/ __/ /_ |
| / |/ / / __ \/ / / ___/ __ `/ /_/ __/ |
| / /| / / /_/ / /___/ / / /_/ / __/ /_ |
| /_/ |_/_/ .___/\____/_/ \__,_/_/ \__/ |
| /_/ |
| |
| Server |
| Version: {{site.latest_version}} |
| Copyright (C) 2020 Apache Software Foundation. |
| |
| ... |
| |
| +-------------------------+ |
| | Server started [19.35s] | |
| +-------------------------+ |
| |
| |
| Mar-11 23:21:04 [INFO ] REST server is listening on 'localhost:8081'. |
| </pre> |
| </section> |
| <section id="probe"> |
| <h2 class="section-title">Data Probe</h2> |
| <p> |
| Just like the REST server the data probe can be started in a <em>standard way</em> from either the |
| command line or IDE such as Eclipse or IntelliJ IDEA: |
| </p> |
| <nav> |
| <div class="nav nav-tabs" role="tablist"> |
| <a class="nav-item nav-link active" data-toggle="tab" href="#nav-probe-cmd" role="tab" aria-controls="nav-home" aria-selected="true">Command Line</a> |
| <a class="nav-item nav-link" data-toggle="tab" href="#nav-probe-jar" role="tab" aria-controls="nav-home" aria-selected="true">Executable JAR</a> |
| <a class="nav-item nav-link" data-toggle="tab" href="#nav-probe-ide" role="tab" aria-controls="nav-home" aria-selected="true">IDE</a> |
| </div> |
| </nav> |
| <div class="tab-content"> |
| <div class="tab-pane fade show" id="nav-probe-jar" role="tabpanel"> |
| <pre class="brush: plain"> |
| $ cd build |
| $ java -Xms1024m -jar apache-nlpcraft-{{site.latest_version}}-all-deps.jar -probe |
| </pre> |
| <div class="bq warn"> |
| <p> |
| <b>NOTE:</b> when using executable JAR to start the data probe you cannot add your |
| own model classes to the classpath. You should either package your classes into JAR |
| file and configure probe accordingly - or use <code>-cp</code> option in |
| command line. |
| </p> |
| </div> |
| </div> |
| <div class="tab-pane fade show active" id="nav-probe-cmd" role="tabpanel"> |
| <pre class="brush: plain"> |
| $ cd build |
| $ java -Xms1024m -cp apache-nlpcraft-{{site.latest_version}}-all-deps.jar:/my/project/classes org.apache.nlpcraft.NCStart -probe -config=/my/project/probe.conf |
| </pre> |
| <p> |
| Directory <code>/my/project/classes</code> should contain all compiled classes for your models. |
| Make sure to replace <code>/my/project/classes</code> and <code>/my/project/probe.conf</code> with |
| the actual paths. |
| </p> |
| </div> |
| <div class="tab-pane fade show" id="nav-probe-ide" role="tabpanel"> |
| <p style="padding-top: 10px"> |
| Configure run configuration with the main class <code>org.apache.nlpcraft.NCStart</code>. |
| </p> |
| <div class="bq info"> |
| <b>Class <code>org.apache.nlpcraft.NCStart</code></b> |
| <p> |
| Note that <code>org.apache.nlpcraft.NCStart</code> class starts both the REST server and the data probe and is the |
| class that is configured as <code>Main-Class</code> in <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> |
| JAR file manifest. |
| </p> |
| </div> |
| </div> |
| </div> |
| Parameters: |
| <dl> |
| <dt> |
| <code>-probe</code> |
| </dt> |
| <dd> |
| <em>Mandatory</em> parameter to indicate that you are starting a data probe. |
| </dd> |
| <dt><code>-config=path</code></dt> |
| <dd> |
| <p> |
| <em>Optional</em> parameter to provide probe configuration file path. |
| Data probe will automatically look for <code>nlpcraft.conf</code> configuration file in the same directory |
| as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file. If the configuration |
| file has different name or in different location use <code>-config=path</code> parameter |
| where <code>path</code> is an absolute path to the data probe configuration file. Note that the server and the data |
| probe can use the same file for their configuration (just like the |
| default <code>nlpcraft.conf</code> contains configuration for both the server and the data probe). |
| </p> |
| </dd> |
| </dl> |
| <div class="bq info"> |
| <b>Adding Your Classes</b> |
| <p> |
| Note that when you are using a <em>command line</em> to start the probe you can also add your own classes that implement |
| your models. To do that you need to use <code>-cp</code> option instead of <code>-jar</code> and |
| construct your JVM classpath to include both the <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> |
| as well as directory where your compiled Java code is located: |
| </p> |
| <pre class="brush: plain, highlight: 2"> |
| $ cd build |
| $ java -cp apache-nlpcraft-{{site.latest_version}}-all-deps.jar:/my/project/classes org.apache.nlpcraft.NCStart -probe -config /my/project/probe.conf |
| </pre> |
| Make sure to replace <code>/my/project/classes</code> with your own directory where your compiled model classes |
| are located. Note that you need to specify class <code>org.apache.nlpcraft.NCStart</code> explicitly in this case. |
| </div> |
| <p> |
| When the data probe started you should see the log output similar to this: |
| </p> |
| <pre class="brush: plain"> |
| _ ____ ______ ______ |
| / | / / /___ / ____/________ _/ __/ /_ |
| / |/ / / __ \/ / / ___/ __ `/ /_/ __/ |
| / /| / / /_/ / /___/ / / /_/ / __/ /_ |
| /_/ |_/_/ .___/\____/_/ \__,_/_/ \__/ |
| /_/ |
| |
| Data Probe |
| Version: {{site.latest_version}} |
| Copyright (C) 2020 Apache Software Foundation. |
| |
| Mar-11 23:25:52 [INFO ] Probe Configuration: |
| +--------------------------------------------------------------------+ |
| | Probe ID | all.examples | |
| | Probe Token | 3141592653589793 | |
| | API Version | {{site.latest_version}}, 2019-03-01 | |
| | Down-Link | localhost:8202 | |
| | Up-Link | localhost:8201 | |
| +-----------------+--------------------------------------------------+ |
| | Models | org.apache.nlpcraft.examples.alarm.AlarmModel | |
| | | org.apache.nlpcraft.examples.echo.EchoModel | |
| | | org.apache.nlpcraft.examples.helloworld.HelloWorldModel | |
| | | org.apache.nlpcraft.examples.time.TimeModel | |
| | | org.apache.nlpcraft.examples.weather.WeatherModel | |
| +-----------------+--------------------------------------------------+ |
| | Lifecycle | | |
| | JARs Folder | | |
| +--------------------------------------------------------------------+ |
| |
| ... |
| |
| Mar-11 23:25:56 [INFO ] Models deployed: 5 |
| |
| +================================================================================+ |
| | Model ID | Name | Ver. | Elements | Synonyms | |
| +================================================================================+ |
| | nlpcraft.alarm.ex | Alarm Example Model | 1.0 | 1 | 419 | |
| | nlpcraft.weather.ex | Weather Example Model | 1.0 | 3 | 9045 | |
| | nlpcraft.helloworld.ex | HelloWorld Example Model | 1.0 | 0 | 0 | |
| | nlpcraft.time.ex | Time Example Model | 1.0 | 1 | 432 | |
| | nlpcraft.echo.ex | Echo Example Model | 1.0 | 0 | 0 | |
| +--------------------------------------------------------------------------------+ |
| |
| ... |
| |
| +--------------------------+ |
| | Probe started [5.12 sec] | |
| +--------------------------+ |
| |
| ... |
| |
| Mar-11 23:25:58 [INFO ] Server connection established. |
| </pre> |
| </section> |
| <section id="config"> |
| <h2 class="section-title">Configuration</h2> |
| <p> |
| Both REST server and the data probe use <a target=_ href="https://github.com/lightbend/config/">Typesafe Config</a> for their configuration: |
| </p> |
| <ul> |
| <li>Both the server and the data probe come with default configuration available in <code>build/<b>nlpcraft.conf</b></code> file.</li> |
| <li>Custom configuration or default overrides can be placed into a file or provided via environment variables.</li> |
| <li>Configuration files use <a target=_ href="https://github.com/lightbend/config/blob/master/HOCON.md">HOCON</a> file format.</li> |
| <li>Server and probe configuration can be placed in the same file or kept in separate files. </li> |
| </ul> |
| |
| <p> |
| By default, when REST server or data probe start they look for <code>nlpcraft.conf</code> configuration file in the same directory |
| as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file and the on their classpath. You can change this behavior with |
| <code>-config=path</code> parameter. |
| </p> |
| <p> |
| Default configuration is available in <code>build/<b>nlpcraft.conf</b></code> file and it is extensively documented. It has subsections |
| for the server and probe configuration. When server and the probe use different files these whole sections should be |
| placed into an individual file: |
| </p> |
| <p> |
| Server configuration file: |
| </p> |
| <pre class="brush: js"> |
| nlpcraft { |
| server { |
| ... |
| } |
| } |
| </pre> |
| <p> |
| Probe configuration file: |
| </p> |
| <pre class="brush: js"> |
| nlpcraft { |
| probe { |
| ... |
| } |
| } |
| </pre> |
| |
| |
| <div class="bq info"> |
| <b>Overriding Configuration</b> |
| <p> |
| There are two ways to provide your own configuration for the server or the probe: |
| </p> |
| <ul> |
| <li> |
| Put your configuration into separate file and use <code>-config=...</code> parameter to specify |
| the path of your configuration file. |
| </li> |
| <li> |
| Use environment variable to override specific single property. |
| </li> |
| </ul> |
| <p> |
| During development and testing, or when running in a container environment like Docker, it is more |
| convenient to use environment variables to override just few specific configuration properties: |
| </p> |
| <ol> |
| <li> |
| Set probe or server JVM system property <code>-Dconfig.override_with_env_vars=true</code> which will instruct |
| configuration framework to look for external overrides. |
| </li> |
| <li>For each configuration property <code>x.y.z</code> set the overriding environment variable <code>CONFIG_FORCE_x_y_z=some_value</code></li> |
| </ol> |
| <p> |
| Consider the following snippet of server configuration: |
| </p> |
| <pre class="brush: js"> |
| nlpcraft { |
| server { |
| ... |
| |
| lifecycle = [ |
| "org.apache.nlpcraft.server.lifecycle.opencensus.NCJaegerExporter" |
| ] |
| |
| rest { |
| host = "0.0.0.0" |
| port = 8081 |
| apiImpl = "org.apache.nlpcraft.server.rest.NCBasicRestApi" |
| } |
| |
| ... |
| } |
| } |
| </pre> |
| <p> |
| You can override these properties with the following environment variables: |
| </p> |
| <p> |
| <code>CONFIG_FORCE_nlpcraft_server_rest_host=1.2.3.4</code><br> |
| <code>CONFIG_FORCE_nlpcraft_server_lifecycle.0=org.apache.nlpcraft.server.lifecycle.opencensus.NCStackdriverTraceExporter</code><br> |
| <code>CONFIG_FORCE_nlpcraft_server_lifecycle.1=org.apache.nlpcraft.server.lifecycle.opencensus.NCStackdriverStatsExporter</code> |
| </p> |
| </div> |
| </section> |
| </div> |
| <div class="col-md-2 third-column"> |
| <ul class="side-nav"> |
| <li class="side-nav-title">On This Page</li> |
| <li><a href="#server">REST Server</a></li> |
| <li><a href="#probe">Data Probe</a></li> |
| <li><a href="#config">Configuration</a></li> |
| {% include quick-links.html %} |
| </ul> |
| </div> |
| |
| |
| |
| |
