---
active_crumb: Server <span class="amp">&amp;</span> Probe
layout: documentation
id: server_and_probe
---

<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<div id="server-and-probes" class="col-md-8 second-column">
    <section>
        <h2 class="section-title">Overview</h2>
        <p>
            As mentioned in <a href="/docs.html">overview</a> section the REST server and data probe are the two
            runtime components that you need to run when using NLPCraft.
            Data probes are used to deploy and host data model, while REST server (or a cluster of servers) is used
            to accept client REST call and route them to the data models via data probes.
        </p>
        <div class="bq info">
            <b>REST Server vs. Data Probe</b>
            <p>
                It's important to remember why REST server is a separate component from a data probe. While a
                typical deployment would have only one REST server (or a cluster of REST servers behind a single
                load balancer), there are maybe multiple data probes hosting different data models deployed in
                different physical locations, managed through different life cycles and requiring different
                security and network configurations.
            </p>
            <p>
                Moreover, REST server is a heavy and resource consuming component that is built around Apache
                Ignite distributing in-memory computing capabilities - while the data probe is a lightweight
                data model container. During the development and testing of data models, the developers need to
                frequently redeploy data models by restarting the data probe. If the REST server and the data probe
                would be one component - this process would be very inefficient.
            </p>
        </div>
        <p>
            Fro the rest of this section we assume that NLPCraft was <a href="/download.html">downloaded</a> and
            <a href="/installation.html">installed</a> via ZIP archive. However, all instructions below are fully
            applicable to any type of installation.
        </p>
        <p>
            Once NLPCraft is downloaded as a ZIP archive and unzipped, the current directory will look like this:
        </p>
        <pre class="console">
├── LICENSE
├── bin
├── sql
├── build
│   └── apache-nlpcraft-{{site.latest_version}}
│       ├── <b>nlpcraft.conf</b>
│       ├── ignite.xml
│       ├── log4j2.xml
│       └── <b>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</b>
├── javadoc
├── openapi
└── src
        </pre>
        <p>
            Regardless of how NLPCraft was installed it comes with a single executable JAR file that includes all
            necessary dependencies: <code>build/apache-nlpcraft-{{site.latest_version}}/<b>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</b></code>.
            This JAR file includes binaries for data probe, data model APIs and the REST server.
        </p>
    </section>
    <section id="server">
        <h2 class="section-title">REST Server</h2>
        <p>
            As mentioned above REST server (or a cluster of servers) is used to accept client REST calls and
            route them to the data model via data probes. Note that both data probe and the REST server start the same way.
        </p>
        <p>
            REST server can be stared in a <em>standard way</em> from either the command line or IDE such as Eclipse or IntelliJ IDEA:
        </p>
        <nav>
            <div class="nav nav-tabs" role="tablist">
                <a class="nav-item nav-link active" data-toggle="tab" href="#nav-srv-jar" role="tab" aria-controls="nav-home" aria-selected="true">Executable JAR</a>
                <a class="nav-item nav-link" data-toggle="tab" href="#nav-srv-cmd" role="tab" aria-controls="nav-home" aria-selected="true">Command Line</a>
                <a class="nav-item nav-link" data-toggle="tab" href="#nav-srv-ide" role="tab" aria-controls="nav-home" aria-selected="true">IDE</a>
            </div>
        </nav>
        <div class="tab-content">
            <div class="tab-pane fade show active" id="nav-srv-jar" role="tabpanel">
                <pre class="brush: plain">
                    $ cd build
                    $ java -Xms1024m -jar apache-nlpcraft-{{site.latest_version}}-all-deps.jar -server
                </pre>
            </div>
            <div class="tab-pane fade show" id="nav-srv-cmd" role="tabpanel">
                <pre class="brush: plain">
                    $ cd build
                    $ java -Xms1024m -cp apache-nlpcraft-{{site.latest_version}}-all-deps.jar org.apache.nlpcraft.NCStart -server
                </pre>
            </div>
            <div class="tab-pane fade show" id="nav-srv-ide" role="tabpanel">
                <p style="padding-top: 10px">
                    Configure run configuration with the main class <code>org.apache.nlpcraft.NCStart</code>.
                    Note that <code>org.apache.nlpcraft.NCStart</code> class starts both the REST server and the data probe and is the
                    class that is configured as <code>Main-Class</code> in <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code>
                    JAR file manifest.
                </p>
            </div>
        </div>
        Parameters:
        <dl>
            <dt>
                <code>-server</code>
            </dt>
            <dd>
                <em>Mandatory</em> parameter to indicate that you are starting the REST server.
            </dd>
            <dt><code>-config=path</code></dt>
            <dd>
                <em>Optional</em> parameter to provide configuration file path.
                Server will automatically look for <code>nlpcraft.conf</code> configuration file in the same directory
                as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file. If the configuration
                file has different name or in different location use <code>-config=path</code> parameter
                where <code>path</code> is an absolute path to the configuration file. Note that the server and the data
                probe can use the same file for their configuration (just like the
                default <code>nlpcraft.conf</code> contains configuration for both the server and the data probe).
            </dd>
            <dt><code>-igniteConfig=path</code></dt>
            <dd>
                <em>Optional</em> parameter to provide <a target=_ href="https://ignite.apache.org/">Apache Ignite</a> configuration file path.
                Note that Apache Ignite is used as a cluster computing plane and a default distributed storage.
                Server will automatically look for <code>ignite.xml</code>
                configuration file in the same directory as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file.
                If the configuration file has different name or in different location use <code>-igniteConfig=path</code> parameter
                where <code>path</code> is an absolute path to the Ignite configuration file.
            </dd>
        </dl>
        <div class="bq warn">
            <b>Java Memory</b>
            <p>
                Make sure to allocate enough memory for server JVM using <code>-Xms</code> JVM option, i.e. <code>-Xms1024m</code>.
                Many 3rd party NLP engines like Stanford CoreNLP are very memory intensive and may require several GBs
                of JVM heap allocated depending on the models used. Note that when server JVM has insufficient heap
                memory the Apache Ignite may throw the following warning logs:
            </p>
            <pre class="brush: text, highlight: 2">
                Jul-22 13:27:56 [INFO ] ...
                Jul-22 13:28:08 [WARN ] Possible too long JVM pause: 11364 milliseconds.
                Jul-22 13:28:11 [INFO ] ...
            </pre>
            <p>
                The abnormally long GC pauses (over 5s) can be caused by the excessive memory swapping performed by OS due to
                insufficient JVM heap memory.
            </p>
        </div>
        <h3 class="section-title"><i class="fab fa-docker"></i> Docker</h3>
        <p>
            You can also run REST server inside of Docker container:
        </p>
        <pre class="brush: plain">
            $ docker run -m 8G -p 8081:8081 -p 8201:8201 -p 8202:8202 nlpcraftserver/server:{{site.latest_version}}
        </pre>
        <p>
            By default, the Docker image runs with a default configuration.
            See <a href="#config">configuration</a> section on how to provide custom configuration via environment variables for the REST
            server running inside of Docker container.
        </p>
        <p>
            When the REST server successfully started you should see the log output similar to this:
        </p>
        <pre class="brush: plain, highlight: 19">
                _   ____      ______           ______
               / | / / /___  / ____/________ _/ __/ /_
              /  |/ / / __ \/ /   / ___/ __ `/ /_/ __/
             / /|  / / /_/ / /___/ /  / /_/ / __/ /_
            /_/ |_/_/ .___/\____/_/   \__,_/_/  \__/
                   /_/

            Server
            Version: {{site.latest_version}}
            Copyright (C) 2020 Apache Software Foundation.

            ...

            +-------------------------+
            | Server started [19.35s] |
            +-------------------------+


            Mar-11 23:21:04 [INFO ] REST server is listening on 'localhost:8081'.
        </pre>
    </section>
    <section id="probe">
        <h2 class="section-title">Data Probe</h2>
        <p>
            Just like the REST server the data probe can be started in a <em>standard way</em> from either the
            command line or IDE such as Eclipse or IntelliJ IDEA:
        </p>
        <nav>
            <div class="nav nav-tabs" role="tablist">
                <a class="nav-item nav-link active" data-toggle="tab" href="#nav-probe-cmd" role="tab" aria-controls="nav-home" aria-selected="true">Command Line</a>
                <a class="nav-item nav-link" data-toggle="tab" href="#nav-probe-jar" role="tab" aria-controls="nav-home" aria-selected="true">Executable JAR</a>
                <a class="nav-item nav-link" data-toggle="tab" href="#nav-probe-ide" role="tab" aria-controls="nav-home" aria-selected="true">IDE</a>
            </div>
        </nav>
        <div class="tab-content">
            <div class="tab-pane fade show" id="nav-probe-jar" role="tabpanel">
                <pre class="brush: plain">
                    $ cd build
                    $ java -Xms1024m -jar apache-nlpcraft-{{site.latest_version}}-all-deps.jar -probe
                </pre>
                <div class="bq warn">
                    <p>
                        <b>NOTE:</b> when using executable JAR to start the data probe you cannot add your
                        own model classes to the classpath. You should either package your classes into JAR
                        file and configure probe accordingly - or use <code>-cp</code> option in
                        command line.
                    </p>
                </div>
            </div>
            <div class="tab-pane fade show active" id="nav-probe-cmd" role="tabpanel">
                <pre class="brush: plain">
                    $ cd build
                    $ java -Xms1024m -cp apache-nlpcraft-{{site.latest_version}}-all-deps.jar:/my/project/classes org.apache.nlpcraft.NCStart -probe -config=/my/project/probe.conf
                </pre>
                <p>
                    Directory <code>/my/project/classes</code> should contain all compiled classes for your models.
                    Make sure to replace <code>/my/project/classes</code> and <code>/my/project/probe.conf</code> with
                    the actual paths.
                </p>
            </div>
            <div class="tab-pane fade show" id="nav-probe-ide" role="tabpanel">
                <p style="padding-top: 10px">
                    Configure run configuration with the main class <code>org.apache.nlpcraft.NCStart</code>.
                </p>
                <div class="bq info">
                    <b>Class <code>org.apache.nlpcraft.NCStart</code></b>
                    <p>
                        Note that <code>org.apache.nlpcraft.NCStart</code> class starts both the REST server and the data probe and is the
                        class that is configured as <code>Main-Class</code> in <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code>
                        JAR file manifest.
                    </p>
                </div>
            </div>
        </div>
        Parameters:
        <dl>
            <dt>
                <code>-probe</code>
            </dt>
            <dd>
                <em>Mandatory</em> parameter to indicate that you are starting a data probe.
            </dd>
            <dt><code>-config=path</code></dt>
            <dd>
                <p>
                    <em>Optional</em> parameter to provide probe configuration file path.
                    Data probe will automatically look for <code>nlpcraft.conf</code> configuration file in the same directory
                    as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file. If the configuration
                    file has different name or in different location use <code>-config=path</code> parameter
                    where <code>path</code> is an absolute path to the data probe configuration file. Note that the server and the data
                    probe can use the same file for their configuration (just like the
                    default <code>nlpcraft.conf</code> contains configuration for both the server and the data probe).
                </p>
            </dd>
        </dl>
        <div class="bq info">
            <b>Adding Your Classes</b>
            <p>
                Note that when you are using a <em>command line</em> to start the probe you can also add your own classes that implement
                your models. To do that you need to use <code>-cp</code> option instead of <code>-jar</code> and
                construct your JVM classpath to include both the <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code>
                as well as directory where your compiled Java code is located:
            </p>
            <pre class="brush: plain, highlight: 2">
                    $ cd build
                    $ java -cp apache-nlpcraft-{{site.latest_version}}-all-deps.jar:/my/project/classes org.apache.nlpcraft.NCStart -probe -config /my/project/probe.conf
            </pre>
            Make sure to replace <code>/my/project/classes</code> with your own directory where your compiled model classes
            are located. Note that you need to specify class <code>org.apache.nlpcraft.NCStart</code> explicitly in this case.
        </div>
        <p>
            When the data probe started you should see the log output similar to this:
        </p>
        <pre class="brush: plain">
            _   ____      ______           ______
           / | / / /___  / ____/________ _/ __/ /_
          /  |/ / / __ \/ /   / ___/ __ `/ /_/ __/
         / /|  / / /_/ / /___/ /  / /_/ / __/ /_
        /_/ |_/_/ .___/\____/_/   \__,_/_/  \__/
               /_/

        Data Probe
        Version: {{site.latest_version}}
        Copyright (C) 2020 Apache Software Foundation.

        Mar-11 23:25:52 [INFO ] Probe Configuration:
        +--------------------------------------------------------------------+
        | Probe ID        | all.examples                                     |
        | Probe Token     | 3141592653589793                                 |
        | API Version     | {{site.latest_version}}, 2019-03-01                                |
        | Down-Link       | localhost:8202                                   |
        | Up-Link         | localhost:8201                                   |
        +-----------------+--------------------------------------------------+
        | Models          | org.apache.nlpcraft.examples.alarm.AlarmModel           |
        |                 | org.apache.nlpcraft.examples.echo.EchoModel             |
        |                 | org.apache.nlpcraft.examples.helloworld.HelloWorldModel |
        |                 | org.apache.nlpcraft.examples.time.TimeModel             |
        |                 | org.apache.nlpcraft.examples.weather.WeatherModel       |
        +-----------------+--------------------------------------------------+
        | Lifecycle       |                                                  |
        | JARs Folder     |                                                  |
        +--------------------------------------------------------------------+

        ...

        Mar-11 23:25:56 [INFO ] Models deployed: 5

        +================================================================================+
        |        Model ID        |           Name           | Ver. | Elements | Synonyms |
        +================================================================================+
        | nlpcraft.alarm.ex      | Alarm Example Model      | 1.0  | 1        | 419      |
        | nlpcraft.weather.ex    | Weather Example Model    | 1.0  | 3        | 9045     |
        | nlpcraft.helloworld.ex | HelloWorld Example Model | 1.0  | 0        | 0        |
        | nlpcraft.time.ex       | Time Example Model       | 1.0  | 1        | 432      |
        | nlpcraft.echo.ex       | Echo Example Model       | 1.0  | 0        | 0        |
        +--------------------------------------------------------------------------------+

        ...

        +--------------------------+
        | Probe started [5.12 sec] |
        +--------------------------+

        ...

        Mar-11 23:25:58 [INFO ] Server connection established.
        </pre>
    </section>
    <section id="config">
        <h2 class="section-title">Configuration</h2>
        <p>
            Both REST server and the data probe use <a target=_ href="https://github.com/lightbend/config/">Typesafe Config</a> for their configuration:
        </p>
        <ul>
            <li>Both the server and the data probe come with default configuration available in <code>build/<b>nlpcraft.conf</b></code> file.</li>
            <li>Custom configuration or default overrides can be placed into a file or provided via environment variables.</li>
            <li>Configuration files use <a target=_ href="https://github.com/lightbend/config/blob/master/HOCON.md">HOCON</a> file format.</li>
            <li>Server and probe configuration can be placed in the same file or kept in separate files. </li>
        </ul>

        <p>
            By default, when REST server or data probe start they look for <code>nlpcraft.conf</code> configuration file in the same directory
            as <code>apache-nlpcraft-{{site.latest_version}}-all-deps.jar</code> file and the on their classpath. You can change this behavior with
            <code>-config=path</code> parameter.
        </p>
        <p>
            Default configuration is available in <code>build/<b>nlpcraft.conf</b></code> file and it is extensively documented. It has subsections
            for the server and probe configuration. When server and the probe use different files these whole sections should be
            placed into an individual file:
        </p>
        <p>
        Server configuration file:
        </p>
        <pre class="brush: js">
nlpcraft {
    server {
        ...
    }
}
            </pre>
        <p>
            Probe configuration file:
        </p>
        <pre class="brush: js">
nlpcraft {
    probe {
        ...
    }
}
            </pre>


        <div class="bq info">
            <b>Overriding Configuration</b>
            <p>
                There are two ways to provide your own configuration for the server or the probe:
            </p>
            <ul>
                <li>
                    Put your configuration into separate file and use <code>-config=...</code> parameter to specify
                    the path of your configuration file.
                </li>
                <li>
                    Use environment variable to override specific single property.
                </li>
            </ul>
            <p>
                During development and testing, or when running in a container environment like Docker, it is more
                convenient to use environment variables to override just few specific configuration properties:
            </p>
            <ol>
                <li>
                    Set probe or server JVM system property <code>-Dconfig.override_with_env_vars=true</code> which will instruct
                    configuration framework to look for external overrides.
                </li>
                <li>For each configuration property <code>x.y.z</code> set the overriding environment variable <code>CONFIG_FORCE_x_y_z=some_value</code></li>
            </ol>
            <p>
                Consider the following snippet of server configuration:
            </p>
            <pre class="brush: js">
nlpcraft {
    server {
        ...

        lifecycle = [
            "org.apache.nlpcraft.server.lifecycle.opencensus.NCJaegerExporter"
        ]

        rest {
            host = "0.0.0.0"
            port = 8081
            apiImpl = "org.apache.nlpcraft.server.rest.NCBasicRestApi"
        }

        ...
    }
}
            </pre>
            <p>
                You can override these properties with the following environment variables:
            </p>
            <p>
                <code>CONFIG_FORCE_nlpcraft_server_rest_host=1.2.3.4</code><br>
                <code>CONFIG_FORCE_nlpcraft_server_lifecycle.0=org.apache.nlpcraft.server.lifecycle.opencensus.NCStackdriverTraceExporter</code><br>
                <code>CONFIG_FORCE_nlpcraft_server_lifecycle.1=org.apache.nlpcraft.server.lifecycle.opencensus.NCStackdriverStatsExporter</code>
            </p>
        </div>
    </section>
</div>
<div class="col-md-2 third-column">
    <ul class="side-nav">
        <li class="side-nav-title">On This Page</li>
        <li><a href="#server">REST Server</a></li>
        <li><a href="#probe">Data Probe</a></li>
        <li><a href="#config">Configuration</a></li>
        {% include quick-links.html %}
    </ul>
</div>