examples/weather_bot.html

---
active_crumb: Weather Bot <code><sub>ex</sub></code>
layout: documentation
id: weather_bot
---

<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<div class="col-md-8 second-column">
    <section id="overview">
        <h2 class="section-title">Overview</h2>
        <p>
            This example demonstrates relatively complete NLI-based weather service with JSON output and a non-trivial
            intent matching logic. It uses Apple's <a target="new" href="https://darksky.net">DarkSky</a>
            REST service for the actual weather information.
        </p>
        <p>
            Complexity: <span class="complexity-two-star"><i class="fas fa-star"></i><i class="fas fa-star"></i><i class="far fa-star"></i></span><br/>
            Source code: <a target="github" href="https://github.com/apache/incubator-nlpcraft/tree/master/src/main/scala/org/apache/nlpcraft/examples/weather">GitHub</a>
        </p>
    </section>
    <section id="new_project">
        <h3 class="section-title">Create New Project</h3>
        <p>
            You can create new Java project in many different ways - we'll use Maven archetype generation
            for that. In your home folder run the following command:
        </p>
        <pre class="brush: text">
            mvn archetype:generate -DgroupId=examples -DartifactId=my-app -DarchetypeVersion=1.4 -DinteractiveMode=false
        </pre>
        <p>
            This will create <code>my-app</code> folder with the following default maven project structure:
        </p>
        <pre class="console">
├── <b>pom.xml</b>
└── src
    ├── main
    │   └── java
    │       └── examples
    │           └── App.java
    └── test
        └── java
            └── examples
                └── AppTest.java
        </pre>
        <div class="bq info">
            <p>
                Note that this setup is same for all examples. Note also that you can use any other tools for
                creating and managing Java project with or without Maven.
            </p>
        </div>
        <p>
            For our example we'll use JetBrain's <a target=_new href="https://www.jetbrains.com/idea/">IntelliJ IDEA</a>.
            Create new IDEA project from this source folder (make sure to pick JDK 8 or later JDK and language support).
            Let's also delete auto-generated files <code>App.java</code> and <code>AppTest.java</code> from our
            project as we won't be using them.
        </p>
    </section>
    <section id="add_nlpcraft">
        <h3 class="section-title">Add NLPCraft</h3>
        <p>
            Next we need to add NLPCraft dependency to our new project. Open <code>pom.xml</code> file and replace
            <code>dependencies</code> section with the following code:
        </p>
        <pre class="brush: xml, highlight: [3, 4, 5]">
            &lt;dependencies&gt;
                &lt;dependency&gt;
                    &lt;groupId&gt;org.apache.nlpcraft&lt;/groupId&gt;
                    &lt;artifactId&gt;nlpcraft&lt;/artifactId&gt;
                    &lt;version&gt;{{site.latest_version}}&lt;/version&gt;
                &lt;/dependency&gt;
            &lt;/dependencies&gt;
        </pre>
        <p>
            Also make sure that you have correct JDK version (1.8 or above) for the maven compiler plugin:
        </p>
        <pre class="brush: xml, highlight: [3, 4]">
            &lt;properties&gt;
                &lt;project.build.sourceEncoding&gt;UTF-8&lt;/project.build.sourceEncoding&gt;
                &lt;maven.compiler.source&gt;1.8&lt;/maven.compiler.source&gt;
                &lt;maven.compiler.target&gt;1.8&lt;/maven.compiler.target&gt;
            &lt;/properties&gt;
        </pre>
        <p>
            IDEA should automatically reload the project with newly updated <code>pom.xml</code> file and
            we should be ready now to develop our data model.
        </p>
    </section>
    <section id="model">
        <h3 class="section-title">Data Model</h3>
        <p>
            We are going to start with declaring the static part of our semantic model using JSON which we will later load using
            <code>NCModelFileAdapter</code> in our Java-based model implementation. Create new <code>weather_model.json</code>
            file and add the following model declaration into it:
        </p>
        <pre class="brush: js, highlight: [10, 42, 50, 59]">
{
    "id": "nlpcraft.weather.ex",
    "name": "Weather Example Model",
    "version": "1.0",
    "description": "Weather example model.",
    "examples": [
        "What's the local weather forecast?",
        "What's the weather in Moscow?"
    ],
    "macros": [
        {
            "name": "&lt;OF&gt;",
            "macro": "{of|for|per}"
        },
        {
            "name": "&lt;CHANCE&gt;",
            "macro": "{chance|possibility|probability|odds|likelihood|potential|risk|opportunity}"
        },
        {
            "name": "&lt;PHENOMENON&gt;",
            "macro": "{high sea|severe weather|hail|heat wave|cold wave|derecho|supercell|avalanche|cyclone|wildfire|landslide|firestorm|dust storm|thunder snow|winter storm|cloudburst|shower|condensation|precipitation|drizzle|rainstorm|rain storm|rainfall|rain|storm|sun|sunshine|cloud|hot|cold|dry|wet|wind||hurricane|typhoon|sand-storm|sand storm|tornado|humid|fog|snow|smog|black ice|haze|thundershower|thundersnow|sleet|drought|wildfire|blizzard|avalanche|mist|thunderstorm}"
        },
        {
            "name": "&lt;CUR&gt;",
            "macro": "{current|present|moment|now}"
        },
        {
            "name": "&lt;WEATHER&gt;",
            "macro": "{weather {condition|temp|temperature|data|*}|condition|temp|temperature}"
        },
        {
            "name": "&lt;FORECAST&gt;",
            "macro": "{forecast|prognosis|prediction}"
        },
        {
            "name": "&lt;HISTORY&gt;",
            "macro": "{history|past}"
        }
    ],
    "elements": [
        {
            "id": "wt:hist",
            "description": "Past weather conditions.",
            "synonyms": [
                "{&lt;WEATHER&gt;|*} &lt;HISTORY&gt;",
                "&lt;HISTORY&gt; {&lt;OF&gt;|*} {&lt;WEATHER&gt;|&lt;PHENOMENON&gt;}"
            ]
        },
        {
            "id": "wt:curr",
            "description": "Current weather conditions.",
            "synonyms": [
                "{&lt;CUR&gt;|*} {&lt;WEATHER&gt;|&lt;PHENOMENON&gt;}",
                "&lt;CHANCE&gt; &lt;OF&gt; &lt;PHENOMENON&gt;",
                "&lt;PHENOMENON&gt; {&lt;CHANCE&gt;|*}"
            ]
        },
        {
            "id": "wt:fcast",
            "description": "Future weather forecast.",
            "synonyms": [
                "{&lt;PHENOMENON&gt;|&lt;WEATHER&gt;|*} &lt;FORECAST&gt;",
                "&lt;FORECAST&gt; {&lt;OF&gt;|*} {&lt;WEATHER&gt;|&lt;PHENOMENON&gt;}"
            ]
        }
    ]
}
        </pre>
        <p>There are number of important points here:</p>
        <ul>
            <li>
                <code>Line 10</code> defines several macros that are used later on throughout the model's elements
                to shorten the synonym declarations. Note how macros coupled with option groups
                shorten overall synonym declarations 1000:1 vs. manually listing all possible word permutations.
            </li>
            <li>
                <code>Lines 42, 50, 59</code> define three model elements: the past, present and future (forecast) weather
                condition.
            </li>
        </ul>
        <p>
            Now that our model is ready let's create a Java class that would load this model and define intents
            that use the model elements we have just defined.
        </p>
    </section>
    <section id="code">
        <h3 class="section-title">Model Class</h3>
        <p>
            Our Java implementation uses one auxiliary class <a target="github" href="https://github.com/apache/incubator-nlpcraft/blob/master/src/main/scala/org/apache/nlpcraft/examples/weather/WeatherResultWrapper.java">WeatherResultWrapper.java</a>
            that is used for JSON result formatting. Note that the main <a target="github" href="https://github.com/apache/incubator-nlpcraft/blob/master/src/main/scala/org/apache/nlpcraft/examples/weather/WeatherModel.java">WeatherModel</a> class
            have number of utility methods that we'll skip here (you can see the entire class by following the link above) and
            we'll concentrate on the intents only:
        </p>
        <pre class="brush: java, highlight: [1, 2, 3, 4, 5, 10, 11, 12, 13, 14, 19, 20, 21, 22, 23]">
    @NCIntent("intent=fcast term={id == 'wt:fcast'} term(city)={id == 'nlpcraft:city'}? term(date)={id == 'nlpcraft:date'}?")
    public NCResult onForecastMatch(
        NCIntentMatch ctx,
        @NCIntentTerm("city") Optional&lt;NCToken&gt; cityTokOpt,
        @NCIntentTerm("date") Optional&lt;NCToken&gt; dateTokOpt
    ) {
        return onPeriodMatch(ctx, cityTokOpt, dateTokOpt, 5);
    }
                
    @NCIntent("intent=hist term={id == 'wt:hist'} term(city)={id == 'nlpcraft:city'}? term(date)={id == 'nlpcraft:date'}?")
    public NCResult onHistoryMatch(
        NCIntentMatch ctx,
        @NCIntentTerm("city") Optional&lt;NCToken&gt; cityTokOpt,
        @NCIntentTerm("date") Optional&lt;NCToken&gt; dateTokOpt
    ) {
        return onPeriodMatch(ctx, cityTokOpt, dateTokOpt, -5);
    }
                        
    @NCIntent("intent=curr term={id == 'wt:curr'} term(city)={id == 'nlpcraft:city'}? term(date)={id == 'nlpcraft:date'}?")
    public NCResult onCurrentMatch(
        NCIntentMatch ctx,
        @NCIntentTerm("city") Optional&lt;NCToken&gt; cityTokOpt,
        @NCIntentTerm("date") Optional&lt;NCToken&gt; dateTokOpt
    ) {
        checkMatch(ctx);

        try {
            Coordinate cr = prepGeo(ctx, cityTokOpt);

            if (dateTokOpt.isPresent()) {
                DateRange range = extractDate(dateTokOpt.get());

                return makeResult(srv.getTimeMachine(cr.latitude, cr.longitude, range.from, range.to), ctx.getIntentId());
            }

            return makeResult(srv.getCurrent(cr.latitude, cr.longitude), ctx.getIntentId());
        }
        catch (DarkSkyException e) {
            throw new NCRejection(e.getLocalizedMessage());
        }
        catch (NCRejection e) {
            throw e;
        }
        catch (Exception e) {
            throw new NCRejection("Weather provider error.", e);
        }
    }
        </pre>
        <p>
            There three methods define three intents. Each intent is defined "in place", i.e. as an annotation on the
            method that acts as a callback for that intent:
        </p>
        <ul>
            <li>
                Line 1 defines intent and the callback for the weather forecast (weather in the future):
                <ul>
                    <li>Intent is ordered (default), supports conversation (default) and has ID <code>fcast</code></li>
                    <li>Intent has one mandatory term and two optional terms:
                        <ul>
                            <li>
                                Mandatory term is defined as a token with ID <code>wt:fcast</code> as defined
                                in the model.
                            </li>
                            <li>
                                Two optional terms (their IDs are <code>city</code> and <code>date</code>) define
                                optional city and date for the weather report. If not provided, their will be taken
                                from either conversation context or assume the default value (current IP geo location
                                and current time, if available).
                            </li>
                        </ul>
                    </li>
                </ul>
            </li>
            <li>
                Lines 3-5 define formal parameters for the <code>fcast</code> intent callback method. Note that we use term IDs
                (<code>city</code> and <code>date</code>) in <code>@NCIntentTerm</code> annotations to automatically
                assign tokens from the detected terms to their corresponding method formal parameters.
            </li>
            <li>
                Lines 10-14 and lines 19-23 define two other intents and their callbacks methods in the same manner.
            </li>
            <li>
                Implementation of all intent callback method is pretty straightforward and deals mostly with figuring
                out the default location and time if either one isn't provided.
            </li>
        </ul>
    </section>
    <section id="tools">
        <h3 class="section-title">External Tools</h3>
        <p>
            This example uses several external tools to implement its functionality:
        </p>
        <ul>
            <li>
                <a target=_ href="https://darksky.net">Apple DarkSky</a> - to provide actual weather data service
                See <code>org.apache.nlpcraft.examples.misc.darksky</code> package for details.
            </li>
            <li>
                <a target="_" href="https://tools.keycdn.com/geo">KeyCDN's IP Location Finder</a> - to provide IP location
                service. See <code>org.apache.nlpcraft.examples.misc.geo.keycdn</code> package for details.
            </li>
            <li>
                City to timezone mapper. See <code>org.apache.nlpcraft.examples.misc.geo.cities</code> package for details.
            </li>
        </ul>
    </section>
    <section id="start_probe">
        <h3 class="section-title">Start Data Probe <sub>optional</sub></h3>
        <div class="bq warn">
            <p><b>Embedded Probe</b></p>
            <p>
                If you are using the <a href="#testing">unit test</a> that comes with this example you <b>do not</b>
                need to start the data probe standalone as this unit test uses embedded probe mode. In this mode, the unit
                test will automatically start and stop the data probe from within the test itself.
            </p>
            <p>
                <b>If using <a href="#testing">unit test</a> below - skip this step, you only need to start the server.</b>
            </p>
        </div>
        <p>
            NLPCraft data models get deployed into data probe. Let's start a standalone data probe with our newly
            created data model. To start data probe we need to configure Run Configuration in IDEA with
            the following parameters:
        </p>
        <ul>
            <li>
                <b>Main class:</b> <code>org.apache.nlpcraft.NCStart</code>
            </li>
            <li>
                <b>VM arguments:</b> <code>-Dconfig.override_with_env_vars=true</code>
            </li>
            <li>
                <b>Environment variable:</b> <code>CONFIG_FORCE_nlpcraft_probe_models.0=org.apache.nlpcraft.examples.weather.WeatherModel</code>
            </li>
            <li>
                <b>Program arguments: </b> <code>-probe</code>
            </li>
        </ul>
        <div class="bq info">
            <p>
                <b>NOTE:</b> instead of supplying a <a href="/server-and-probe.html">full configuration file</a> we just
                use the default configuration and override one configuration property using
                configuration override via environment variables.
            </p>
        </div>
        <p>
            Start this run configuration and make sure you have positive console output indicating that our model
            has been successfully loaded and probe started.
        </p>
    </section>
    <section id="start_server">
        <h3 class="section-title">Start REST Server</h3>
        <p>
            REST server listens for requests from client applications and routes them to the requested data models
            via connected data probes. REST server starts the same way as the data probe. Configure new
            Run Configuration in IDEA with the following parameters:
        </p>
        <ul>
            <li>
                <b>Main class:</b> <code>org.apache.nlpcraft.NCStart</code>
            </li>
            <li>
                <b>Program arguments: </b> <code>-server</code>
            </li>
        </ul>
        <p>
            Once started ensure that your REST server console output shows that data probe is connected and the
            REST server is listening on the default <code>localhost:8081</code> endpoint.
        </p>
        <p>
            At this point we've developed our data model, deployed it into the data probe, and started the REST server.
            To test it, we'll use the built-in <a href="/tools/test_framework.html">test framework</a>
            that allows you to write convenient unit tests against your data model.
        </p>
    </section>
    <section id="testing">
        <h3 class="section-title">Testing</h3>
        <p>
            NLPCraft comes with easy to use <a href="/tools/test_framework.html">test framework</a> for
            data models that can be used with
            any unit testing framework like JUnit or ScalaTest. It is essentially a simplified
            version of Java REST client that is custom designed for data model testing.
        </p>
        <p>
            We would like to test with following user requests:
        </p>
        <ul>
            <li><code>"What's the local weather forecast?"</code></li>
            <li><code>"What's the weather in Moscow?"</code></li>
            <li><code>"Chance of snow?"</code> (using conversation context)</li>
            <li><code>"Moscow?"</code> (using conversation context).</li>
        </ul>
        <p>
            Let's create new Java class <code>WeatherTest.java</code> with the following code:
        </p>
        <pre class="brush: java, highlight: [47, 43, 55, 59]">
package org.apache.nlpcraft.examples.weather;

import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import org.apache.nlpcraft.common.NCException;
import org.apache.nlpcraft.model.tools.test.NCTestClient;
import org.apache.nlpcraft.model.tools.test.NCTestClientBuilder;
import org.apache.nlpcraft.model.tools.test.NCTestResult;
import org.apache.nlpcraft.probe.embedded.NCEmbeddedProbe;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.lang.reflect.Type;
import java.util.HashMap;
import java.util.Map;

import static org.junit.jupiter.api.Assertions.*;

class WeatherTest {
    private static final Gson GSON = new Gson();
    private static final Type TYPE_MAP_RESP = new TypeToken&lt;HashMap&lt;String, Object&gt;&gt;() {}.getType();
    private NCTestClient cli;
            
    private void checkIntent(String txt, String intentId, boolean shouldBeSame) throws NCException, IOException {
        NCTestResult res = cli.ask(txt);

        assertTrue(res.isOk(), () -&gt; res.getResultError().get());

        assert res.getResult().isPresent();

        Map&lt;String, Object&gt; map = GSON.fromJson(res.getResult().get(), TYPE_MAP_RESP);

        if (shouldBeSame)
            assertEquals(intentId, map.get("intentId"));
        else
            assertNotEquals(intentId, map.get("intentId"));
    }

    @BeforeEach
    void setUp() throws NCException, IOException {
        NCEmbeddedProbe.start(WeatherModel.class);

        cli = new NCTestClientBuilder().newBuilder().build();

        cli.open("nlpcraft.weather.ex");  // See weather_model.json
    }

    @AfterEach
    void tearDown() throws NCException, IOException {
        if (cli != null)
            cli.close();

        NCEmbeddedProbe.stop();
    }

    @Test
    void test() throws NCException, IOException {
        // Empty parameter.
        assertTrue(cli.ask("").isFailed());

        // Only latin charset is supported.
        assertTrue(cli.ask("El tiempo en España").isFailed());

        // Should be passed.
        checkIntent("What's the local weather forecast?", "fcast", true);
        checkIntent("What's the weather in Moscow?", "curr", true);
        // Can be answered with conversation.
        checkIntent("Chance of snow?", "curr", true);
        checkIntent("Moscow", "curr", true);

        cli.clearConversation();

        // Cannot be answered without conversation.
        assertTrue(cli.ask("Moscow").isFailed());
    }
}
        </pre>
        <p>
            This test is pretty straight forward:
        </p>
        <ul>
            <li>
                On line 47 we open the test client with the model ID (see <code>weather_model.yaml</code>
                file for where we declared it).
            </li>
            <li>
                Test on line 59 is where we issue our test sentences and we should see
                the confirmation messages in our test console output.
            </li>
        </ul>
        <div class="bq info">
            <p><b>Embedded Prove</b></p>
            <p>
                This test uses <a href="/tools/embedded_probe.html">embedded probe</a> which automatically
                start and stops the data probe from within the tests itself. See lines 43 and 55 for details.
            </p>
            <p>
                <b>NOTE:</b> when using test you don't need to start data probe standalone in a previous step.
            </p>
        </div>
        <p>
            Right click on this class in the project view and run it. You should be getting standard output in
            JUnit panel as well as the output in the data probe console.
        </p>
    </section>
    <section>
        <h2 class="section-title">Done! 👌</h2>
        <p>
            You've created pretty full featured weather bot data model, deployed it into the data probe, started the
            REST server and tested this model using JUnit 5 and the built-in test framework.
        </p>
    </section>
</div>
<div class="col-md-2 third-column">
    <ul class="side-nav">
        <li class="side-nav-title">On This Page</li>
        <li><a href="#overview">Overview</a></li>
        <li><a href="#new_project">New Project</a></li>
        <li><a href="#add_nlpcraft">Add NLPCraft</a></li>
        <li><a href="#model">Data Model</a></li>
        <li><a href="#code">Model Class</a></li>
        <li><a href="#tools">External Tools</a></li>
        <li><a href="#start_probe">Start Probe <sub>opt</sub></a></li>
        <li><a href="#start_server">Start Server</a></li>
        <li><a href="#testing">Testing</a></li>
        {% include quick-links.html %}
    </ul>
</div>