examples/weather_bot.html

---
active_crumb: Weather Bot <code><sub>ex</sub></code>
layout: documentation
id: weather_bot
fa_icon: fa-cube
---

<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<div class="col-md-8 second-column example">
    <section id="overview">
        <h2 class="section-title">Overview <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            This example demonstrates relatively complete NLI-based weather service with JSON output and a non-trivial
            intent matching logic. It uses <a target="new" href="https://openweathermap.org/api/one-call-api">OpenWeather</a>
            REST service for the actual weather information.
        </p>
        <div class="bq info">
            <p>
                <b>NOTE:</b> you must provide OpenWeather API key in <code>OWM_API_KEY</code> system property when running
                the data probe. See <a target=_ href="https://openweathermap.org/api">https://openweathermap.org/api</a> for more information.
            </p>
        </div>
        <p>
            <b>Complexity:</b> <span class="complexity-two-star"><i class="fas fa-square"></i> <i class="fas fa-square"></i> <i class="far fa-square"></i></span><br/>
            <span class="ex-src">Source code: <a target="github" href="https://github.com/apache/incubator-nlpcraft/tree/master/nlpcraft-examples/weather">GitHub <i class="fab fa-fw fa-github"></i></a><br/></span>
            <span class="ex-review-all">Review: <a target="github" href="https://github.com/apache/incubator-nlpcraft/tree/master/nlpcraft-examples">All Examples at GitHub <i class="fab fa-fw fa-github"></i></a></span>
        </p>
    </section>
    <section id="new_project">
        <h2 class="section-title">Create New Project <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            You can create new Java project in many ways - we'll use <a href="/tools/script.html">NLPCraft CLI</a>
            to accomplish this task:
        </p>
        <nav>
            <div class="nav nav-tabs" role="tablist">
                <a class="nav-item nav-link active" data-toggle="tab" href="#nav-prj-cmd" role="tab">Command</a>
                <a class="nav-item nav-link" data-toggle="tab" href="#nav-prj-out" role="tab">Output <i class="fa fa-desktop output"></i></a>
            </div>
        </nav>
        <div class="tab-content">
            <div class="tab-pane fade show active" id="nav-prj-cmd" role="tabpanel">
                <pre class="brush: bash">
                    $ bin/nlpcraft.sh gen-project --baseName=Weather --outputDir=~ --pkgName=demo --mdlType=json
                </pre>
                <p>
                    <b>NOTES:</b>
                </p>
                <ul>
                    <li>
                        New project created in <code>/home/Weather</code> directory.
                    </li>
                    <li>
                        <code>gen-project</code> command defaults to Java and  Maven as its built tool.
                    </li>
                    <li>
                        Run <code class="script">bin/nlpcraft.sh help --cmd=gen-project</code> to get a full help on <code>gen-project</code> command.
                    </li>
                    <li>
                        <a href="/tools/script.html">NLPCraft CLI</a> is available as <code>nlpcraft.sh</code> for
                        <i class="fab fa-fw fa-linux"></i> and <code>nlpcraft.cmd</code>
                        for <i class="fab fa-fw fa-windows"></i>.
                    </li>
                </ul>
            </div>
            <div class="tab-pane fade show" id="nav-prj-out" role="tabpanel">
                <p></p>
                <img alt="" class="img-fluid" src="/images/weather_bot_fig1.png">
            </div>
        </div>
    </section>
    <section id="model">
        <h2 class="section-title">Data Model <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            We are going to start with declaring the static part of our model using JSON which we will later load using
            <code>NCModelFileAdapter</code> in our Java-based model implementation. Open <code>src/main/resources/<b>weather.json</b></code>
            and replace its content with the following JSON:
        </p>
        <pre class="brush: js, highlight: [10, 18, 28, 38]">
{
  "id": "nlpcraft.weather.ex",
  "name": "Weather Example Model",
  "version": "1.0",
  "description": "Weather example model.",
  "macros": [
  ],
  "elements": [
    {
      "id": "wt:phen",
      "description": "Weather phenomenon.",
      "synonyms": [
        "{high sea|severe weather|hail|heat wave|cold wave|derecho|supercell|avalanche|cyclone|wildfire|landslide|firestorm|dust storm|thunder snow|winter storm|cloudburst|shower|condensation|precipitation|drizzle|rainstorm|rain storm|rainfall|rain|storm|sun|sunshine|cloud|hot|cold|dry|wet|wind|hurricane|typhoon|sand-storm|sand storm|tornado|humid|fog|snow|smog|black ice|haze|thundershower|thundersnow|sleet|drought|wildfire|blizzard|avalanche|mist|thunderstorm}",
        "{weather {condition|temp|temperature|data|_}|condition|temp|temperature}"
      ]
    },
    {
      "id": "wt:hist",
      "description": "History (past) indicator.",
      "groups": [
        "indicator"
      ],
      "synonyms": [
        "{history|past|previous}"
      ]
    },
    {
      "id": "wt:curr",
      "description": "Current indicator.",
      "groups": [
        "indicator"
      ],
      "synonyms": [
        "{current|today|now|right now}"
      ]
    },
    {
      "id": "wt:fcast",
      "description": "Forecast (future) indicator.",
      "groups": [
        "indicator"
      ],
      "synonyms": [
        "{future|forecast|prognosis|prediction}"
      ]
    }
  ]
}
        </pre>
        <p>There are number of important points here:</p>
        <ul>
            <li>
                <code>Line 10</code> defines an element <code>wt:phen</code> for various weather phenomenon.
            </li>
            <li>
                <code>Line 18</code> defines an element <code>wt:hist</code> whose presence will indicate the
                request for the past (history) weather information.
            </li>
            <li>
                <code>Line 28</code> defines an element <code>wt:curr</code> whose presence will indicate the
                request for the current (local) weather information.
            </li>
            <li>
                <code>Line 38</code> defines an element <code>wt:fcast</code> whose presence will indicate the
                request for the future (forecast) weather information.
            </li>
        </ul>
        <p>
            Now that our model is ready let's create a Java class that would load this model and define the intent
            that uses the model elements we have just defined.
        </p>
        <h2 class="section-title">Model Class <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            Below is a full source code for our implementation. Note that this code uses several
            external tools for IP-based geo-location and the weather information
            provider (<a target=_ href="https://openweathermap.org/api">OpenWeather</a>). Note also that despite its apparent simplicity the model logic implementation is non-trivial.
            Significant portion of the code deals with a complex <b>temporal and geographical ambiguity</b>, i.e. the sentences like the these:
        </p>
        <dl>
            <dt>
                <code>'show my local weather'</code> vs <code>'show local Boston weather'</code>
            </dt>
            <dd>
                In the first sentence the word <code>local</code> indicates that the user is asking about her local weather
                (i.e. the weather for the location based on the IP address of the user's REST agent). However, in the
                second sentence the same word <code>local</code> is effectively ignored as user clearly indicated the city
                for the weather request (Boston).
            </dd>
            <dt>
                <code>'show weather forecast'</code> vs <code>'show weather forecast for today'</code> vs <code>'show last week weather forecast'</code>
            </dt>
            <dd>
                In these three sentences the word <code>forecast</code> defines future, current and past timeframe for the
                weather request. It acts as a future indicator in the first sentence and is effectively ignored in the second
                and third sentences.
            </dd>
        </dl>
        <div class="bq info">
            <p><b>Temporal <i class="amp">&amp;</i> Geographical Ambiguity</b></p>
            <p>
                Despite seeming triviality of these ambiguities for the human comprehension, these represent a significant problem
                for the most deep learning neural networks unless specifically trained to resolve these particular cases. Procedural
                (deterministic) approach - as shown below - often yields a dramatically simpler and more robust solution, albeit the
                one that works only in a specific context.
            </p>
        </div>
        <p>
            Open <code>src/main/java/demo/<b>Weather.java</b></code> file and replace its content with the
            following code:
        </p>
        <pre class="brush: java, highlight: [30, 43, 130, 90, 132, 133, 134, 177]">
package demo;

import com.google.gson.Gson;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.nlpcraft.examples.weather.openweathermap.OpenWeatherMapException;
import org.apache.nlpcraft.examples.weather.openweathermap.OpenWeatherMapService;
import org.apache.nlpcraft.utils.keycdn.GeoManager;
import org.apache.nlpcraft.utils.keycdn.beans.GeoDataBean;
import org.apache.nlpcraft.model.NCIntent;
import org.apache.nlpcraft.model.NCIntentMatch;
import org.apache.nlpcraft.model.NCIntentSample;
import org.apache.nlpcraft.model.NCIntentTerm;
import org.apache.nlpcraft.model.NCModelFileAdapter;
import org.apache.nlpcraft.model.NCRejection;
import org.apache.nlpcraft.model.NCResult;
import org.apache.nlpcraft.model.NCToken;

import java.time.Instant;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;

import static java.time.temporal.ChronoUnit.DAYS;

public class WeatherModel extends NCModelFileAdapter {
    // System property for OpenWeatherMap API key.
    public final String OWM_API_KEY = "OWM_API_KEY";

    // Please register your own account at https://openweathermap.org/api and
    // replace this demo token with your own.
    // We are using the One Call API (https://openweathermap.org/api/one-call-api) in this example
    private final OpenWeatherMapService openWeather;

    private final GeoManager geoMrg = new GeoManager();
    private static final int DAYS_SHIFT_BACK = 5;
    private static final int DAYS_SHIFT_FORWARD = 7;
    private static final Gson GSON = new Gson();
    private static final Set&lt;String&gt; LOCAL_WORDS = new HashSet&lt;&gt;(Arrays.asList("my", "local", "hometown"));

    private Pair&lt;Double, Double&gt; prepGeo(NCIntentMatch ctx, Optional&lt;NCToken&gt; geoTokOpt) throws NCRejection {
        if (geoTokOpt.isPresent()) {
            NCToken geoTok = geoTokOpt.get();

            Map&lt;String, Object&gt; cityMeta = geoTok.meta("nlpcraft:city:citymeta");

            Double lat = (Double)cityMeta.get("latitude");
            Double lon = (Double)cityMeta.get("longitude");

            if (lat == null || lon == null) {
                String city = geoTok.meta("nlpcraft:city:city");

                throw new NCRejection(String.format("Latitude and longitude not found for: %s", city));
            }

            return Pair.of(lat, lon);
        }

        Optional&lt;GeoDataBean&gt; geoOpt = geoMrg.get(ctx.getContext().getRequest());

        if (geoOpt.isEmpty())
            throw new NCRejection("City cannot be determined.");

        // Manually process request for local weather. We need to separate between 'local Moscow weather'
        // and 'local weather' which are different. Basically, if there is word 'local/my/hometown' in the user
        // input and there is no city in the current sentence - this is a request for the weather at user's
        // current location, i.e. we should implicitly assume user's location and clear conversion context.
        // In all other cases - we take location from either current sentence or conversation STM.

        // NOTE: we don't do this separation on intent level as it is easier to do it here instead of
        // creating more intents with almost identical callbacks.

        @SuppressWarnings("SuspiciousMethodCalls")
        boolean hasLocalWord =
            ctx.getVariant().stream().anyMatch(t -&gt; LOCAL_WORDS.contains(t.meta("nlpcraft:nlp:origtext")));

        if (hasLocalWord)
            // Because we implicitly assume user's current city at this point we need to clear
            // 'nlpcraft:city' tokens from conversation since they would no longer be valid.
            ctx.getContext().getConversation().clearStm(t -&gt; t.getId().equals("nlpcraft:city"));

        // Try current user location.
        GeoDataBean geo = geoOpt.get();

        return Pair.of(geo.getLatitude(), geo.getLongitude());
    }

    @NCIntent(
        "intent=req " +
        "term~{tok_id() == 'wt:phen'}* " + // Zero or more weather phenomenon.
        "term(ind)~{" +
            "@isIndicator = has(tok_groups(), 'indicator') " + // Just to demo term variable usage.
            "@isIndicator" +
        "}* " + // Optional indicator words (zero or more).
        "term(city)~{tok_id() == 'nlpcraft:city'}? " + // Optional city.
        "term(date)~{tok_id() == 'nlpcraft:date'}?" // Optional date (overrides indicator words).
    )
    // NOTE: each samples group will reset conversation STM during auto-testing.
    @NCIntentSample({
        "Current forecast?",
        "Chance of rain in Berlin now?"
    })
    // NOTE: each samples group will reset conversation STM during auto-testing.
    @NCIntentSample({
        "Moscow forecast?",
        "Chicago history"
    })
    // NOTE: each samples group will reset conversation STM during auto-testing.
    @NCIntentSample({
        "What's the local weather forecast?",
        "What's the weather in Moscow?",
        "What's the current forecast for LA?",
        "What is the weather like outside?",
        "How's the weather?",
        "What's the weather forecast for the rest of the week?",
        "What's the weather forecast this week?",
        "What's the weather out there?",
        "Is it cold outside?",
        "Is it hot outside?",
        "Will it rain today?",
        "When it will rain in Delhi?",
        "Is there any possibility of rain in Delhi?",
        "Is it raining now?",
        "Is there any chance of rain today?",
        "Was it raining in Beirut three days ago?",
        "How about yesterday?"
    })
    public NCResult onMatch(
        NCIntentMatch ctx,
        @NCIntentTerm("ind") List&lt;NCToken&gt; indToksOpt,
        @NCIntentTerm("city") Optional&lt;NCToken&gt; cityTokOpt,
        @NCIntentTerm("date") Optional&lt;NCToken&gt; dateTokOpt
    ) {
        try {
            Instant now = Instant.now();

            Instant from = now;
            Instant to = now;

            if (indToksOpt.stream().anyMatch(tok -&gt; tok.getId().equals("wt:hist")))
                from = from.minus(DAYS_SHIFT_BACK, DAYS);
            else if (indToksOpt.stream().anyMatch(tok -&gt; tok.getId().equals("wt:fcast")))
                to = from.plus(DAYS_SHIFT_FORWARD, DAYS);

            if (dateTokOpt.isPresent()) { // Date token overrides any indicators.
                NCToken dateTok = dateTokOpt.get();

                from = Instant.ofEpochMilli(dateTok.meta("nlpcraft:date:from"));
                to = Instant.ofEpochMilli(dateTok.meta("nlpcraft:date:to"));
            }

            Pair&lt;Double, Double&gt; latLon = prepGeo(ctx, cityTokOpt); // Handles optional city too.

            double lat = latLon.getLeft();
            double lon = latLon.getRight();

            return NCResult.json(GSON.toJson(from == to ? openWeather.getCurrent(lat, lon) : openWeather.getTimeMachine(lat, lon, from, to)));
        }
        catch (OpenWeatherMapException e) {
            throw new NCRejection(e.getLocalizedMessage());
        }
        catch (NCRejection e) {
            throw e;
        }
        catch (Exception e) {
            throw new NCRejection("Weather provider error.", e);
        }
    }

    /**
     * Loads the model.
     */
    public WeatherModel() {
        // Load model from external JSON file on classpath.
        super("weather_model.json");

        // Try system variable first.
        String apiKey = System.getProperty(OWM_API_KEY);

        if (apiKey == null)
            // Try environment variable next.
            apiKey = System.getenv(OWM_API_KEY);

        if (apiKey == null)
            throw new OpenWeatherMapException(String.format("Provide OpenWeatherMap API key using '-D%s=&lt;your-key-here&gt;' system property.", OWM_API_KEY));

        openWeather = new OpenWeatherMapService(apiKey, 5, 7);
    }

    @Override
    public void onDiscard() {
        openWeather.stop();
    }
}
        </pre>
        <ul>
            <li>
                <code>Line 177</code> loads the model configuration from the external <code>weather_model.json</code>
                file.
            </li>
            <li>
                Method <code>preGeo(...)</code> on the <code>line 43</code> handles the geolocation processing including
                IP-based geo-location and resolution of the geographical ambiguity.
            </li>
            <li>
                <code>Line 130</code> defines a callback for the intent defined on the <code>line 90</code>. Note that
                callback implementation also deals with the temporal ambiguity.
            </li>
            <li>
                <code>Lines 132-134</code> define formal callback method parameters that correspond
                to the intent's terms (see <code>line 90</code>).
            </li>
            <li>
                <code>Lines 101-129</code> define input sentence samples that are used for both documentation and
                auto-validation purposed (see <a href="#testing">testing</a> second for details).
            </li>
            <li>
                Make sure to provide you <a target="new" href="https://openweathermap.org/api/one-call-api">OpenWeather API</a>
                key on the <code>line 30</code>.
            </li>
        </ul>
    </section>
    <section id="build_project">
        <h2 class="section-title">Build Project <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            Once we have our model ready let's go to <code>~/Weather</code> directory and run the Maven build:
        </p>
        <pre class="brush: bash">
            $ cd ~/Weather
            $ mvn clean package
        </pre>
        <p>
            At this stage we have our project built and we are ready to start testing.
        </p>
    </section>
    <section id="start_server">
        <h2 class="section-title">Start Server <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            Run the following command to start local REST server, if it hasn't been started already, from the NLPCraft installation directory:
        </p>
        <nav>
            <div class="nav nav-tabs" role="tablist">
                <a class="nav-item nav-link active" data-toggle="tab" href="#nav-srv-cmd" role="tab">Command</a>
                <a class="nav-item nav-link" data-toggle="tab" href="#nav-srv-out" role="tab">Output <i class="fa fa-desktop output"></i></a>
            </div>
        </nav>
        <div class="tab-content">
            <div class="tab-pane fade show active" id="nav-srv-cmd" role="tabpanel">
                <pre class="brush: bash">
                    $ bin/nlpcraft.sh start-server
                </pre>
            </div>
            <div class="tab-pane fade show" id="nav-srv-out" role="tabpanel">
                <p></p>
                <p>
                    <img class="img-fluid" alt="" src="/images/server-fig1.png">
                </p>
            </div>
        </div>
        <p>
            <b>NOTES:</b>
        </p>
        <ul>
            <li>
                <i style="color: #F39C12" class="fa fa-exclamation-triangle"></i> REST server is a "fire-and-forget" component that you generally need to start it only once.
            </li>
            <li>
                Run <code class="script">bin/nlpcraft.sh help --cmd=start-server</code> to get a full help on this command.
            </li>
            <li>
                <a href="/tools/script.html">NLPCraft CLI</a> is available as <code>nlpcraft.sh</code> for
                <i class="fab fa-fw fa-linux"></i> and <code>nlpcraft.cmd</code>
                for <i class="fab fa-fw fa-windows"></i>.
            </li>
        </ul>
    </section>
    <section id="testing">
        <h2 class="section-title">Testing <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            Remember the <a target="javadoc" href="/apis/latest/org/apache/nlpcraft/model/NCIntentSample.html">@NCIntentSample</a>
            annotation we have used in our model code next to intent definition?
        </p>
        <p>
            Part of the <a href="/tools/test_framework.html">test framework</a>, the auto-validator class <a
                target="javadoc"
                href="/apis/latest/org/apache/nlpcraft/model/tools/test/NCTestAutoModelValidator.html">NCTestAutoModelValidator</a> takes one or more model IDs
            (or class names) and performs validation. Validation consists of starting an  <a href="/tools/embedded_probe.html">embedded probe</a> with a given model,
            scanning for <a target="javadoc" href="/apis/latest/org/apache/nlpcraft/model/NCIntentSample.html">@NCIntentSample</a> and
            <a target="javadoc" href="/apis/latest/org/apache/nlpcraft/model/NCIntentSampleRef.html">@NCIntentSampleRef</a> annotations
            and their corresponding callback methods, submitting each sample input
            sentences from these annotations and checking that resulting intent matches the intent the sample was attached to.
            Note that auto-testing does not require any additional code to be written - the class gathers all required information from the model
            itself.
        </p>
        <p>
            As always, you can launch model auto-validator as any other Java class but we'll use NLPCraft CLI
            to do it more conveniently:
        </p>
        <pre class="brush: bash">
            $ bin/nlpcraft.sh test-model --cp=~/Weather/target/classes --mdls=demo.Weather
        </pre>
        <p>
            <b>NOTES:</b>
        </p>
        <ul>
            <li>
                Run <code class="script">bin/nlpcraft.sh help --cmd=test-model</code> to get a full help on this command.
            </li>
            <li>
                Note that you can use <code>retest-model</code> command in REPL mode to re-run the last model test
                avoiding the retyping of all required parameters.
            </li>
            <li>
                <a href="/tools/script.html">NLPCraft CLI</a> is available as <code>nlpcraft.sh</code> for
                <i class="fab fa-fw fa-linux"></i> and <code>nlpcraft.cmd</code>
                for <i class="fab fa-fw fa-windows"></i>.
            </li>
        </ul>
        <p>
            Look at the output of this command and you will see the test results for all our sample utterances:
        </p>
        <p>
            <img style="max-width: 871px !important;" class="img-fluid" alt="" src="/images/weather-bot-test.png">
        </p>
    </section>
    <section id="rinse">
        <h2 class="section-title">Rinse <span class="amp">&amp;</span> Repeat <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            Typical development cycle consists of:
        </p>
        <ul>
            <li>
                <a href="#model">Modifying the model</a>
            </li>
            <li>
                <a href="#build_project">Re-building the project</a>
            </li>
            <li>
                <a href="#testing">Re-running the test</a>
            </li>
        </ul>
        <p>
            All of these operations can be performed from <a href="/tools/script.html">NLPCraft CLI</a> in REPL mode or from any IDE.
        </p>
        <p>
            NOTE: you don't need to restart REST server every time - it only needs to be started once.
        </p>
    </section>
    <section>
        <h2 class="section-title">Done! 👌 <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
        <p>
            You've created weather bot data model, started the REST server and tested this model using the built-in test framework.
        </p>
    </section>
</div>
<div class="col-md-2 third-column">
    <ul class="side-nav">
        <li class="side-nav-title">On This Page</li>
        <li><a href="#overview">Overview</a></li>
        <li><a href="#new_project">New Project</a></li>
        <li><a href="#model">Data Model</a></li>
        <li><a href="#build_project">Build Project</a></li>
        <li><a href="#start_server">Start Server</a></li>
        <li><a href="#testing">Testing</a></li>
        <li><a href="#rinse">Rinse <span class="amp">&amp;</span> Repeat</a></li>
        {% include quick-links.html %}
    </ul>
</div>