| --- |
| active_crumb: Shot-Term Memory |
| layout: documentation |
| id: short_term_memory |
| --- |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| <div id="short-term-memory" class="col-md-8 second-column"> |
| <section id="overview"> |
| <h2 class="section-title">Conversation <span class="amp">&</span> STM <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| NLPCraft provides automatic conversation management right out of the box that is fully integrated with |
| <a href="/intent-matching.html">intent matching</a>. Conversation management is based on the idea of short-term memory (STM) - |
| the mechanism by which NLPCraft "remembers" the context of the conversation for a certain amount of time and certain |
| depth of the conversation. STM is automatically maintained by NLPCraft per each user and data model combination. |
| Intent matching algorithm can "recall" the missing tokens from STM when trying to find a match for the |
| conversational terms. |
| </p> |
| <p> |
| Why is this so important? |
| Maintaining conversation state is necessary for effective context resolution, so that users |
| could ask, for example, the following sequence of questions using example weather model: |
| </p> |
| <dl class="stm-example"> |
| <dd><i class="fa fa-fw fa-angle-right"></i>What’s the weather in London today?</dd> |
| <dt> |
| <p> |
| User gets the current London’s weather. |
| STM is empty at this moment so NLPCraft expects to get all necessary information from |
| the user sentence. Meaningful parts of the sentence get stored in STM. |
| </p> |
| <div class="stm-state"> |
| <div class="stm"> |
| <label><b>STM</b> Before:</label> |
| <span> </span> |
| </div> |
| <div class="stm"> |
| <label><b>STM</b> After:</label> |
| <span>weather</span> |
| <span>london</span> |
| <span>today</span> |
| </div> |
| </div> |
| </dt> |
| <dd><i class="fa fa-fw fa-angle-right"></i>And what about Berlin?</dd> |
| <dt> |
| <p> |
| User gets the current Berlin’s weather. |
| The only useful data in the user sentence is name of the city <code>Berlin</code>. But since |
| NLPCraft now has data from the previous question in its STM it can safely deduce that we |
| are asking about <code>weather</code> for <code>today</code>. |
| <code>Berlin</code> overrides <code>London</code> in STM. |
| </p> |
| <div class="stm-state"> |
| <div class="stm"> |
| <label><b>STM</b> Before:</label> |
| <span>weather</span> |
| <span>London</span> |
| <span>today</span> |
| </div> |
| <div class="stm"> |
| <label><b>STM</b> After:</label> |
| <span>weather</span> |
| <span><b>berlin</b><br/><i>london</i></span> |
| <span>today</span> |
| </div> |
| </div> |
| </dt> |
| <dd><i class="fa fa-fw fa-angle-right"></i>Next week forecast?</dd> |
| <dt> |
| <p> |
| User gets the next week forecast for Berlin. |
| Again, the only useful data in the user sentence is <code>next week</code> and <code>forecast</code>. |
| STM supplies <code>Berlin</code>. <code>Next week</code> override <code>today</code>, and |
| <code>forecast</code> override <code>weather</code> in STM. |
| </p> |
| <div class="stm-state"> |
| <div class="stm"> |
| <label><b>STM</b> Before:</label> |
| <span>weather</span> |
| <span>Berlin</span> |
| <span>today</span> |
| </div> |
| <div class="stm"> |
| <label><b>STM</b> After:</label> |
| <span><b>forecast</b><br/><i>weather</i></span> |
| <span>berlin</span> |
| <span><b>next week</b><br/><i>today</i></span> |
| </div> |
| </div> |
| </dt> |
| </dl> |
| <p> |
| Note that STM is maintained per user and per data model. |
| Conversation management implementation is also smart enough to clear STM after certain |
| period of time, i.e. it “forgets” the conversational context after few minutes of inactivity. |
| Note also that conversational context can also be cleared explicitly |
| using {% scaladoc NCConversation %}. |
| </p> |
| <p> |
| To understand the algorithm behind the STM management let's back up a few steps... |
| </p> |
| </section> |
| <section id="parsing"> |
| <h2 class="section-title">Parsing User Input <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| One of the key objectives when parsing user input sentence for Natural Language Understanding (NLU) is to |
| detect all possible semantic entities, a.k.a <em>named entities</em>. Let's consider a few examples: |
| </p> |
| <ul> |
| <li> |
| <code>"What's the current weather in Tokyo?"</code><br/> |
| This sentence is fully sufficient for the processing |
| since it contains the topic <code>weather</code> as well as all necessary parameters |
| like time (<code>current</code>) and location (<code>Tokyo</code>). |
| </li> |
| <li> |
| <code>"What about Tokyo?"</code><br/> |
| This is an unclear sentence since it does not have the subject of the |
| question - what is it about Tokyo? |
| </li> |
| <li> |
| <code>"What's the weather?"</code><br/> |
| This is also unclear since we are missing important parameters |
| of location and time for our request. |
| </li> |
| </ul> |
| <p> |
| Sometimes we can use default values like the current user's location and the current time (if they are missing). |
| However, this can lead to the wrong interpretation if the conversation has an existing context. |
| </p> |
| <p> |
| In real life, as well as in NLP-based systems, we always try to start a conversation with a fully defined |
| sentence since without a context the missing information cannot be obtained and the sentenced cannot be interpreted. |
| </p> |
| </section> |
| <section id="ne"> |
| <h2 class="section-title">Named Entities <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| Let's take a closer look at the named entities from the above examples: |
| </p> |
| <ul> |
| <li> |
| <code>weather</code> - this is an indicator of the subject of the conversation. Note that it indicates |
| the type of question rather than being an entity with multiple possible values. |
| </li> |
| <li> |
| <code>current</code> - this is an entity of type <code>Date</code> with the value of <code>now</code>. |
| </li> |
| <li> |
| <code>Tokyo</code> - this is an entity of type <code>Location</code> with two values: |
| <ul> |
| <li><code>city</code> - type of the location.</li> |
| <li><code>Tokyo, Japan</code> - normalized name of the location.</li> |
| </ul> |
| </li> |
| </ul> |
| <p> |
| We have two distinct classes of entities: |
| </p> |
| <ul> |
| <li> |
| Entities that have no values and only act as indicators or types. The entity <code>weather</code> is the |
| type indicator for the subject of the user input. |
| </li> |
| <li> |
| Entities that additionally have one or more specific values like <code>current</code> and <code>Tokyo</code> entities. |
| </li> |
| </ul> |
| </section> |
| <section id="incomplete"> |
| <h2 class="section-title">Incomplete Sentences <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| Assuming previously asked questions about the weather in Tokyo (in the span of the ongoing conversation) one |
| could presumably ask the following questions using a <em>shorter, incomplete</em>, form: |
| </p> |
| <ul> |
| <li> |
| <code>"What about Kyoto?</code><br/> |
| This question is missing both the subject and the time. However, we |
| can safely assume we are still talking about current weather. |
| </li> |
| <li> |
| <code>"What about tomorrow?"</code><br/> |
| Like above we automatically assume the weather subject but |
| use <code>Kyoto</code> as the location since it was mentioned the last. |
| </li> |
| </ul> |
| <p> |
| These are incomplete sentences. This type of short-hands cannot be interpreted without prior context (neither |
| by humans nor by machines) since by themselves they are missing necessary information. |
| In the context of the conversation, however, these incomplete sentences work. We can simply provide one or two |
| entities and rely on the <em>"listener"</em> to recall the rest of missing information from a |
| short-term memory. |
| </p> |
| <p> |
| In NLPCraft, the intent-matching logic will automatically try to find missing information in the |
| conversation context (that is automatically maintained). Moreover, it will properly treat such recalled |
| information during weighted intent matching since it naturally has less "weight" than something that was |
| found explicitly in the user input. |
| </p> |
| </section> |
| <section id="stm"> |
| <h2 class="section-title">Short-Term Memory <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| The short-term memory is exactly that - a memory that keeps only small amount of recently used information |
| and that evicts its contents after a short period of inactivity. |
| </p> |
| <p> |
| Let's look at the example from a real life. If you called your friend in a couple of hours asking <code>"What about a day after?"</code> |
| (still talking about weather in Kyoto) - he'll likely be thoroughly confused. The conversation is timed out, and |
| your friend has lost (forgotten) its context. You will have to explain again to your confused friend what is that you are asking about... |
| </p> |
| <p> |
| NLPCraft has a simple rule that 5 minutes pause in conversation leads to the conversation context reset. However, |
| what happens if we switch the topic before this timeout elapsed? |
| </p> |
| </section> |
| <section id="ctx-switch"> |
| <h2 class="section-title">Context Switch <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| Resetting the context by the timeout is, obviously, not a hard thing to do. What can be trickier is to detect |
| when conversation topic is switched and the previous context needs to be forgotten to avoid very |
| confusing interpretation errors. It is uncanny how humans can detect such switch with seemingly no effort, and yet |
| automating this task by the computer is anything but effortless... |
| </p> |
| <p> |
| Let's continue our weather-related conversation. All of a sudden, we ask about something completely different: |
| </p> |
| <ul> |
| <li> |
| <code>"How much is mocha latter at Starbucks?"</code><br/> |
| At this point we should forget all about previous conversation about weather and assume going forward |
| that we are talking about coffee in Starbucks. |
| </li> |
| <li> |
| <code>"What about Peet's?"</code><br/> |
| We are talking about latter at Peet's. |
| </li> |
| <li> |
| <code>"...and croissant?"</code><br/> |
| Asking about Peet's crescent-shaped fresh rolls. |
| </li> |
| </ul> |
| <p> |
| Despite somewhat obvious logic the implementation of context switch is not an exact science. Sometimes, you |
| can have a "soft" context switch where you don't change the topic of the conversation 100% but yet sufficiently |
| enough to forget at least some parts of the previously collected context. NLPCraft has a built-in algorithm |
| to detect the hard switch in the conversation. You can also use {% scaladoc NCConversation %} to perform a |
| selective reset on the conversation in case of "soft" switch. |
| </p> |
| <p> |
| See {% scaladoc NCConversation %} interface for API details for STM management. |
| </p> |
| </section> |
| <section id="override"> |
| <h2 class="section-title">Overriding Entities <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| As we've seen above one named entity can replace or override an older entity in the STM, e.g. <code>"Peet's"</code> |
| replaced <code>"Starbucks"</code> in our previous questions. <b>The actual algorithm that governs this logic is one |
| of the most important part of STM implementation.</b> In human conversations we perform this logic seemingly |
| subconsciously — but the computer algorithm to do it is not that trivial. Let's see how it is done in NLPCraft. |
| </p> |
| <p> |
| One of the important supporting design decision is that an entity can belong to one or more groups. You can think of |
| groups as types, or classes of entities (to be mathematically precise these are the categories). The entity's |
| membership in such groups is what drives the rule of overriding. |
| </p> |
| <p> |
| Let's look at the specific example. |
| </p> |
| <p> |
| Consider a data model that defined 3 entities: |
| </p> |
| <ul> |
| <li> |
| <code>"sell"</code> (with synonym <code>"sales"</code>) |
| </li> |
| <li> |
| <code>"buy"</code> (with synonym <code>"purchase"</code>) |
| </li> |
| <li> |
| <code>"best_employee"</code> (with synonyms like <code>"best"</code>, <code>"best employee"</code>, <code>"best colleague"</code>) |
| </li> |
| </ul> |
| <p> |
| Our task is to support for following conversation: |
| </p> |
| <ul> |
| <li> |
| <code>"Give me the sales data"</code><br/> |
| We return sales information since we detected <code>"sell"</code> entity by its synonym <code>"sales"</code>. |
| </li> |
| <li> |
| <code>"Who was the best?"</code><br/> |
| We return the best salesmen since we detected <code>"best_employee"</code> and we should pick <code>"sell"</code> entity from the STM. |
| </li> |
| <li> |
| <code>"OK, give me the purchasing report now."</code><br/> |
| This is a bit trickier. We should return general purchasing data and not a best purchaser employee. |
| It feels counter-intuitive but we should NOT take <code>"best_employee"</code> entity from STM and, in fact, we should remove it from STM. |
| </li> |
| <li> |
| <code>"...and who's the best there?"</code><br/> |
| Now, we should return the best purchasing employee. We detected <code>"best_employee"</code> entity and we should pick <code>"buy"</code> entity from STM. |
| </li> |
| <li> |
| <code>"One more time - show me the general purchasing data again"</code><br/> |
| Once again, we should return a general purchasing report and ignore (and remove) <code>"best_employee"</code> from STM. |
| </li> |
| </ul> |
| </section> |
| <section id="rule"> |
| <h2 class="section-title">Overriding Rule <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| Here's the rule we developed at NLPCraft and have been successfully using in various models: |
| </p> |
| <div class="bq success"> |
| <div class="bq-idea-container"> |
| <div><div>💡</div></div> |
| <div> |
| <b>Overriding Rule</b> |
| <p> |
| The entity will override other entity or entities in STM that belong to the same group set or its superset. |
| </p> |
| </div> |
| </div> |
| </div> |
| <p> |
| In other words, an entity with a smaller group set (more specific one) will override entity with the same |
| or larger group set (more generic one). |
| Let's consider an entity that belongs to the following groups: <code>{G1, G2, G3}</code>. This entity: |
| </p> |
| <ul> |
| <li> |
| WILL override existing entity belonging to <code>{G1, G2, G3}</code> groups (same set). |
| </li> |
| <li> |
| WILL override existing entity belonging to <code>{G1, G2, G3, G4}</code> groups (superset). |
| </li> |
| <li> |
| WILL NOT override existing entity belonging to <code>{G1, G2}</code> groups. |
| </li> |
| <li> |
| WIL NOT override existing entity belonging to <code>{G10, G20}</code> groups. |
| </li> |
| </ul> |
| <p> |
| Let's come back to our sell/buy/best example. To interpret the questions we've outlined above we need to |
| have the following 4 intents: |
| </p> |
| <ul> |
| <li><code>intent=sale term={# == 'sale'}</code></li> |
| <li><code>intent=best_sale_person term={# == 'sale'} term={# == 'best_employee'}</code></li> |
| <li><code>intent=buy term={# == 'buy'}</code></li> |
| <li><code>intent=buy_best_person term={# == 'buy'} term={# == 'best_employee'}</code></li> |
| </ul> |
| <p> |
| (this is actual <a href="/intent-matching.html">Intent Definition Language</a> (IDL) used by NLPCraft - |
| <code>term</code> here is basically what's often referred to as a slot in other systems). |
| </p> |
| <p> |
| We also need to properly configure groups for our entities (names of the groups are arbitrary): |
| </p> |
| <ul> |
| <li>Entity <code>"sell"</code> belongs to group <b>A</b></li> |
| <li>Entity <code>"buy"</code> belongs to group <b>B</b></li> |
| <li>Entity <code>"best_employee"</code> belongs to groups <b>A</b> and <b>B</b></li> |
| </ul> |
| <p> |
| Let’s run through our example again with this configuration: |
| </p> |
| <ul> |
| <li> |
| <code>"Give me the sales data"</code> |
| <ul> |
| <li>We detected entity from group <b>A</b>.</li> |
| <li>STM is empty at this point.</li> |
| <li>Return general sales report.</li> |
| <li>Store <code>"sell"</code> entity with group <b>A</b> in STM.</li> |
| </ul> |
| </li> |
| <li> |
| <code>"Who was the best?"</code> |
| <ul> |
| <li>We detected entity belonging to groups <b>A</b> and <b>B</b>.</li> |
| <li>STM has entity belonging to group <b>A</b>.</li> |
| <li><b>{A, B}</b> does NOT override <b>{A}</b>.</li> |
| <li>Return best salesmen report.</li> |
| <li>Store detected <code>"best_employee"</code> entity.</li> |
| <li>STM now has two entities with <b>{A}</b> and <b>{A, B}</b> group sets.</li> |
| </ul> |
| </li> |
| <li> |
| <code>"OK, give me the purchasing report now."</code> |
| <ul> |
| <li>We detected <code>"buy"</code> entity with group <b>A</b>.</li> |
| <li>STM has two entities with <b>{A}</b> and <b>{A, B}</b> group sets.</li> |
| <li><b>{A}</b> overrides both <b>{A}</b> and <b>{A, B}</b>.</li> |
| <li>Return general purchasing report.</li> |
| <li>Store <code>"buy"</code> entity with group <b>A</b> in STM.</li> |
| </ul> |
| </li> |
| </ul> |
| </section> |
| <section id="switch"> |
| <h2 class="section-title">Explicit Context Switch <a href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></h2> |
| <p> |
| In some cases you may need to explicitly clear the conversation STM without relying on algorithmic behavior. |
| It happens when current and new topic of the conversation share some of the same entities. Although at first |
| it sounds counter-intuitive there are many examples of that in day to day life. |
| </p> |
| <p> |
| Let’s look at this sample conversation: |
| </p> |
| <ul> |
| <li> |
| <b>Q</b>: <code>"What the weather in Tokyo?"</code><br/> |
| <b>A</b>: Current weather in Tokyo... |
| </li> |
| <li> |
| <b>Q</b>: <code>"Let’s do New York after all then!"</code><br/> |
| <b>A</b>: Without an explicit conversation reset we would return current New York weather 🤔 |
| </li> |
| </ul> |
| <p> |
| The second question was about going to New York (booking tickets, etc.). In real life - your |
| counterpart will likely ask what you mean by "doing New York after all" and you’ll have to explain |
| the abrupt change in the topic. |
| You can avoid this confusion by simply saying: "Enough about weather! Let’s talk about this weekend plans" - after |
| which the second question becomes clear. That sentence is an explicit context switch which you can also detect |
| in the NLPCraft model. |
| </p> |
| <p> |
| In NLPCraft you can also explicitly reset conversation context through |
| {% scaladoc NCConversation %} interface or by switching the model on the request. |
| </p> |
| </section> |
| </div> |
| <div class="col-md-2 third-column"> |
| <ul class="side-nav"> |
| <li class="side-nav-title">On This Page</li> |
| <li><a href="#overview">Conversation <span class="amp">&</span> STM</a></li> |
| <li><a href="#parsing">Parsing User Input</a></li> |
| <li><a href="#ne">Named Entities</a></li> |
| <li><a href="#incomplete">Incomplete Sentences</a></li> |
| <li><a href="#stm">Short-Term-Memory</a></li> |
| <li><a href="#ctx-switch">Context Switch</a></li> |
| <li><a href="#override">Overriding Entities</a></li> |
| <li><a href="#rule">Overriding Rule</a></li> |
| <li><a href="#switch">Context Switch</a></li> |
| {% include quick-links.html %} |
| </ul> |
| </div> |
