Dialogue Systems: Intent, State Tracking and Conversational Repair
Published:
A turn is interpreted inside a history
User: “Find a table for two tomorrow.” Later: “Actually, make that three.” The second turn does not contain a complete standalone request, but it updates the party size while preserving the date and intent. Treating each turn as an independent classification problem loses that relationship.
An intent label describes an action such as searching or cancelling. Slots describe values such as date, location and party size. The dialogue state records the accumulated interpretation, including unknown values and corrections. A full-text conversation history and a structured state are different representations with different failure modes.
Track updates explicitly
Consider this simplified state sequence:
| Turn | Update | State after update |
|---|---|---|
| “A table for two tomorrow” | intent=search, size=2, date=tomorrow | size=2; date resolved from dialogue time |
| “Near the station” | area=station | same size/date; area=station |
| “Actually three people” | size=3 | size replaced; other constraints retained |
| “Any price is fine” | price=don’t-care | price unconstrained, not unknown |
Unknown and don’t-care must be distinct. Unknown may require a clarification; don’t-care explicitly removes a restriction. A negative statement such as “not near the station” requires a representation for exclusion or a policy for resolving it.
Temporal expressions must be grounded in the user’s conversational time context. Storing the literal string tomorrow and resolving it again a day later changes the requested date.
Maintain uncertainty when needed
A probabilistic state tracker can maintain a belief over possible states. A schematic update is
where \(u_t\) is the new utterance and \(a_{t-1}\) the preceding system action. This is a modelling template: practical trackers may directly predict slot values or distributions from the history instead of implementing this exact factorisation.
The useful idea is to separate what is known from what was guessed. If two candidate dates remain plausible, immediately collapsing to one can create an avoidable downstream mistake.
Choose an action, then phrase it
A policy chooses between actions such as asking for a missing slot, presenting options or confirming a result. Natural-language generation phrases that action. Keeping the chosen action explicit makes it possible to verify that “I found a table” corresponds to an actual successful lookup rather than a fluent guess.
Conversational repair includes corrections, contradictions and requests to revise earlier choices. An update should replace only the affected state fields unless the user changes the overall task. “Make that three” should not silently reset the date.
A simple clarification calculation
Suppose an uncertain slot has probability 0.6 of being correct. Acting with the wrong value has cost ten, while asking a clarification has cost one and resolves the ambiguity in this simplified example. Acting immediately has expected error cost \(0.4\times10=4\), so clarification is preferable.
This is an illustrative decision model, not a universal conversational rule. Real systems must estimate the cost of interruption, the likelihood that the user can answer, and whether a provisional action is reversible. The key point is that confidence should influence the action rather than only appear as an internal number.
Evaluate the conversation, not just the turn
Intent accuracy and slot F1 are useful component metrics. Joint goal accuracy requires the complete state to match the reference at a turn and is sensitive to one incorrect slot. Task success measures whether the conversation actually achieved the requested outcome under the evaluation definition.
A system can have high slot accuracy because most slots remain empty while failing the few values needed to complete the task. It can also produce natural replies while repeatedly losing corrections. Include multi-turn tests with changed values, negation, references to previous options and recovery from misunderstanding.
Split data by dialogue rather than by individual turns. Otherwise later turns from the same conversation can leak the state into evaluation. Record annotation and dataset versions, since corrected state labels can materially change the apparent quality of a tracker.
Recap
- Interpret turns relative to the conversation history.
- Preserve unchanged constraints while applying corrections.
- Separate state, policy action and response realisation.
- Evaluate multi-turn task success alongside component metrics.
References
- Young et al. POMDP-Based Statistical Spoken Dialog Systems: A Review. Proceedings of the IEEE, 2013.
- Budzianowski et al. MultiWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. EMNLP, 2018.