IBM Watson Assistant Now Has Improved Intent Recognition

IBM Watson Assistant Now Has Improved Intent Recognition

Improve Your Chatbot’s Ability To Understand Customers With A New Intent Detection Model

Introduction

IBM Watson Assistant is introducing new functionality at a remarkable cadence. Notably these changes are not disruptive to existing features and augment or compliment existing functionality.

The previous big announcement from Watson Assistant was Actions…prior to the enhanced intent detection model.

November 2020 IBM Watson Assistant released a beta version of their new intent detection model. Intents are the frontline of any conversational interface.

Intents are categories of what users’ intentions are; really the frontline of any chatbot. Accurately classifying the user utterance according to an intent is of utmost importance.

Image for post
Under Skill Options in IBM Watson Assistant the new feature is visible.

Various parameters can be set in each dialog node to determine relevance, and the most popular measure here is intent scores.

What Is New?

According to IBM the new intent detection model is faster and more accurate. It combines:

  • Traditional Machine Learning
  • Transfer learning, and
  • Deep Learning Techniques.

Less training data is required, which is convenient for prototyping.

The new model is available as a beta feature for English dialog and actions skills only.

Once the new feature is activated, Watson Assistant needs to perform training.

Image for post
Training Of The Assistant When Intent Models Are Switched.

Enhanced detection can improve your assistant’s ability to classify what customers say with the right intent, more often.

Re-test your intents after applying the enhanced version. Changing versions will automatically retrain the skill, which might cause slight differences in results. You may keep or switch back to the classic version.

Enabling The Enhanced Intent Detection Model

Firstly, what I find interesting is that the new model is used by Action skills by default and cannot be disabled. This most probably is an indication if what the future holds.

In order to enable the new model in a dialog skill:

  1. From the Skills page, open your skill.
  2. From the skill menu, click Options.
  3. On the Intent detection page, choose Enhanced version.

Test the new intent detection capabilities in the “Try it out” pane.

Image for post
Enabling The Enhanced Intent Detection Model

Testing The Result

One thing to note is the ability to switch back to the classic version should the results yielded not be satisfactory.

Alternatively, if left on, rigorous test is required. One of a few reasons for this is that many dialog nodes have conditions set based not only on the intent detected, bot the score of the intent.

These scores are used as thresholds to determine which node to invoke.

I looked at two scenarios:

  • A non-ambiguous short user utterance.
  • An ambiguous user utterance.

For the utterance: “What are the hours?” The intent score for the correct intent is 0.02 higher. The two next best intents are also marginally lower.

Image for post
Intent Detection Model: Classic Version

This simple tests shows a general improvement in the model.

Image for post
Intent Detection Model: Enhanced Version

The second utterance tested was “Thanks you really were a great help.” Which is an awkward sentence, and ambiguous looking at the list of intents available.

Image for post
Intent Detection Model: Classic Version

Seemingly the new model detected the ambiguate by bringing down the score of the correct intent; #Thanks. The intent #Help moved up.

Image for post
Intent Detection Model: Enhanced Version

Conclusion

With the promise of less training data and improved accuracy, the new model is compelling.

A drawback is the lack of insight or understanding of what is happening under the hood. For others this might be an advantage.

The fact that IBM Watson Assistant is evolving at this pace bode well for organizations having invested in the technology.



Leave a Reply

Your email address will not be published. Required fields are marked *