The Chatbot Revolution and the Role of Multilingual AI

When we think about the past, the chatbot has always revolved around the world of customer support, and often multilingual customer support focuses on content on a screen or in a document, which has been known to create significant challenges to overcome.

A lot of companies have thousands of pages of customer support content, so with that comes a high volume of work. There’s also very limited engagement. We (language service providers) will translate pages of content that nobody will ever look at. The content becomes quickly outdated, as our products and services evolve, then all that content needs to be updated, retranslated, and edited. As a result of all of this, it’s very expensive, and a great deal of clients are reluctant to spend money on localizing support content.

Our response as an industry was to use machine translation (MT), which has been a smart approach. But now the industry is turning to chatbots as they provide a more pleasurable, seamless user experience.

"The goal has been to use the technology to make the translation process itself less expensive, less time-intensive, faster turnarounds, etc. But I think we’ve reached the limit of that model largely because conversational AI or chatbots have emerged as an alternative. The reason we see this emerging as the new future is that it offers just a more effective and efficient experience.” - Aaron Schliem, Director of AI Enablement, Welocalize

How Do We Respond to This Demand?  

As we’ve seen with the MT revolution, often our response can be fear. How are we going to survive? Is this going to put us out of business? Are we not going to have any more work to do because these machines are going to be able to do our jobs for us? It’s now a matter of the language services industry evolving. There are ways that we can adapt to this new reality and continue to have important roles to play.

The Four Types of Chatbots 

No matter the complexity of the chatbot there are opportunities at every level for LSPs, translators, and other language professionals to engage and be a part of this revolution to deliver more effective and efficient solutions. Here are the various approaches to developing chatbots and advanced AI and NLP, which are critical  for global brands seeking to enhance multilingual communicate and engage with customers using digital agents.

Simple Rule-Based Chatbots

Two examples of these are interactive FAQs and form-based interactions. With interactive FAQs, it’s a simple system where there are a set of keywords, pre-defined, mapped into the system and then if a user writes a sentence that has that keyword in it, it will direct them to a specific set of information related to that keyword. Form-based interactions are very similar, as you are essentially being asked to fill out a form, but it feels a little more natural because that chatbot is talking to you and prompting you.

In these scenarios, there are several relevant language services that apply. There’s the standard translation of the FAQ content, and this is very similar to the kind of work the localization industry does right now. There’s also transcreation of search keywords and phrases, which is a little more creative. This involves predicting a list of keywords that we’re going to assume humans will use based on a chosen topic or action. For example, if a person wanted to change their flight information, a translator would need to create a list of keywords and phrases surrounding this topic to ensure the chatbot has sufficient breadth of keyword options to cover many human language patterns and get them to the right answers.

Predicting human noise variations is also a valuable exercise. Here we’re not looking only at the transcreation of one keyword but the ten other ways that somebody might say it.

In the world of chatbots, our goal is to predict what kinds of things humans really say? Humans make mistakes, we all know that, and when I say noise variations I’m referring to the kinds of spelling mistakes people make. What kinds of typos do people make? What kind of texting abbreviations do people use when they're typing into an interactive chatbot on a website? We need to predict those kinds of things so that then, even in those simple rule-based systems, the rules are prepared to ingest that kind of human noise.” - Aaron Schliem, Director of AI Enablement, Welocalize

Complex Rule-Based Chatbots

These chatbot systems are based around decision trees, and these now begin to feel more human. Essentially what we’re doing is we’re mapping human inputs and we’re taking other content-specific information and then creating dialogues with the users. The Kuki chatbot is a good example of this. It was developed by Steve Worswick, and it has been a multiple Loebner Prize winner.

The chatbot is built using AIML (artificial intelligence markup language) and while the experience for many people feels very human, at the end of the day it is a rule-based system. In this system the AIML allows designers of chatbots to do things like normalize inputs, and by that, we mean writing rules that say, if the user types in “A, B, C, or D”, convert those into “E” and then use that as the input into your system.

Some relevant services for these types of chatbots include cultural consulting, which is thinking about how humans will step through a process based on their culture and applying this to the decision tree. The rules on how you have a conversation won’t be the same in every culture, so that’s part of adapting that complex rule-based system. You would need to think about common language patterns, which involves thinking through inputs and applying knowledge of each language’s own grammar, syntax, and word order to predict the way a person would interact.

Basic Machine Learning Chatbots

One example of this kind of chatbot structure is question answering. We want to train an engine to recognize query patterns and generate the best possible results.

Intent-based systems are similar. We want to think about which queries a user might make that map to a particular intention. What is it that they want to achieve with this query? By having the machine learn which queries align with which intents we can then more easily route users to either information or to database endpoints where we can help to facilitate interactions.

Complex Hybrid Chatbots 

These types of chatbots are a combination of both rules and machine learning. The “rule” element is often related to building grammars, which is something this industry is well suited to perform. The machine learning piece is also something that we can contribute to as in these models we are essentially building language models that are capable of performing at an NLU (natural language understanding) level so that the chatbot actually “understands” what you’re trying to say and maps this to a particular intent. These chatbots are having conversations with you and they’re adapting on the fly. To do this there is a whole array of language services that may need to be applied.

The modeling of conversation patterns is incredibly important for these chatbots as they need to understand how a conversation would proceed, and what are different kinds of conversations can arise. For example, we might start a conversation by asking to change our flight, we might then ask to check our frequent flyer status; those are complex conversations that need to be modeled and there’s a role for humans to play in generating data to teach these chatbots how to do this.

We need to look at the evolution of the multilingual chatbot as yet another innovative step forward towards better customer communication and engagement. And for opening new and exciting areas for language professionals to evolve and develop their skills for the future.

There are roles for us to play that are actually really exciting. For those of you who are into linguistic anthropology, sociolinguistics, a lot of the things that I think drive people to get into this industry in the first place I think there’s an exciting space for us to grow and develop.” - Aaron Schliem, Director of AI Enablement, Welocalize

This article is based on Aaron Schliem's session, The Chatbot: Revolution and the Role of Multilingual AIpresented at GALA Connect 2021.

For more information on Welocalize AI Services, contact the team here.