Chatbot interest is at an all-time high for businesses looking to more optimally engage with potential and existing customers.  Desired outcomes range from a better customer support experience to question answering to deflect calls, quick product & service research inquiry summarization, and ensuring interest converts into revenue on an eCommerce site.  

But is this actually happening with the chatbot you chose?  Should it be happening more? Do you even have a chatbot on your site?  To help with focusing on what matters, I’ve provided recommendations below to help evaluate your current state and make a decision focused on desired business outcomes. The measure of value for any software or AI tool investment is the following, usually in order of: 

• Increased revenue 
• Mitigated risk 
• Reduced costs

So, how do you assess and rank chatbots for your business model or use case to meet your growth and profit goals?

Start with the Different Levels of Chatbot Intelligence (from most basic to most advanced):

• Rule-based chatbots: only respond to a limited set of pre-programmed questions and answers.
• Pattern-matching chatbots: recognize patterns in language and use the patterns to generate responses to a broader range of queries and task requests.
• Machine learning chatbots: can learn from human interactions and incorporate these exchanges to improve their answers.
• Natural language processing (NLP) chatbots: understand what humans really mean when they submit a lexical inquiry, and based upon this semantic know-how, have more fluid and realistic conversations with humans.
• Artificial intelligence (AI) chatbots: most advanced chatbot category using AI to understand and respond to complex questions and requests.

Understanding the various tiers of chatbot intelligence is essential when aligning with your targeted customer experience. More intelligent chatbots are designed to properly address complicated tasks along with enabling a more personalized approach. This alignment will directly impact (either positively or negatively) customer satisfaction, cost of doing business, and effectiveness of the outcome (converting a sale, deflecting an expensive call, protecting brand reputation).

With these levels, there are 2 main applications of chatbot technology:

1. Human Agent Q&A Assistant: self-service Q&A for call center human agent – real-time feedback to avoid escalation
2. Digital Agent: fully autonomous chatbot to attain 100% call deflection or achieve a higher eCommerce conversion rate

My Five Tenets of Choosing a Chatbot

1. Ease of use based upon how humans interact (keep it simple)

Nothing earth shattering here —if it’s not intuitive and straightforward for humans, it won’t be used, hence a waste of money.  Your chatbot must understand natural language and accept / respond to questions in a human-like (semantic) manner.  Your chatbot should have the capability of answering simple questions along with handling more complicated tasks.

2. Humans love that you truly understand them (personalization)

TM

At Vectara, we encourage our users to “build products that speak for themselves” , which requires more than basic chatbot software. It requires powerful large language models (LLMs).  LLMs are within the AI realm using deep learning and extremely large data sets to understand, summarize and generate outputs.  There are different categories such as:

1. Generative: generate content based upon user prompt(s). Ex. Conversational AI/Chatbots, code/marketing copy/image creation.
2. Question Answering: search & summarization of articles, podcasts, videos, and earnings calls.
3. Rewrite: convert or transform text from one form to another.  Ex. language translation, grammar error correction, text/voice alterations.
4. Clustering: group data (e.g. documents) together based on the content they contain
5. Classification:  group known clusters in order to identify to which class(es) a new piece of text belongs. Ex. intent classification, sentiment detection and prohibited behavior identification.

These LLMs provide the intelligence empowering chatbots to personalize interactions with humans, like remembering a customer’s preferences or historical questions. In today’s market, this is becoming the minimum expectation along with a deep understanding of tonality and cross-lingual ability.

For example, if I launch my customer support chatbot on my eCommerce website, is my goal to have the chatbot answer any & all questions and close the sale without human intervention?  Or do I want a rule or policy-based approach to pull in a human agent when triggered?

3. Empower your chatbot to do most of the work (automation AI)

Your chatbot should be capable of giving customers relevant, precise, accurate and current information. In addition, humans will want comprehensive assistance with tasks such as finding products, summarizing reviews, picking the right product or service, and finishing a purchase…all without human intervention.

4. Humans like to laugh, smile and tell a story about their experience (fun factor)

Regardless of a hybrid or fully autonomous chatbot, a chatbot needs a digital personality that humans enjoy and will want to revisit in the future. Curiosity, humor and style will help your brand stand apart from the competition.

5. Chatbot performance reviews are encouraged (feedback loop)

So, you’ve launched the right chatbot for your business model and brand, and now it’s time to collect feedback from the humans (vis a vis, your users or customers). Is the chatbot helping or hurting the human experience? Take heed of this feedback and make alterations to your chatbot’s look & feel, functionality, and tone based on that feedback.

Observations from My Personal Chatbot Experiences

Major US telecom provider

A major telecom provider had a rule-based chatbot that simply did not understand my question or intent. This led to irritation and me having to get on the phone. I’d rather have just been told to call in and ask a human.

What could they have done better? They could have used an NLP-powered chatbot that understood the context of my query.

Animal supply stores

While looking at multiple animal supply stores, two of them did not have a chatbot on their ecommerce website and the one that did was piloted by a human. 

When I asked “How do I keep my kids’ guinea pigs healthy?” the answer was: “Come in to the store and talk to an expert”. I was stunned by this response, and despite being ready to spend hundreds of dollars on organic animal health products, I exited the website.

What could they have done better? A hybrid between a full digital agent and human agent assistant chatbot would work better here. My initial question should be fielded by the NLP chatbot and if it doesn’t lead to an eCommerce transaction, then queue a customer experience agent using an LLM-powered search solution to further the discovery process, identify the products needed, then add the supplies into my cart to make my eCommerce transaction easy.

Leading consumer electronics company

When I bought a new laptop, I wanted a new magnetic screen protector. While the website said an older protector would fit, I wanted to double-check so I asked their chatbot. The chatbot said “yes”, but I wasn’t able to order it on the website. Next, I was routed to a human customer service agent who did a great job, but this call took 30 minutes just to confirm there was an error in their inventory database. I’m still waiting for an update on the screen protector availability.

What could they have done better? Outside of fundamental inventory and product compatibility process improvements, I’d prefer to see a fully autonomous digital agent here. I just wanted to order the part and move on (whether it’s available now or later – take my order and ship it to me later).

Takeaways

After these experiences (and many others), I felt like the technology was simply making it harder to be a customer because of frustration, lost time and opportunity cost of other tasks I could be doing. I know my undeflected calls into my vendors’ customer support teams increase (or disallow the reduction of) the overhead of the company that I’ll eventually see in the form of higher prices.  

The non-conversion of my “buying question” about animal health products, for instance, led to a loss of top line revenue and customer satisfaction. I expect and hope that any added technology solutions positively impact the buyers’ journey via a simplified, automated and accelerated experience.  

Where do you go from here?

I believe we are going to see amazing things from evolving chatbots, but with hundreds of companies touting they are the best fit for you, it can be noisy and confusing.  

To recap, here are tips for assessing chatbots:

• Decide what level of intelligence you need. 
• If NLP level or higher, ask what type of and size of data set the chatbot was trained on and if it can understand natural language, and then respond in a summarized and comprehensive manner.
• Try out the chatbot yourself (be your own customer). If it takes more than a few minutes to figure out how to use it, move on.
• Monitor the chatbot’s performance and analyze customer feedback. Make adjustments accordingly.
• Prominently display and promote use of your chatbot for question answering, research & analysis and reviews.

Vectara Security OverviewVectara takes security very seriously and prioritizes it into everything we build. This document summarizes Vectara platform security and the features available to users when implementing their Vectara solution, pertaining to storage, privacy, encryption, RBAC, and security compliance measures. Learn More