The vision of AI revolutionizing customer service is compelling.

Businesses across industries seem to agree. Many of them are investing billions in AI-powered chatbots, with banks alone projected to spend US$18.4 billion by 2028.

Image credit: Timmy Loen

Still, how useful AI is depends on how it’s used as part of a customer experience strategy. In many cases, businesses aren’t fully realizing the tech’s potential.

To better understand where AI chatbots struggle and how companies can improve their effectiveness, we analyzed chatbot interactions across seven industries, reviewing over 200 real-world customer interactions. Our goal was to identify breakdowns in chatbot experiences and see how they could be improved.

The findings confirmed that they still leave a lot to be desired. Here’s what that means for the customer service sector.

Can AI chatbots hear us?

Our goal was to identify where the gaps were between a company’s current and intended chatbot experience.

The findings proved our hypothesis: Chatbots excel in structured interactions but struggle with context, nuance, and complex queries.

See also: Battle of the AI transcribers: Meeting.ai takes on Otter.ai

Think of times when you’ve received responses that felt scripted or when the bot misses your point entirely.

Our findings were also supported by a recent Gartner survey, which found that only 14% of customer issues are resolved through self-service. This underscores the reality that many businesses are still struggling to use AI to improve customer service.

Some of our evaluation is based on these questions:

• Did the chatbot answer the customer’s question directly?
• Did it do what it was asked?
• Did it actually solve the problem?

Based on our data, the overall effectiveness of chatbots scored 60%, and the resolution rate was slightly higher at 67%, so there’s room for improvement.

Balancing efficiency with customer experience

Smooth like butter