Guide · Bots & AI
Measuring chatbot performance: metrics that matter
A chatbot can look busy — thousands of conversations — while frustrating customers and saving nobody time. Or it can quietly resolve the most common questions and hand everything else to the right person. The difference shows only in the right metrics. This guide lists the ones that matter and how to use them.
[ 01 ]
In short
In short
- 01
Measure outcomes: tasks completed, enquiries created, orders placed — not just conversations started.
- 02
Containment (resolved without a person) is useful only alongside satisfaction; a bot that traps people "contains" them too.
- 03
Track handovers: how often, why and how fast a person replies afterwards.
- 04
Review a sample of conversations regularly; numbers show where to look, transcripts show why.
- 05
Connect bot outcomes to the CRM and sales to see business value.
[ 02 ]
Guide
Core metrics
Usage
- Conversations started, by channel and entry point.
- Unique users and returning users.
Completion
- Share of conversations where the user completed the intended scenario — booking, order, enquiry, status check.
- Drop-off by step: where users abandon the flow.
Containment and handover
- Share of conversations resolved without human involvement.
- Share handed to a person, by reason: user request, failure, sensitive topic.
- Time from handover to first human reply.
Quality
- Customer satisfaction after the conversation, from a short rating.
- Fallbacks: how often the bot did not understand.
- For AI assistants: share of "I do not know" answers and share of answers flagged as wrong.
Business outcomes
- Enquiries, bookings and orders created by the bot.
- Conversion of bot-created enquiries into customers, from the CRM.
- Time saved for staff: questions answered without agents.
Reading the numbers together
- High containment, low satisfaction: the bot may be blocking access to people.
- High handover rate for one topic: add a scenario or knowledge for it.
- Drop-off at one step: the question is confusing, too personal or asked too early.
- Many fallbacks: users phrase things the bot does not expect; add examples or AI understanding.
Conversation review
Each week, read a sample: completed, abandoned and handed-over conversations. Look for misunderstandings, missing information, tone problems and new questions. Turn findings into changes and track their effect. For AI assistants, review answers for accuracy. See AI hallucinations.
Dashboards
A simple dashboard with weekly trends — conversations, completion, containment, handover time, satisfaction and outcomes — is enough for most businesses. Connect it with CRM data to show revenue from bot-created leads. See dashboards.
Privacy
Conversation logs contain personal data. Store them securely, limit access, and keep them only as long as needed, in line with your privacy policy.
[ 03 ]
FAQ
Frequently asked questions
What is a good containment rate?
It depends on the scenario. For simple status checks, high containment is realistic; for complex sales, the bot's job is to qualify and hand over. Judge containment together with satisfaction and outcomes.
How do we collect satisfaction ratings?
A one-tap rating at the end of the conversation, optionally with a comment. Keep it short; few people answer long surveys in a chat.
How often should the bot be improved?
Continuously in the first months, based on weekly reviews; later, monthly or when services, prices or policies change.