The containment rate as a CX KPI is a governance trap

Updated on 8 October 2026: we have added the transition period for Article 50(2).

The bot reports success, and customers leave anyway

Picture this monthly report: the chatbot dashboard shows 80% of conversations handled. The vendor is pleased, the service team feels the load has eased, and the curve is pointing up.

In the same month, more customers call about issues they had supposedly already sorted out in the chat. There are more complaints in the inbox. In sales you hear things like "I tried three times and nobody helped me". And among the cancellations are customers whom nobody had considered at risk.

Both can be true at the same time. The dashboard calculates correctly. It just measures something other than what you assume.

The containment rate measures whether the bot kept the conversation. It does not measure whether the customer's issue was resolved.

If this figure becomes the yardstick of success for your customer service, you are steering by a value that says little about what the service is for. That turns an operational metric into a question for management.

What the containment rate really counts

"Containment" means keeping something in. The metric counts the share of conversations that end without being handed over to a person. Every conversation that does not reach an employee therefore counts as "handled", for example when:

• the customer gets an unsuitable answer three times and closes the window,
• the customer looks for a way to reach a person, cannot find it and gives up,
• the customer ends the chat and calls your hotline an hour later,
• the customer writes to their contact in sales the next day, because there at least they know a name.

Under the common definition, the metric goes up in every one of these cases. In none of them is the problem resolved. Some systems count abandonments or timeouts separately, so check the counting logic of your own system.

The cause is the definition itself, whichever tool you use.

A Gartner survey of 5,728 customers from December 2023, published in August 2024, shows how large this gap can be. Only 14% of service issues were fully resolved in self-service. Even for issues that customers described as "very simple", the figure was only 36%. 43% found no content relating to their problem. The figures apply to self-service as a whole, not just to chatbots, and they do not come from the DACH region.

The route to a person is not a side issue either. In June 2023, the US Consumer Financial Protection Bureau (CFPB) documented complaints about so-called "doom loops": loops in which the bot keeps giving the same unhelpful answers and offers no way out to a person. For the containment rate, every one of these loops is a success.

Why this becomes a governance trap

As long as the containment rate stays one operational figure among several, it does little harm. It becomes critical as soon as it turns into a target: on the management dashboard, in the service team's annual objectives, in the contract with the bot vendor.

From then on, everyone works towards keeping conversations with the bot. Everyone optimises for what they are measured on: the vendor its rules, the internal team its priorities.

If you go through the bot logs of an operation like this, you find three patterns that reinforce each other:

• Hidden handover. The route to a person exists but is hard to find. There is no visible button, the option only appears after several follow-up questions, or a contact form appears instead of a handover.
• Loops. The bot answers, but not the question that was asked. The customer rephrases and gets a similar answer. These are the doom loops from the CFPB complaints.
• Premature closing. The bot closes the conversation after a generic answer or a link to a help page. Technically handled. Nobody knows whether the customer got any further.

All three improve the metric. None of them solves a problem.

Behind this is Goodhart's law: as soon as a metric becomes a target, it loses its meaning. It then shows how well people have learned to hit it, and no longer how well something works. With the containment rate this is doubly awkward. Even in normal operation it cannot see who gave up. Once it becomes a target, exactly this invisible share grows.

The governance question therefore lies in how the bot is steered. Check what money and objectives depend on:

• Is your bot vendor or service provider paid by containment, "automated conversations" or avoided handovers?
• Does the business case for the bot only add up with a high containment rate?
• Is the containment rate part of your service team's target agreement?

Every yes means: when in doubt, you are paying or rewarding for customers not reaching a person. Step 3 in the roadmap below shows how to change that.

The hidden costs: where unresolved enquiries go

An enquiry the bot does not resolve does not disappear. It changes channel. On the dashboard it shows as "handled". In the company it resurfaces hours or days later, usually at higher cost and with an annoyed customer.

Phone and email. The customer abandons the chat and calls. Your team gets a case with a history, but usually sees none of it. The customer explains everything again, this time more irritated.

Sales and field service. In B2B business, the second attempt often goes to the person the customer knows. Service work gets done there that does not appear in any service statistics. That time is missing for advice and new business, and because no ticket is ever created, the case stays invisible to management.

Public channels. Anyone who cannot find a way out complains where others can read along: on review portals, on social networks, to supervisory bodies. The CFPB cases show that such complaints reach the regulator.

Silent churn. The most expensive channel, because you see nothing there. The customer does not complain. They order less often and at some point stop renewing. Many customers' attitudes make this plausible: in a further analysis of the same Gartner survey from July 2024, 60% said they fear that AI will make it harder to reach a person. 53% would consider switching provider if a company planned to use AI in customer service. That is an intention. It does not yet mean anyone has cancelled. But a bot that blocks the route to a person confirms exactly this worry.

The repeat contact as an early signal

A single question makes the path of unresolved enquiries visible: does the same customer get in touch again about the same topic within a fixed time window?

For early churn detection this is valuable, because two things come together here: a concrete, unresolved problem and a customer who is already trying for the second time. If this signal goes into your health score, you see the risk before the cancellation arrives. A score like this assesses individual customers. That is why we agree it with the data protection adviser before we set it up. And we never leave decisions with a noticeable effect on the customer to the score alone.

A calculation for your own data

You can calculate the gap between the containment rate, actual resolution and churn with your own figures.

First, a note on data protection: here you link chat, ticket, phone and cancellation data via a customer ID, which means data about individual people. The ID does not make it anonymous. In our projects, before the first analysis, we clarify with the data protection adviser what the data is used for, who has access and when it is deleted. Wherever possible, work with aggregated figures.

This gives you three values:

1. Displaced contacts per month = A × B × C
2. Displaced handling costs per month = displaced contacts × D
3. Additional contribution margin at risk per year = H × (E − F) × G

The first value shows how much work the bot has merely shifted elsewhere, and the second puts a price on it. The third is the decisive one: it puts a figure on what it costs when customers with unresolved issues leave more often than others. H counts customers instead of conversations, because a customer can only cancel once. Contacts via sales and field service are missing as long as they are not recorded. The result is therefore more of a lower bound. Bear in mind: the difference E − F shows a correlation. It does not prove that the repeat contact causes the cancellation. If the two customer groups also differ in other characteristics, that distorts the value.

Can you not determine C and H today because the bot, ticketing system and phone are not connected via the same customer ID? Then that is already a result. You are steering your bot by a metric whose consequences you cannot see.

What you should measure instead

For steering, a different question counts: is the issue resolved? In many companies, four metrics from systems that are already running are enough for this.

1. Resolution rate from the customer's perspective

At the end of the bot conversation there is a single question: "Has your issue been resolved?" Yes or no, without a questionnaire. The metric shows the share of conversations in which the customer confirms that they were helped. Check with the vendor whether your chatbot stores the answer for each conversation.

2. Repeat contact rate for the same issue

Some customers click "yes" just to close the window. So the confirmation needs a cross-check: does the customer get in touch again about the same topic within the defined time window, through whatever channel? The rate shows how many bot conversations merely postponed the problem. The data comes from linking customer number and issue category across chatbot, ticketing system and phone.

3. Abandonment rate in the bot

The share of conversations that end in the middle of the dialogue: the customer closes the window without answering the resolution question. This figure also captures those who never come back afterwards, which are exactly the candidates for silent churn that the repeat contact rate misses. It is available straight away in the chatbot's logs, without linking to other systems.

4. Quality of the handover to people

If the customer has to tell the whole story again after the handover, the bot was a detour. In the Gartner survey, 45% of those who had started in self-service said the company had not understood what they wanted. You can measure this with a monthly sample of handed-over conversations and one question per case: was the full conversation history passed on?

You don't have to abolish the containment rate for this. As an operational metric for workload and staff planning, it remains useful. As the target, the resolution rate and the repeat contact rate take over together: the first shows what the customer says, the second what they do afterwards.

Handover to people as a mark of quality

If you make containment the target, every handover counts as a defeat for the bot. A handover at the right moment and with context is part of the solution: the bot has recognised the issue, classified it and passed it to the place that can resolve it.

A publicly documented example is Klarna. In February 2024, the company reported that its AI assistant had handled 2.3 million conversations in its first month. In May 2025, it started hiring customer service staff again so that customers can always speak to a person.

Three design patterns make the handover a mark of quality:

A visible way out. The customer should not have to guess a keyword. The option to reach a person is there in every dialogue and can be reached in a few steps. On top of that come fixed triggers for an automatic handover, for example when the bot has failed to answer the same question twice or the customer is clearly annoyed.

Handover with the conversation history. The employee sees what the customer wrote, what the bot replied and which issue was recognised. For this, the chat log including the issue category has to land in the ticketing system or CRM. Without this connection, the handover is just a change of channel with a restart.

Topics the bot does not handle alone. Cancellation, complaints and billing disputes belong on this list. Here the customer relationship is at stake, and a wish to cancel is one of the most valuable early signals you get. The bot recognises the issue, records the necessary data and hands over. A case from Canada shows what wrong information from a bot can cost: in February 2024, the Civil Resolution Tribunal of British Columbia ruled in favour of a customer who had been given wrong fare information by an airline's website chatbot. The argument that the bot was a separate entity did not convince the tribunal. This is a single Canadian case. We still take one principle from it for our projects: from the customer's point of view, what the bot says, the company says.

For steering, this means: a low handover rate is not a goal in itself. What matters is that the right issues are handed over and that the customer does not have to start from scratch afterwards.

Then there is the AI Act. According to the European Commission, the transparency rules in Article 50 have applied since 2 August 2026. For AI systems placed on the market before that date, providers only have to meet the obligation to mark AI-generated content (Article 50(2)) from 2 December 2026, according to the Commission. The Commission explains there what they provide for in detail. Our position: customers should know whether they are writing to a machine. If you use a third-party bot, ask the vendor how the notice is implemented. According to the European Parliament, the AI Omnibus mainly postponed the high-risk deadlines. An AI notice does not replace handover rules. If you are revising your bot dialogues anyway, do both in one pass. The AI notice in customer service has an article of its own.

Governance in three steps: how to change the way you steer

A better metric alone changes little. As long as nobody defines what it means, who owns it and what service providers are measured on, your operation keeps working towards the old target. The switch takes three decisions. You don't need a new tool for it. The handover checklist from pilot to routine operation covers which operating questions remain open after the launch.

Step 1: Define what "resolved" means

For the bot, "resolved" often means "conversation ended", for the service team "ticket closed", for the customer "my problem has gone away". Set one definition that applies to all channels. A useful starting point:

• The customer confirms when asked that their issue has been dealt with, and
• they do not get in touch again about the same topic within a fixed time window, whatever the channel.

Choose the time window to suit your business and then keep it constant. Billing questions often only resurface with the next invoice and need a longer window than a delivery problem. Write the definition down on one page as an operating rule that management, service and service providers all refer to. Without this page, you will later argue about figures instead of results.

Step 2: Clarify ownership

Two questions each need a name on the organisation chart. A working group will not do.

Who looks at the figures every month? One person from operations owns the resolution rate and the repeat contact rate. They present both to management once a month, together with the three most common topics on which customers get in touch again after a bot conversation. You don't need a BI team for this. A monthly export is enough to start with.

Who may change the bot's rules? New answers, changed handover routes, topics the bot will handle alone in future: changes like these shift where customers end up. Decide who approves them and document every change with a date. Only then can you tell later whether a rise in repeat contacts is linked to a particular adjustment.

Step 3: Change contracts and objectives

This is where the biggest lever sits, because remuneration decides what your service provider works towards. Check contracts and target agreements for three points:

1. Target metric: replace containment as a remuneration or bonus criterion with the resolution rate under your definition from step 1.
2. Counter-metric: agree on the repeat contact rate as a limit. If it rises, a good resolution rate does not count in full.
3. Handover quality: set down that a clean handover with the conversation history is not counted as a failure.

Where a contract cannot be changed at short notice, first add the new metrics to reporting as mandatory and tie remuneration to them at the next renewal.

What is different on Monday morning?

The dashboard still shows the containment rate, but as an operational value. Next to it are the resolution rate and the repeat contact rate. The responsible person knows they will have to explain these figures in the monthly meeting. Anyone who wants to change a bot rule gets approval first. And the next conversation with your service provider opens with the question of how many issues were resolved. How many conversations the bot kept is no longer where it starts.

Checklist: is your chatbot steered correctly?

Seven questions, each to be answered yes or no. Yes is always the good answer.

1. Is it set down in writing what "resolved" means for you? This means a definition from the customer's perspective, with confirmation and a fixed time window.
2. Do you know the repeat contact rate after bot conversations? That is, the share of customers who get in touch again about the same issue after a chat, whatever the channel.
3. Is there a named person who reviews these figures every month? Without ownership, a bad value gets noticed but changes nothing.
4. Is your vendor's remuneration independent of the containment rate? This applies to bonus, price and targets.
5. Can a customer reach a person in two steps at most? Test it yourself.
6. Do cancellations, complaints and billing disputes go straight to people? These conversations decide whether a customer stays.
7. Are bot abandonments matched against later cancellations? A customer who abandons the chat and cancels weeks later shows up in the containment rate as a success. Only matching via the customer ID makes this visible.

Every no marks the point where the switch begins. That does not mean you have to switch the bot off.

Back to the monthly report from the beginning: 80% of conversations handled. How many of them were actually resolved? If you cannot answer that today and want to know where to start, we are happy to talk about it. A first conversation clarifies which data you already have and which step pays off first. More on this under Reactivation Audit.

Frequently asked questions

Is a high containment rate bad in principle? No. It is useful for staff planning. As a target for customer success, it is of no use.

Which metric best replaces the containment rate? No single one. The resolution rate from the customer's perspective and the repeat contact rate belong together, with a time window you set yourself.

How do I measure repeat contacts without a BI team? Usually it is enough to link two fields across chatbot, ticketing system and phone: customer ID and issue category. All three channels need the same categories. To start, a monthly export into a spreadsheet is enough. You can automate later. The linking involves personal data, so clarify it in advance with your data protection adviser.

Should the chatbot respond at all when a customer wants to cancel? It should recognise the issue and hand it over to a person together with the conversation history so far.

Where do I get a binding answer on the AI notice in the chat? The European Commission explains the transparency rules in Article 50 in its questions and answers. Your legal counsel can clarify what applies to your bot. For us, the topic is above all a reason to set down the operating rules for the bot in writing, including the resolution definition and handover rules.

This article describes our practical perspective. It does not contain a legal assessment; for that, we refer you to the linked primary sources and your legal counsel.

Sources

• Gartner customer survey on self-service (December 2023, published 19 August 2024), reported by CX Today
• Gartner customer survey on AI in customer service (published 9 July 2024), reported by The Register
• CFPB, "Chatbots in consumer finance" (6 June 2023): consumerfinance.gov
• Klarna, press release on the AI assistant (27 February 2024): klarna.com
• Klarna hires customer service staff again (9 May 2025): Customer Experience Dive
• Civil Resolution Tribunal of British Columbia, decision 2024 BCCRT 149 (14 February 2024): CanLII
• European Commission, FAQ on transparency obligations under Article 50 of the AI Act: digital-strategy.ec.europa.eu
• European Parliament, Legislative Train "Digital Omnibus on AI" (as of 1 August 2026): europarl.europa.eu