Proving the value of CX: why the business case needs a control group

The diagnosis is right

Customer experience (CX) programmes rarely fail for lack of ideas. They fail because nobody can show what they bring the company. A recent article in marketing-BÖRSE (in German) makes exactly this point: CX lacks proof that it moves cancellations and revenue. As a solution, the article proposes a strategy map, a map of cause and effect based on the balanced scorecard method, together with a balanced set of leading and lagging metrics and a business case that puts a figure on the expected return.

We agree with the diagnosis. We also agree with the solution, with one reservation: it stops too early. A business case justifies why you invest in a measure. It does not show whether the measure works. Only ongoing measurement with a control group shows that.

A word on our own interest: UNIT ONE builds exactly this kind of measurement with companies. So we have a stake in this argument, and you should know that as you read.

Where the business case alone falls short

A business case is a forecast. It records which measure is supposed to move which metric and what that brings in at the end. For the decision, that is right and necessary. Whether the assumptions hold in your company is something it cannot show, because it is written before there is any data. The strategy map behind it is a model too: academic criticism of the balanced scorecard finds that its cause-and-effect links are justified by logic more than tested empirically.

Then there is the kind of figures many business cases rest on. The article quotes Forrester: according to this, customer-oriented companies report 41% faster revenue growth, 49% faster profit growth and 51% better customer retention than others. These figures compare companies with each other. They show that customer orientation and growth occur together, but they do not show that one causes the other. It is just as possible that growing companies can simply afford more customer orientation. The article itself warns against this fallacy. A business case built on such figures inherits the gap.

And finally, the metrics themselves. Among the early indicators, the proposed set lists the effort customers have to make to get their issue resolved (Customer Effort Score), the share of issues resolved at first contact (First Contact Resolution) and satisfaction after a contact (CSAT). These are sensible measures. But they only tell you what you are observing. They do not tell you who does what at which value. If the first contact resolution rate drops in a customer segment and nobody is responsible and no procedure is defined, the figure simply appears in the next report.

Classifying the Net Promoter Score as a lagging indicator is defensible. A longitudinal study in the Journal of Marketing (2007), however, could not confirm its claimed superiority as a predictor of growth.

Business case for the decision, measurement for the proof

The proof comes from a routine that runs alongside day-to-day operations. It has five steps, and the order matters:

1. Early signal. The health score flags a customer where something is shifting, for example falling usage or missing repeat orders. Which signals work for this is covered in the article on early warning signals like in the cockpit.
2. Ownership and playbook. The signal goes to a named person and does not end up in a monthly report. This person has a playbook: which steps, in which order, by which deadline. Without authority and budget, the ownership exists only on paper. What that involves is described in the article on the implementation mandate for the health score.
3. Control group. A small, randomly drawn share of the flagged customers does not get the playbook. In experimentation practice, this is called a holdout or control group.
4. Measurement after 90 days. You count how many customers in both groups have cancelled and how their revenue has developed. The gap between the groups is the effect of the playbook. The number of calls made proves nothing. Why you need to separate activity from outcome for this is explained in a separate article.
5. Report every quarter. The difference is converted into euros and presented at the quarterly meeting. There, one of three decisions is made: keep the playbook, change it or end it.

The business case stays in place. It justifies the first investment. The measurement tests its assumptions and gradually replaces them with your own values.

Why the control group is mandatory

Without a control group, all you measure is that something happened after the measure. In the same quarter, the season, a price change or a sales campaign also act on the same customers. If cancellations fall after a playbook starts, that may be down to the playbook or to a quiet quarter. The figure alone does not answer that.

How expensive this gap can get is shown by a field experiment published in the Journal of Marketing Research in 2016. It studied around 65,000 customers of a South American mobile operator. One group proactively received a retention offer, namely a recommendation for a better-suited tariff. Another group received no offer and was looked after as before. In the group with the offer, 10% cancelled; in the control group, 6%. The campaign triggered cancellations instead of preventing them. Without a control group, the team would have counted the damage as a success, because the customers it contacted had been "looked after", or would not have noticed it at all.

The obvious objection from management is: "So we deliberately neglect some customers?" No. The control group is small, randomly drawn and limited in time. It does not get nothing. It gets the standard that all customers get today. Only the new measure, whose effect nobody knows yet, is held back. The experiment above shows that such a measure can also do harm.

In an earlier article, the control group was one of five things to settle before a measure starts, as proof before the measure. Here we go one step further: every playbook keeps its control group for as long as it runs. Only then does the difference stay measurable from quarter to quarter.

Quarter by quarter instead of once a year

Without ongoing measurement, the value of CX is negotiated once a year, in the budget round, on the basis of last year's business case. With measurement, a new figure is on the table every quarter: this many fewer cancellations in the group with the playbook, this much revenue retained, in euros.

The calculation is simple. The difference in cancellation rates between the two groups, multiplied by the number of customers handled, gives the cancellations prevented. Multiplied by the average annual revenue per customer, that gives the revenue retained. From this you deduct the cost of the playbook. Your finance team can recalculate every step, because every figure comes from your own data.

That changes the sentence in the budget round. Until now it was: "We expect the playbook to reduce churn." Now it is: "Here is the difference to the control group, quarter by quarter." The measurement also shows when a playbook does not work. If the difference fails to appear or reverses, the playbook is changed or ended before it ties up money for another year.

There is one caveat. With a few hundred customers, both groups may produce only a handful of cancellations in 90 days. Then individual cases shift the picture, and the first quarter shows a direction more than a reliable result. With every further quarter, the finding becomes more certain. You see this uncertainty openly in your own data. The business case contains it too, only hidden in its assumptions.

Start small: one signal, one playbook, one control group

To get started, you need neither a BI team nor a new tool. A CRM in which cancellations are recorded properly is enough, plus a spreadsheet showing which customer is in which group. Before you start, you decide four things:

• The one signal. An early signal you can already pull from the CRM today, such as falling orders or less frequent usage. Pick exactly one.
• The person responsible. They trigger the playbook when this signal appears and are accountable for the result. A department or a committee is not enough for this.
• The share held back. Which share of the affected customers falls randomly into the control group and continues to receive today's standard. Randomly means by a random number in the spreadsheet. The sales team's judgement does not decide it.
• The first measurement date. 90 days after the start, fixed in the calendar. On that date, the person responsible and management look at the difference and decide: keep, change or end.

It would be tempting to attach a measure to every signal from the health score straight away. But ten measures without a control group deliver ten claims. One measure with a control group delivers a difference in euros that you can discuss in the budget round. And if the first playbook achieves nothing, that is a result too.

We can set up a first signal and playbook with you in the Reactivation Audit.

Which CX measure in your company has a control group today?

Frequently asked questions

Does this make the business case unnecessary? No. To decide to invest in a measure, you need a well-founded expectation, and the business case provides it. The ongoing measurement comes on top: it tests its assumptions and gradually replaces them with measured values. A Gartner survey from 2019 points in the same direction: only 22% of respondents had calculated how CX affects company finances, and this group received significantly larger budget increases.

How big does the control group need to be? There is no fixed number that applies to every company. The usual practice is to withhold the measure from a small, randomly drawn share of the affected customers. How quickly the difference becomes reliable depends on how many customers you have and how often they cancel. The fewer customers and the rarer the cancellations, the more quarters you need.

Why is the NPS not enough as proof of value? The NPS measures what customers think of you. It does not measure what a specific measure has achieved. If the score rises after a campaign, you do not know whether the campaign, a new price or the season is behind it. The value of a measure shows in cancellations and revenue compared with the control group.

Sources

• marketing-BÖRSE: how CX proves its contribution to value (October 2026, in German)
• Emerald: criticism of the cause-and-effect assumptions of the balanced scorecard
• MediaPost: field experiment on proactive retention (Journal of Marketing Research)
• MeasuringU: longitudinal study on Net Promoter Score and revenue growth
• destinationCRM: Gartner survey on CX measurement (2019)
• Optimizely: Global Holdouts