AI triage can shorten resolution time while quietly deleting the customer language a product team needs to make good decisions. When support tickets arrive pre-labelled, summarised and routed, the team may solve each case faster but stop seeing why customers are leaving.
Consider Kojo, a composite founder running a small SaaS company in Accra. At 8:17 on Monday morning, he was holding a mug of coffee over a dashboard that looked better than it had in weeks.
Resolution time was down. The support queue was shorter. The AI layer was reading new messages, assigning categories and sending each one to the right person with a neat summary.
Then Kojo opened the cancellation report.
Three customers had left over the weekend. Their reasons appeared as “pricing,” “missing feature” and “other.” The original messages were harder to find, and nobody on the product team had read them.
One cancellation could have been noise. Three vague labels, after a month of cleaner support metrics, suggested a different problem. If the team waited another week, they could lose more customers without learning what those customers had tried to tell them.
The metric improved before the product did
The AI triage system had done exactly what the team asked. It reduced the work required to sort incoming tickets.
That mattered. A small team cannot spend every morning copying messages into categories and deciding who should answer them. Faster routing gave engineers fewer interruptions, support had a more manageable queue, and customers received replies sooner.
But the system also compressed each customer’s account into the fields the team had chosen in advance. A frustrated paragraph became “pricing.” A description of a failed workflow became “missing feature.” Anything that did not fit became “other.”
The dashboard gained order while the evidence underneath it lost detail.
This is a familiar product mistake. We measure the cost of handling information, then automate that cost away without measuring the value of the information itself. Support looks like an operational queue, so we treat every uncategorised message as work waiting to be processed.
For an early-stage company, those messages are also product research that customers volunteered at the moment something mattered enough to complain or leave.
What disappeared inside the summaries
Kojo pulled up one of the original cancellation messages.
The customer had not objected to the monthly price alone. She had spent Friday evening trying to complete a recurring task, reached the same confusing step twice and assumed the higher plan would not solve it. “Pricing” described where she stopped. It did not explain why.
That distinction could change the next product decision. A discount might retain someone who cannot afford the product. It would do little for someone who no longer trusts the workflow.
The same problem appears when teams measure signup as adoption. Access can rise while actual use stalls, as I explored in what happens when signup measures access, not adoption?. A tidy metric can describe activity while hiding the decision a customer is struggling to make.
By 10:00, Kojo had read the other two original messages. One “missing feature” complaint described a feature that already existed but was difficult to find. The “other” cancellation came from someone who had expected a different kind of product from the sales conversation.
Three labels had suggested three backlog items. Three customer messages revealed problems in onboarding, product clarity and positioning.
The summaries were accurate at sentence level. They were misleading at decision level.
The Monday call
The team had planned to spend that week improving the AI classifier. They had already discussed more categories, better prompts and a confidence threshold for uncertain tickets.
Kojo stopped the work.
Removing AI triage meant the support queue would become slower again. It also meant admitting that a visible improvement had concealed a more important loss. With limited runway, abandoning work that appears successful is difficult, especially when the dashboard gives everyone a green number to defend.
The bad ending remained possible: another week of clean categories, followed by another set of cancellations the team could explain only after the customers were gone.
So the team restored the original messages to the front of the workflow. A person would read every cancellation and every ticket tied to failed activation before any summary or category appeared. AI could still handle work beneath that point: removing duplicates, finding account context, retrieving earlier conversations and drafting internal notes for review.
That boundary mattered. The team kept automation where compression was useful and removed it where ambiguity carried information.
Put AI below the judgment
The practical question is not whether AI belongs in support. It is where AI can reduce effort without deciding what the team gets to notice.
Start with the messages closest to revenue, trust and product failure. Read the originals. Compare them with the summaries and categories your system produces. If the summary would lead you to a different product decision, the automation sits too high in the workflow.
Then move AI underneath the judgment. Let it collect context, search history, flag repeated phrases and prepare a response. Keep a human close to cancellation reasons, failed onboarding and unusual complaints until the patterns are stable enough to name without flattening them.
This resembles the risk in an AI-generated financial answer that looks complete while hiding how it reached the result. Visible reasoning matters when a wrong conclusion can damage trust, as in Youssef’s AI showing the wrong closing balance.
On Tuesday morning, Kojo’s support dashboard looked worse. Resolution time had risen, and several tickets still had no category.
Beside them were the customers’ full words. The team could finally see what to fix.
Comments
No comments yet.