Before I automate a workflow, I need to see the decisions people make, the exceptions they handle, and the actions that require approval. If those three things remain implicit, automation will execute an incomplete version of the work faster and more consistently than a person ever could.
On August 1, 2012, Knight Capital began routing customer orders through newly deployed software. The code had reached seven of the firm’s eight servers. On the eighth, an old function remained available.
When trading opened in New York, that server started sending unintended orders into the market. Knight accumulated positions it did not want, and the problem continued for roughly 45 minutes. The firm lost more than $460 million.
Knight’s chief executive, Thomas Joyce, had to secure emergency financing to keep the company operating. The sequence is documented in the US Securities and Exchange Commission’s 2013 order against Knight Capital.
The failure did not begin when the software ran. It began earlier, when a deployment process allowed one server to become an exception without forcing anyone to detect, stop, or approve it.
Show me where judgment enters the workflow
When a founder asks me to automate purchasing, customer support, invoicing, or lead qualification, I start with the moments where someone has to decide.
“Finance reviews the request” tells me almost nothing.
What does finance review? The supplier, the amount, the budget, the currency, the delivery country, or the requester’s authority? Does the same rule apply to a recurring software renewal and a first payment to a new contractor? What makes the reviewer approve one and hold the other?
The useful explanation sounds more like this:
A team lead can approve routine purchases within an agreed budget. Finance must review a new supplier, a changed bank account, an unusual currency, or any request with incomplete supporting documents. A director decides when the purchase falls outside the approved plan.
Now I can see the decision points. I can also see which parts software may prepare and which parts still belong to a person.
This is where a screen recording often helps more than a polished process diagram. I want to watch someone handle one ordinary request and one awkward request. The pauses matter. So do the browser tabs they open and the colleague they message before clicking approve.
Those small actions usually contain the real workflow.
Exceptions deserve their own design
Most teams describe the happy path first: a request arrives, the system checks it, someone approves it, and the order goes out.
Then the exceptions appear.
The supplier changed its payment details. The requester entered the wrong legal entity. The amount is valid, but the timing is suspicious. The customer replied after the support ticket had already been marked resolved. The AI produced a confident answer using an old product document.
I do not treat these as edge cases to clean up later. They define the safe boundary of the first version.
For every exception, I need to know what happens next. Should the system stop, ask for missing information, route the case to a named role, or create a draft without sending anything? Who can release the stopped transaction? What evidence should that person see?
This was the central issue in Kwame’s unexplained payment retry. A retry may look like a technical detail until one duplicate payment reaches a customer or supplier. The exception needs an owner before the automation needs another feature.
Knight Capital’s eighth server was also an exception. The damage came from having no effective control that converted that difference into a mandatory stop.
Approval boundaries must name the action
“Keep a human in the loop” sounds responsible, but it leaves the most important question unanswered: which loop?
A person may review an AI-generated summary without controlling the payment, email, refund, purchase order, or account change that follows. That is observation, not approval.
I define the boundary around the irreversible or costly action.
The system can collect information, compare records, flag inconsistencies, calculate totals, and prepare a recommendation. It can draft the supplier email. It can place the proposed purchase in a queue. But a named role must approve before money moves, a contract changes, or an external message creates a commitment.
The boundary can move later. Early versions should keep it close to the consequence.
That was also the reasoning behind shipping an AI agent with human approval in Berlin. The approval step was part of the product design because the team had not yet earned enough evidence to remove it.
Bring one normal case and two uncomfortable ones
Before our first automation session, I ask for three recent examples.
Bring one request that moved through the workflow without trouble. Then bring two that forced someone to pause, ask a colleague, correct information, or make an exception. Remove sensitive customer details, but keep the sequence intact.
For each example, write down who made the decision, what they checked, what could have gone wrong, and which action required permission. If two people give different answers, do not smooth over the disagreement. That difference is part of the system we need to understand.
Only then should we discuss triggers, integrations, agents, or models.
Knight Capital showed the cost of a process that could treat seven successful deployments as sufficient evidence while one unresolved difference remained. Your workflow may involve smaller stakes, but the design question is the same: when one case falls outside the expected path, does the system continue, stop, or ask the right person?
Bring me the awkward cases first. They tell us what the automation is allowed to do.
Comments
No comments yet.