AI-generated code that passes a demo should not be merged until someone can explain its critical paths, failure modes and rollback plan. Working software proves one scenario succeeded; understood software gives the team a reasonable chance of fixing the next scenario when it fails.
Consider a composite three-person team in Accra. At 4:46 on Friday afternoon, Kwame sat between a cooling takeaway box and a laptop showing twelve green checks. The AI-generated pull request had completed the checkout demo without an error. Their prospective partner expected to see it on Monday.
Ama, the founder, asked one question before approving it: “What happens if the payment confirmation arrives twice?”
Nobody answered.
Twelve green checks and one blank silence
Kwame had prompted the coding tool, adjusted the output and cleared two visible errors. Sena had tested the happy path in the browser. Ama understood what the feature needed to do for the buyer.
Yet the pull request touched the order state, payment callback and notification queue. None of them could trace why one helper retried three times while another recorded completion before the final response returned.
They had a choice. Merge it and protect Monday’s demo, or stop and risk showing less than they had promised.
The bad ending was specific. A duplicate confirmation could create two fulfilment requests for one order. If that happened during the partner’s test, the team would have to explain a flaw in code they could not confidently change under pressure.
For a team with limited runway, delaying a demo hurts. Losing trust in the room can hurt more.
This is where AI coding changes the shape of product risk. It can compress the time between an idea and something visible. It can also produce more code than a small team has the attention to understand. Output rises immediately. Review capacity does not.
That gap now shows up beyond one fictional Friday. A Harris Poll survey of 1,528 developers and technology buyers found that AI coding tools are accelerating code output while review, traceability and governance fail to keep pace. The practical problem is already sitting inside pull requests.
Understanding the code means knowing where it can break
Teams sometimes treat understanding as the ability to describe a feature in broad terms. “It handles the payment callback” sounds sufficient until the callback arrives late, arrives twice or contains an unexpected state.
For this pull request, Ama did not need every person to recite every line. She needed one named owner who could answer four operational questions:
- What assumptions does this code make about incoming data?
- Which actions can safely run more than once?
- What gets written before an external service responds?
- How would we disable or reverse the change?
Those questions move the review away from whether the demo looks right and towards what the team will do when reality differs from the demo.
I have found the same distinction useful when reviewing AI workflows outside software engineering. A model may produce the right answer during a prepared demonstration, yet the important decision concerns the wrong answer: who sees it, who can stop it and what evidence remains afterward. That is why human approval mattered in this AI purchasing assistant demo.
The relevant standard depends on the blast radius. A generated colour change needs less scrutiny than code that moves money, changes permissions, deletes records or sends messages to customers. Review effort should follow consequence.
The team made the pull request smaller
At 5:18, Sena found the turn.
Instead of choosing between merging everything and missing the demo, she separated the visible checkout flow from the automated fulfilment step. The demo could record a confirmed payment, but fulfilment would require a manual approval on Monday. Kwame then removed the retry helper he could not explain and added a test for the duplicate confirmation.
They lost part of the automation. They kept control of the outcome.
This was not a clever technical compromise. It was a product decision about what the team was prepared to own. The smaller pull request had fewer assumptions, a clear stopping point and a person who could explain each state change.
Infrastructure changes deserve the same restraint. When a tool proposes a large production change, the command completing successfully tells you little about whether the consequence is acceptable. The useful pause comes before approval, as in the case of Terraform proposing to replace a production database.
A small team cannot inspect unlimited generated code. It can limit how much code enters a release, isolate risky actions and require ownership before merging.
Monday’s demo had one deliberate pause
On Monday, Ama showed the partner a completed checkout, then clicked the approval control before fulfilment continued. She explained that the manual step would remain until the team had tested duplicate and delayed confirmations.
The demo contained less automation than Friday’s version. It also gave the team an honest answer when the partner asked what would happen if the payment provider sent the same event twice.
After the call, Kwame returned to the pull request. The twelve green checks were still there. Beside them sat a shorter diff, a duplicate-event test and his written explanation of the rollback path.
That is a useful release threshold for AI-generated code: before you merge it, one person should be able to point to the dangerous assumption, describe the failure and show how the team gets back.
Comments
No comments yet.