“Delivered” needs a test both sides can apply to the same output. On Tuesday, one folder of malformed customer records meant I could demonstrate the product, but I could not honestly claim the agreed milestone was complete.
In 1999, NASA’s Mars Climate Orbiter was approaching Mars when the navigation team discovered that its trajectory had brought it much closer to the planet than intended. The spacecraft disappeared, and the mission was lost.
NASA’s investigation found a unit mismatch. Lockheed Martin had produced thruster data in pound-force seconds. The navigation software at NASA’s Jet Propulsion Laboratory in Pasadena expected newton-seconds. Both teams had working systems. Both produced outputs. The handoff between them failed because the same data carried two different meanings.
Arthur Stephenson chaired the Mars Climate Orbiter Mishap Investigation Board. Its Phase I report documented the mismatch and the failures around verification. The spacecraft had reached Mars. Nobody could call that delivery.
The demo passed while the delivery failed
My Tuesday problem had smaller stakes, but the mechanism was familiar.
The demo was finished. The milestone had been agreed. The invoice was ready. Then one folder of customer records arrived in a shape the product could not interpret consistently.
Some records could be processed. Others could be corrected manually. The difficult cases were the ones that looked valid enough to enter the system but carried fields whose meaning was unclear. A blank value might mean missing information, an intentional omission or a failed export. A repeated identifier might represent a duplicate customer or two legitimate entries created under different conditions.
I could make the demonstration run by cleaning the folder first. That would prove the product worked with the cleaned records. It would say nothing about whether the delivered system could handle the customer’s actual input.
Sending the invoice would have converted that ambiguity into a claim: the milestone was complete.
I left it unsent.
A milestone needs a boundary, not a label
“Working demo” and “delivered system” sounded close when we agreed the milestone. The malformed folder exposed the distance between them.
A working demo answered one question: can the product complete the intended workflow with data it understands?
Delivery required answers to several harder questions. Which input formats were included? What happened when a required field was empty? Should the system reject an entire file or isolate the affected records? Who decided whether two similar records were duplicates? What evidence would show that the output was complete?
Those questions belonged in the acceptance criteria before the build started. We had discussed the visible workflow because that was easy to picture. We had not defined the ugly edge of the workflow, where customer data stops behaving like sample data.
This is the same source-of-truth problem I wrote about in the third broken product design handoff. Agreement at the level of intent can survive several meetings. It breaks when two people must decide whether the same artifact is finished.
A milestone name does not settle that decision. A boundary does.
Bad records are product requirements
The folder changed the work because it revealed a requirement we had treated as an implementation detail.
The product needed a policy for malformed records. That policy could reject them, quarantine them for review or accept them under explicit rules. Any of those choices could be reasonable. Quietly repairing the input before a demo was not a product policy. It was a temporary act performed by someone who knew where the defects were.
Founders building AI and automation products face this often. The impressive path works first: a document is classified, a workflow is triggered, a summary appears. Then live data arrives with missing fields, inconsistent labels and records copied across several systems.
At that point, the central product question changes. The model’s capability matters less than the system’s behaviour when it cannot trust the input.
This also changes how I think about a successful demonstration. A demo can impress the user while leaving the person responsible for payment unsure what has been completed. I explored that gap in what happens when a demo wows the user, but not the person paying. The answer usually sits in the acceptance test, not the presentation.
Define failure before asking for acceptance
The next useful artifact was no longer the invoice. It was a short set of decisions about the malformed folder.
We needed to identify which records the system must accept, which it must reject and how rejected records would be reported. We needed a shared sample containing the awkward cases, not another clean demonstration file. Completion would mean running that sample and producing an agreed result that either side could inspect.
NASA’s Mars Climate Orbiter investigation did not treat the unit mismatch as a strange piece of bad luck. It examined why the interface, verification and communication practices allowed two valid-looking outputs to carry incompatible meanings.
That is the useful bridge for a founder with a milestone waiting to be billed. Do not ask whether the feature works. Ask what each side will observe when it fails, and what result will still count as acceptable.
Before the next build begins, put one malformed file beside the milestone description. Decide what should happen to every record in it. Then write the invoice condition in those terms.
Comments
No comments yet.