A model’s accuracy never gives you the right to train on data shared for a different purpose. If a partner supplied records for operations, pause the launch until the permission for model training is explicit and documented.
At 8:12 on launch morning, the Nigerian fintech founder saw the message that changed the day. The partner dataset behind the model’s unusually good results had been shared to help resolve operational cases. Nobody had agreed that it could be used to train a product.
The demo was ready. The team had already seen what happened when the model had access to those records: better classifications, fewer awkward edge cases, and an answer that looked credible in front of prospective customers. Remove the data, and the launch became uncertain again.
That is the trap. A dataset can feel earned because it sits in your warehouse, your engineer has cleaned it, and your model performs better with it. None of those facts answers the only question that matters: what did the partner actually permit?
Accuracy can hide a permission problem
The founder’s first instinct was to separate the commercial risk from the technical work. The model had worked. The partner relationship could be repaired later.
That sequence reverses the real risk.
A partner may be comfortable with its data being used to complete a specific operational task. Training can create a different kind of use: the data may shape a reusable model, influence outputs for other customers, or stay embedded in evaluation sets after the original project ends. Even where the records have been cleaned or reduced, the permission question remains.
The cleanest rule is also the least convenient on a launch morning: purpose comes before access. Write down the original purpose, the proposed training use, who can approve it, and what happens to the data if approval does not arrive. If those answers are missing, accuracy is a temporary property of a model you cannot safely depend on.
This is close to the ownership and review problem in AI Code Approval: What Daniel Learned About Ownership and Human Review. A working output can still depend on authority that was never granted.
Henrietta Lacks’ cells carried a larger question
In 1951, Henrietta Lacks was treated for cervical cancer at Johns Hopkins Hospital in Baltimore. During her treatment, cells from her tumour were taken and sent to George Gey’s laboratory. Gey had been trying to grow human cells that would survive and reproduce outside the body. Lacks’ cells did.
They became HeLa cells, one of the most widely used human cell lines in medical research. The scientific value was enormous. The question of consent remained.
Rebecca Skloot documents the history in The Immortal Life of Henrietta Lacks: Lacks did not know her cells were taken for research, and her family learned much later how extensively they had been used. The value created from the cells did not settle the question of whether their use had been properly authorised.
The comparison has limits. Patient tissue and fintech partner data carry different legal, ethical, and practical obligations. The mechanism is the same: access to something useful can be mistaken for permission to reuse it. The more valuable the output becomes, the easier it is for a team to rationalise the original gap.
For a founder, that rationalisation often sounds practical: “We will replace the data after launch.” But a partner has to trust you before agreeing to the next dataset, the next pilot, or the next integration. A launch built on unclear permission spends that trust before it has earned revenue.
The launch decision should change
The founder had three realistic choices that morning.
He could launch with the dataset and hope the partner accepted a clarification after the fact. That would preserve the demo while turning a relationship question into a potential breach of trust.
He could remove the data, accept a weaker model, and launch only what the team could support with clearly permitted inputs. That could make the product look less polished, but it would make its limits visible.
Or he could delay the launch long enough to ask for a specific agreement: whether the partner approved training, whether derived model artefacts could be retained, whether the data could support other customers, and who could revoke that approval.
There is no elegant version of this choice when runway is short. The first option can feel like survival. But a signed commercial relationship with a partner is often worth more than a model that performs well for one demonstration.
Build the permission check into model work
Treat data permission as a product requirement, alongside latency, cost, and output quality. Before a dataset enters training or evaluation, keep a short record with its source, original purpose, approved uses, retention period, and the person who confirmed the approval.
Then test the model without it.
That test matters because it tells you what belongs to the product and what belongs to the partner’s data. You may find a narrower initial use case, a synthetic evaluation set, or a slower route to validation. You may also find that the impressive demo was carrying more borrowed value than you realised.
George Gey’s lab learned that Henrietta Lacks’ cells could keep growing. The lesson for founders is less dramatic and more immediate: useful inputs can outlive the decision that allowed them into the room. Make sure that decision can carry the weight you are placing on it.
Comments
No comments yet.