When a narrowly designed AI demo reaches thousands of people, the first decision is to avoid rebuilding it for the crowd. Stop distribution, preserve what happened, and separate failures against the original use case from expectations the demo was never designed to meet.
Consider Kojo, an invented composite of several founder situations I have seen. At 4:17 on a Friday afternoon in Accra, he was holding his phone above a laptop that had started running hot. The demo on screen turned short customer notes into draft follow-up emails for small sales teams. He had built it for six founders who already knew its limits.
Then someone shared the link publicly.
The audience changed before the product did
At 4:23, the first unfamiliar visitors arrived. By 4:31, people were pasting in job applications, legal complaints, medical questions and entire pages of text. The output slowed. One response confidently completed a detail that the input had never supplied.
A screenshot started circulating without the original explanation.
Kojo had planned to use the weekend to review six structured tests. He now had thousands of visits from people carrying different assumptions about what the tool did, what it stored and how accurate its answers should be. The server could fail. Worse, someone could act on an answer produced outside the demo’s intended context.
He opened the code editor and reached for the fastest visible fix: increase capacity, widen the input limits and add more examples to the prompt.
That would have been the wrong first decision.
More capacity would have kept the demo available to an audience it had never been evaluated for. A broader prompt would have made the tool appear more capable while removing the narrow boundaries that made its original test useful. The traffic felt like validation because the graph moved sharply upward. It mainly proved that a public link could travel.
This is the same measurement problem I explored in [the 186 accepted connections](\/blog\/the-186-accepted-connections-and-the-commitment-the-metric-couldn-t-show-0a88a6a5\/): a visible action can rise while the commitment you care about remains unknown.
The first hour needs containment, not interpretation
At 4:38, Kojo stopped changing the model. He removed the public path to the demo and left a plain message explaining that access was paused. Then he preserved the request logs available to him, checked what information had entered the system, and wrote down the original test condition before the traffic distorted his memory of it.
The sequence matters.
A founder under public pressure wants to explain, repair and ship at once. Those actions contaminate the evidence. If you modify the prompt after every screenshot, you lose the ability to tell whether the original failure was consistent. If you keep the link open to collect more data, you may be collecting sensitive material from people who were never part of the intended test. If you announce a roadmap in response to comments, you turn strangers’ assumptions into product commitments.
The first hour should answer three narrower questions:
- Can continued access cause harm or expose information?
- What did the demo promise at the moment people encountered it?
- Which inputs belonged to the intended workflow?
Everything else can wait.
This is also why validation must begin with a real workflow rather than enthusiasm around an output. [Ada’s Monday and Femi’s validation decision](\/blog\/ai-product-validation-what-ada-s-monday-taught-femi-about-building-the-right-workflow-a4a5aba2\/) reached the same boundary from another direction: a convincing demonstration still leaves the underlying job unproven.
Public attention creates several products at once
By 4:51, Kojo had three piles of evidence.
The first came from the six founders he built for. Their questions concerned follow-up quality, editing time and whether the drafts matched how they spoke to prospects.
The second came from adjacent users. Recruiters and account managers could imagine related workflows, but their needs required different inputs, review steps and safeguards.
The third came from the public crowd. They treated the text box as a general assistant because that was the interface they saw.
Mixing those piles would have produced a confused roadmap. The loudest requests belonged to the least qualified audience. The most useful criticism came from people doing the original job under its original constraints.
Public distribution changes the meaning of an interface. A blank text box invites broad use. A polished result implies readiness. A shareable link removes the conversation in which a founder would normally say, “This only drafts follow-ups from these fields, and you must review every output.”
That gap between intended use and perceived permission is a product decision. It cannot be repaired by a longer prompt alone.
Monday began with a smaller demo
Kojo spent the weekend resisting the flattering interpretation that thousands of visits meant he had found demand. On Monday morning, the demo returned behind controlled access. The input accepted only the fields required for a sales follow-up. The review step was visible before anyone could copy the draft. The description named the workflow and its limits in plain language.
Most of Friday’s accidental audience could no longer use it.
That was useful.
He still had the harder commercial question ahead: would the six founders return, use the drafts in real work and ask to keep the tool? The escaped demo had revealed interface ambiguity, safety gaps and adjacent demand worth investigating. It had not answered that question.
At 9:06, Kojo sent the revised link to the original six people. His traffic graph looked almost empty again. For the first time since Friday afternoon, the number on the screen measured the test he was actually running.
Comments
No comments yet.