Alfred AnyanInsights
← All insights

The Square Canisters That Didn’t Fit, and What It Almost Cost Apollo 13

Two professionals discussing a 3D architectural design on a laptop in a modern office.

Photo by Grove Brands on Pexels

The founder should interrupt the sprint when the flaw creates a credible path to customer harm, data exposure, or loss of control. The interruption should be narrow: confirm reachability, contain the exposure, record the decision, then decide whether the promised feature can safely resume.

In April 1970, Apollo 13’s crew had moved into the lunar module after an oxygen tank explosion crippled the command module. Then carbon dioxide began accumulating. The command module carried square lithium hydroxide canisters, while the lunar module used round openings. The crew had the filters they needed, but the parts did not fit.

NASA engineer Ed Smylie and a team in Houston had to build an adapter using materials already available aboard the spacecraft. Until the crew assembled and tested it, nobody knew whether the improvised device would reduce the carbon dioxide enough. The fix worked. NASA’s official Apollo 13 Mission Report documents the accident, the rising carbon dioxide concentration, and the modified purification system used during the mission.

Classify the consequence before touching the roadmap

A security alert arriving mid-sprint creates two clocks.

One belongs to the customer waiting for the promised feature. The other belongs to the flaw, which may already be reachable in production. Treating both clocks as equally urgent wastes time. Ignoring either one creates a different kind of damage.

My first question would be concrete: what can happen if this flaw is exploited before the next planned release?

A vulnerability that exposes customer records, permits unauthorized actions, leaks credentials, or allows an attacker to cross account boundaries earns an immediate interruption. A theoretical weakness behind several unproven assumptions may earn a short investigation instead. The agent’s severity label cannot make that decision for me.

AI security tools are useful because they can trace code, suggest attack paths, and produce candidate patches quickly. They can also state uncertain conclusions with unnerving confidence. I need the engineer who understands the affected system to establish three facts: whether the vulnerable path is reachable, what assets sit behind it, and whether we can contain it without changing unrelated code.

That initial check should have a time limit. Thirty focused minutes can answer more than three hours of anxious discussion in Slack.

Give the engineer a smaller interruption

The wrong request is, “Drop everything and fix security.”

That sentence has no boundary. It can consume the sprint because “secure” has no finish line.

I would ask for a narrower sequence:

  • Reproduce or disprove the reported path in the production configuration.
  • Disable the affected capability, rotate the exposed secret, or restrict access if containment is possible.
  • Write the smallest test that fails because of the flaw.
  • Review the agent’s proposed patch against that test and the surrounding authorization logic.
  • Decide explicitly whether to merge, revise, or reject the patch.

This protects the engineer’s attention. It also separates containment from repair. A feature flag or temporary restriction may reduce immediate exposure while preserving most of the customer delivery. Sometimes the patch must ship first. Sometimes containment buys enough time to finish the promised work without gambling with production.

The difficult case is an AI-generated fix that appears correct but changes behavior nobody fully understands. I explored that decision more directly in Should We Merge an AI Security Fix We Cannot Yet Explain?. Speed helps only when the team can explain what changed, what the test proves, and what remains uncertain.

Put the commercial decision beside the technical one

The first paying customer matters. Their payment may be carrying the cloud bill, the contractor invoice, or the next month of runway. Missing a commitment can damage trust before the company has enough customers to absorb the loss.

That pressure belongs in the decision. It cannot be allowed to quietly redefine acceptable security risk.

I would tell the customer early if the containment or repair changes the delivery date. The useful version is specific: we found a production risk in the path this feature touches, we paused that work, and we will provide a revised date after validating the fix. I would avoid claims that the system was breached unless evidence supports them. I would also avoid promising that no impact occurred before the relevant logs were checked.

This is where a short decision record earns its place. Capture the alert, the engineer’s reachability assessment, the containment action, the person approving the next step, and the condition for returning to feature work. If the same question appears on Thursday, the team should not have to reconstruct Tuesday from chat messages.

Decide the restart condition before the interruption expands

Apollo 13’s engineers were solving a bounded problem. The adapter had to use materials aboard the spacecraft and reduce carbon dioxide. Those constraints focused the work.

A founder needs the same discipline at 11:17 on Tuesday. Define what must become true before the sprint resumes: the exposed path is disabled, the failing test passes, the patch has been reviewed by someone other than its generator, and production monitoring shows the expected behavior.

Then restart.

Do not turn one credible flaw into an unplanned audit of the entire product. Put adjacent concerns into a separate queue, assign owners, and preserve the evidence that made them worth revisiting. The immediate job is to remove the dangerous path without losing the week to every weakness the alert happens to suggest.

Before the next sprint begins, write this response path down. Specify who may interrupt delivery, what evidence they need, how containment differs from repair, and who informs the customer. The next alert should trigger a practiced decision, not another argument at 11:17.

Comments

No comments yet.