The demo is a different test
A pilot demo is a bounded event. The presenter chooses the inputs, conditions, and story because the purpose is to show a capability clearly. That is legitimate. The problem begins when success under those conditions is treated as sufficient evidence for production, where inputs, volume, dependencies, and users are less controlled.
Production asks at least four additional questions: how often the system is wrong and by how much; what happens downstream; what it costs at expected volume; and which role responds when a threshold is crossed. Without acceptable ranges for those questions, the pilot has no explicit pass, fail, or handoff condition.
Three specific failure patterns
Success was never falsifiable
Goals such as 'improve efficiency in support' or 'explore AI for document review' cannot be contradicted by a result. When there is no criterion that could turn out false, there is no agreed moment at which the pilot is done or should stop.
The corrective step is to write down the metric, baseline, threshold, and stop condition before building. For example: 'median handling time below four minutes on the top three ticket categories, with resolution quality no worse than the current QA baseline.' The figures must come from the actual operation; the important property is that the statement can lose.
The transition role was unnamed
A pilot champion is not automatically responsible for production operation. The receiving team needs an evaluation harness, runbook, monitoring, support responsibility, and budget before it can accept the system. If that operating role and acceptance package are unnamed, the handoff remains an open risk rather than an implementation detail.
The unit economics were never checked at scale
Pilot volume can hide costs that matter at the expected production load: tokens, retries, embedding refreshes, human review, monitoring, and the cost of incorrect output. Model those inputs using the organization's own volume and error assumptions before the pilot is approved, then update the model with observed data.
These are planning and operating gaps rather than model-selection questions. Addressing them before handoff avoids building a production path around assumptions with no recorded approver or responsible role.
What to do instead
Run the pilot as a rehearsal rather than a demonstration. The difference is the intent: a demonstration is designed to succeed, a rehearsal is designed to find out where the plan breaks while breaking it is still cheap. Practically, that means four things before anyone builds.
- State success as a number with a threshold, and state the failure condition that would cause you to stop.
- Name the production role before the pilot starts, and have them define what they need in order to accept the handoff.
- Do the volume arithmetic on day one, including retries, review, and the cost of being wrong.
- Reserve time to attack the system deliberately — bad inputs, missing dependencies, and the ways busy people will actually misuse it.
The rehearsal step is easy to compress when a demo already looks convincing. Keep it explicit: deliberately test bad inputs, dependency failures, misuse, and recovery before a customer or operator encounters them unintentionally.