The pilot proved the model. Nobody had agreed whose job changed
Most stalled AI projects were not beaten by the model. They were beaten by the fact that shipping one means changing what a named person does on a Tuesday, and that was never on the plan.
Because a pilot proves the model can do the task, and production requires somebody to stop doing it. That is an operational decision with an owner, a handover and a failure path, and pilots are usually scoped to avoid all three.
What we used to get wrong
We used to scope the first phase around accuracy: get the extraction above some percentage and the rest would follow. It did not follow. We delivered work that passed its own test and then sat in a branch, because nobody had answered what happens when it is wrong at four in the afternoon and the person who used to catch it has been moved.
The accuracy was never the blocker. The blocker was that going live meant somebody accepted a new failure mode on behalf of their team, and no phase of the project had asked them to.
What a production plan has that a pilot does not
Three things, and none of them are technical. A named owner who accepts the output. A written description of what a wrong answer looks like and who sees it. And a way to turn it off that does not require us.
- Who accepts the output, by name, not by department
- What a wrong answer looks like and who is shown it
- How the old process is restarted, and who is allowed to restart it
- What the first month is measured on, agreed before launch
How we scope it now
The first phase now ends with one workflow live for one team, with the fallback written down, rather than with a model that scores well on a held-out set. It is a smaller claim and it survives contact with an actual Tuesday.
What to take from this
- 01Pilots fail at the handover, not at the model
- 02Production means a named person stops doing something
- 03A rollback nobody needs us for is part of the deliverable
- 04Scope the first phase to one live workflow, not to an accuracy number
Ask the question directly
- Does this mean you will not run a pilot?
- We will, when the question is genuinely whether something is possible. What we will not do is run one where the answer is already known and the real risk is operational.
- Who usually ends up being the owner?
- Whoever currently absorbs the error when the manual process goes wrong. If nobody does, that is worth knowing before any of this is automated.
Related answers
AI transformation
Most AI projects fail because they add a chat box next to the problem instead of removing the step that costs the hours. We start from the workflow, and from the systems and the data underneath it.
SEO or Google Ads first?
Bring the decision you are stuck on
A call, forty five minutes, no deck. We will tell you if we are the wrong firm.