From POC to production: why so many AI projects stop at the demo

The demo has just finished and the client is happy. Then the head of operations asks what sounds like a simple question: “When can my team actually use it?” If you answer “in a few weeks” because the demo went smoothly, you have probably just promised something you have not estimated.

A great many AI projects stall at exactly that question. Gartner predicted that at least 30% of GenAI projects would be abandoned after the proof-of-concept stage by the end of 2025.

MIT NANDA’s report The GenAI Divide goes further, putting the failure rate for enterprise AI solutions at 95%. That figure, however, comes from preliminary research on a small sample and should not be taken as an industry-wide rate.

For a forward deployed engineer, this matters directly. You are hired to get systems running in production at the client’s site. The most valuable skill you can have is to see the gap between POC and production on day one, not to discover it in month six.

A demo answers a different question from production

A demo answers the question “can the model do this job?” Production has to answer a harder one: “can this system survive inside the client’s organisation?” The work needed to answer each question differs enormously.

The paper “Hidden Technical Debt in Machine Learning Systems” by Sculley and colleagues at Google, presented at NeurIPS 2015, made this point more than ten years ago. In many ML systems, only a very small fraction of the code actually does learning or prediction.

The authors estimated that a mature system might be at most 5% ML code and at least 95% glue code. They also found that real-world ML systems tend to carry very large maintenance costs that continue for a long time.

A demo consists almost entirely of that 5%. So when you estimate a timeline from the demo, you are estimating a small part of the project.

Five gaps the demo hides

Gartner names four main reasons GenAI projects stop after the POC: poor data quality, inadequate risk controls, escalating costs and unclear business value. HatchWorks, a services company that runs its own blog on forward deployed engineering, adds a fifth gap: integration.

The data gap is usually discovered late. HatchWorks puts it briefly: the pilot runs on a curated extract, while production has to run on live data sources, which are messy and constantly changing. The integration gap tends to be underestimated.

According to HatchWorks, connecting to existing systems often accounts for most of the actual work, yet it is frequently left out when the budget is set from the demo.

Cost is the hardest gap to predict. Rita Sallam of Gartner has observed that GenAI has no one-size-fits-all solution, and its costs are harder to predict than those of other technologies.

MIT also points to a root cause that a demo cannot show, which it calls the “learning gap”: generic AI tools do not learn from or adapt to an enterprise’s workflows. According to MIT, that is where the problem lies, not in model quality.

The learning gap should not be treated as a sixth gap. It makes more sense to read it as a question that sits inside the integration row: is the system wired into the workflow tightly enough to learn from the way users correct its mistakes?

An example: an invoice-reading bot for the accounts team

Imagine you are building a POC for a logistics company: a pipeline that reads PDF invoices and extracts the tax ID, amount and date so the accounts team no longer has to key them in by hand. The client sends you a few dozen sample invoices, every one of them clean and correctly formatted. The demo gets almost everything right.

Now go through the five gaps in turn. On data, the live source is actually an email inbox containing skewed scans, phone photos and invoices from foreign suppliers. On integration, the results have to be written into the ERP, pass through the chief accountant’s approval step, and there has to be a way to correct the bot when it misreads something.

On risk, if a wrong tax ID makes it into the books, who is accountable, and where are the logs? On cost, how many tokens does each invoice consume at real volumes, not across a few dozen demo files? On value, what is the client measuring: accountant minutes spent per invoice, or the number of errors caught at audit?

Then there is the learning-gap question: when an accountant corrects a misread field, does that correction feed back to improve the system, or does it simply vanish? If it vanishes, users will fix the same error over and over, and at some point they will abandon the tool.

The first thing to do is measure again on live data. The code below is very simple, but it often changes the conversation with the client entirely:

import random

def accuracy(pipeline, samples):
    ok = sum(pipeline(s["file"]) == s["expected"] for s in samples)
    return ok / len(samples)

curated = load_labeled("demo_set/")            # set curated by the client
live = random.sample(load_inbox_last_30_days(), 20)
live = label_by_accountant(live)               # ask the accountants to label

print("demo:", accuracy(extract_invoice, curated))
print("live:", accuracy(extract_invoice, live))

If the two numbers differ widely, you have concrete evidence for renegotiating the project’s scope, instead of having to explain the delay after go-live.

A checklist before you promise a go-live date

From the example above you can draw up a template that works for most projects. Fill it in with the client, right after the demo:

Gap	What the POC has	What production needs	Who owns it on the client side
Data	A curated sample	Results measured on a random sample from live sources	Owner of the source system
Integration	An output file or demo screen	Writes to the real system, has an approval step, user corrections flow back into the system	IT, process owner
Risk	Nothing	An approver, logs, a rollback plan	Compliance, head of department
Cost	A few dozen calls	An estimate at real volumes, with alert thresholds	Budget holder
Value	“Looks great”	One metric measured before and after	Project sponsor

Every empty cell is a risk, and that risk must be written up as its own work item with its own estimate. The integration row in particular should be estimated separately from the AI work. If you lump them together, the finished demo will drag it down to a number that is far too low.

Mistakes that leave projects stuck in pilot

The first mistake is treating demo accuracy as a commitment. That number holds only for the dataset the client chose, and clients usually choose their best files.

The second is planning as though the project were only ML, when, according to Sculley’s team, most of the system is glue code and maintenance costs are usually large and long-lasting.

The third is building everything from scratch. The MIT report found that buying from specialised vendors and working in partnership succeeded about 67% of the time, far more often than internal builds.

For an FDE, this means making use of what already works and putting your effort into fitting the system into the client’s workflows.

The last mistake is leaving business value vague until the contract comes up for renewal. When nobody can measure what the project has saved, the client’s sponsor finds it easiest to cut.

How to show this skill when job hunting

If you are moving from backend or data work into an FDE role, integration experience is a real advantage, because, according to HatchWorks, connecting to existing systems often makes up most of the actual work on a project. When reading job descriptions, look for phrases such as “deployment”, “integration with customer systems” or “own the path to production”.

These phrases tell you the company already understands that the POC is only the first step.

On your CV, do not just write “built a RAG chatbot”. Spell out which system you took from pilot to production, which of the client’s systems you connected it to and which metric you used to measure its impact.

In interviews, if you can explain how you found the gap between the demo set and live data, you are demonstrating exactly the skill FDE employers are looking for.

The next time someone asks “when can we actually use it?”, answer with the five-gap checklist rather than a rough guess.