Paid or free AI pilot: three things an FDE must settle before day one

Picture the final week of a pilot. Your invoice reconciliation agent is running well, the head of accounting praised it in the demo, and when you ask about next steps, the answer is: “Let’s see if there’s budget next quarter.”

Nobody says no. Nobody has the authority to say yes either. Weeks of work on real data now hang in the air, and you have no grounds for pushing further.

It is easy to treat this as sales’s problem. Price and contract are, but the baseline, the pass threshold and the method of measurement can only be written accurately by someone sitting close to the customer’s data. So the skill worth building is settling those terms on a single page before the pilot starts.

Is payment the answer?

Jason Lemkin of SaaStr argues that well-run paid pilots convert to annual contracts at above 70%, ranging from about 60% to more than 90%. On free pilots he is blunt: “unpaid pilots are a waste of time”.

The exception he accepts is a free trial when deployment is very easy, and an agent that touches a customer’s data and permissions is rarely that easy.

For AI specifically, the numbers are far lower. IDC, cited by CIO, found that for every 33 AI proofs of concept only 4 reached production. Nadkarni of IDC observed that many gen AI POCs were “severely underfunded or not funded at all”.

Investor Brian Lewis of Millennium, speaking at the AI Engineer World’s Fair, estimated that about one in four pilots ends up as a contract, and that pilots are getting shorter.

Do not set these figures against each other and conclude that charging will triple your close rate. They measure different things, and customers willing to pay were more serious from the start.

According to Stackmatix’s playbook, the fee brings accountability on both sides: inside the customer, someone has to answer for the outcome.

So a more useful question than “paid or free” is this: if the pilot fails, who on the customer’s side loses what?

Three things to have before day one

The first is the criteria. Stackmatix requires criteria that are measurable, time-bound and tied to a business outcome the customer cares about. SoftwareSeni gives the reason: when criteria are set after results are in, the go/no-go decision no longer has an objective basis.

Next comes a named buyer, with authority over budget, involved from the start. Without this person, nobody can say yes at the end of the pilot, exactly as in the opening scene. A team lead praising the agent is a good sign, but praise is not a signature.

Last is the conversion path. When a pilot cannot charge, Stackmatix advises designing the conversion route and an agreed contract price in advance.

For established vendors, Lemkin says many have flipped paid pilots to opt-out, meaning the contract continues automatically unless the customer says stop, and churn is lower than with opt-in. A young company may not yet have the standing to do this, but it is worth knowing so it can be proposed when the opportunity arises.

Do not worry that raising price early is indelicate. Redress Compliance, a firm that advises Palantir buyers on negotiation, also tells customers to demand a production price list before running a new pilot.

If buyers are already being told to ask about price early, putting the number on paper yourself simply answers a question that was coming anyway.

One page for an invoice reconciliation pilot

Take a hypothetical example: a logistics company wants an agent to reconcile invoices. There are 400 invoices a day, each taking 6 minutes, which comes to 2,400 minutes or 40 labour hours. You want the agent to handle 60% on its own, or 240 invoices, with a person spending only 1 minute checking each of those.

The calculation yields the target: 240 minutes of checking is 4 hours, the remaining 160 invoices done by hand take 16 hours, for a total of 20 labour hours a day. The customer is not buying “an agent that handles 60% of invoices”. They are buying a halving of labour hours, so that is the number that goes on the page.

Item	What is settled before the pilot
Baseline	40 labour hours/day, measured from logs for the 2 weeks before the pilot
Pass criterion	At most 20 labour hours/day, measured over the final 2 weeks
Safety threshold	Errors slipping through no more than 1% on the audit sample
Duration	4 weeks
Decision-maker	Chief financial officer, attends the kick-off and the closing meeting
If passed	Sign an annual contract at the price stated on this page
If near miss	Above 20 and up to 24 hours: one extension only, of 2 weeks, naming the errors to fix
If failed	Above 24 hours or over the error threshold: stop, hand the data back and record the lessons

Suppose the agent ends the pilot handling only 55% on its own. That means 220 invoices checked in 3 hours 40 minutes and 180 done by hand in 18 hours, for a total of 21 hours 40 minutes.

Without the page, the meeting becomes a debate over whether 55% is good or not good enough. With it, there is nothing to debate: this is a near miss, so extend by 2 weeks and fix exactly the error group that has been named. Even when the number is bad, both sides know what to do next.

The same calculation reveals the remaining boundary: at 48%, or 192 invoices checked and 208 done by hand, the total is exactly 24 hours; at 45%, the total is 25 hours and the pilot stops. So you know from the outset how much the agent must handle on its own for the pilot to keep its chance of an extension.

What to ask in the scoping session

These questions sound ordinary, but each one fills a box on the page. The following dialogue is a sample script:

FDE: How long does one invoice take at the moment, and where does that figure come from?

Customer: Just a few minutes.

FDE: Could we have two weeks of logs from the accounting system? Both sides will then use the same measured figure as the benchmark.

Customer: Sure, I’ll have accounting export them.

FDE: Suppose labour hours fall by half. Who decides on the purchase, and where does the budget sit?

Customer: The CFO, but she’s very busy.

FDE: She only needs to attend the first session and the closing one. If she nods through the criteria at the start, there will be someone to make the decision at the end of the pilot.

If the customer refuses both the logs and a meeting with the buyer, you have valuable information before the pilot begins. Tell sales and reconsider the scope before putting effort into integration.

Do it yourself: from measurement to signature

Start with the baseline, because without it every criterion is just a feeling. Then sit down with the direct users to set thresholds, including the safety threshold, and convert everything into the units the buyer cares about: labour hours, processing time, errors that slip through.

The next step is to give the draft to sales to fill in price and contract form, then invite the buyer to approve it before kick-off.

Finally, keep the pilot short. Brian Lewis observes that short pilots leave promises about the future less room to make up for missing functionality. That forces you to choose one narrow process and make it actually work, rather than present a roadmap.

The mistakes that leave a pilot looking good but unsigned

The first mistake to avoid is using technical metrics as criteria. Accuracy or F1 helps you debug, but the buyer needs to see a business outcome, so keep those as internal metrics.

The second is mistaking a supporter for a buyer. An enthusiastic user helps you get data, not necessarily budget. The third is allowing unlimited extensions: a free pilot that drags on forever is the politest form of rejection, so state clearly that there is only one extension.

The last is leaving price open because “we’ll work it out once there are results”. Redress writes from the buyer’s side that a pilot without terms hands the vendor the advantage on price. Customers who think this way will be on their guard, and a guarded customer is slow to sign, so early transparency works in your own favour.

If a pilot has to be free, do not give it away. Trade it for logs, users’ time, the buyer’s presence and an agreed price, because those are what make a customer truly commit.

How to put this skill on your CV

A line such as “designed acceptance criteria from a baseline measured on logs, worked with sales to settle the conversion path” says more than “deployed agents for customers”, provided you have really done it.

When reading job postings, look out for the words scoping, success criteria and pilot conversion, then ask directly at interview: do FDEs take part from the moment criteria are written, or do they only receive the work after signing? Keywords are only a hint; the answer tells you what the job really is.

At interview, tell the story of a pilot through a calculation like the one above, even if it failed. How you made a decision on a bad number is the story most worth telling.

Exercise for this week

Take your most recent pilot, successful or not, and fill in the page above with real numbers. Any box you cannot fill is where that pilot was left to chance.

The next time you hear “let’s see next quarter”, you will know which week the conversation went off course, and that time you should ask sooner.