From ML engineer to FDE: when a good model still goes unused

Imagine you have just raised the F1 score of an insurance claims classification model from 0.81 to 0.93. Two weeks after the demo, the customer’s claims team is still assigning work by hand in a spreadsheet. The model is not wrong. It simply has not reached the place where the work actually happens.

A comparison of the two roles on Perspective AI’s blog sums up the gap neatly: ML engineers make the model good, while FDEs make the model useful to a real organisation. The same source says FDEs are judged on whether a deployment actually changes how the customer operates.

The skills you bring still count, but the way results are measured has to change. Your test-set score is no longer what you are graded on. The new yardstick is whether end users work differently.

A new yardstick means a new working day

At Sierra, according to a summary in ZenML’s LLMOps database, FDEs and agent engineers are accountable for “measurable business outcomes” rather than uptime or the number of features shipped. The same summary describes the agent engineer as owning the full lifecycle of an agent in production and tracking it through outcome metrics.

The change in yardstick reshapes the calendar too. Perspective AI estimates that FDEs spend around 60% of each week working directly with customers, while ML engineers spend almost none.

Paraform describes the FDE as a hybrid role, sitting between software engineering and hands-on customer deployment. The core skills it lists are a backend language, cloud infrastructure, data engineering, applied AI and the ability to communicate with customers.

Applied AI is just one of those five. An article from Scaler argues that the biggest hurdle for data practitioners moving into FDE work is the step from “Insight” to “Integration”, and calls technical discovery with stakeholders the hardest part of the role.

Where does a good model get stuck?

Go back to the claims example, this time as an FDE. The first job is not to open the notebook. You need to trace the path of a single claim, from the moment it reaches the company to the moment someone finishes handling it.

Suppose you find that claims arrive from two places: email and a web portal. Both feed into a legacy claims management system, and the team lead then exports a list to a spreadsheet to assign the work.

The dataset you trained on was a CSV exported from that legacy system. You have never seen the email flow, and the team lead, the person who actually makes the decisions, works only in the spreadsheet.

Seen this way, the bottleneck is obvious. Emailed claims may never pass through the model, and the predictions do not appear where the team lead assigns work. This is exactly the kind of work ZenML points to when it describes how the FDE role at Palantir gradually expanded: integrating data from many disconnected sources.

Another point of F1 will not help here. What is needed is a pipeline that ingests both claim sources and writes the predicted label into the very column the team lead is reading.

To learn any of this, you have to ask. Below is a short script for a 30-minute discovery session with the claims team lead.

FDE: Where do you start when you assign claims in the morning? Could you share your screen?

Team lead: I export the list from the system to a spreadsheet, then assign people by claim type.

FDE: Are claims sent by email on that list?

Team lead: No, someone has to enter them by hand first.

FDE: If the “claim type” column were already filled in, which step could you drop? And when it is filled in wrongly, how would you find out?

The last question matters most. It turns a model metric into an operational one, such as the time from a claim’s arrival to its assignment. It also tells you that you need a feedback mechanism so users can report when the model gets it wrong.

Working step by step on the customer’s site

Step one is to map the process before touching the model. Sit next to a real user, capture every screen they open and note every manual step. Step two is to identify the write-back point: the system and data field the decision-maker looks at every day.

Step three is to agree on one operational metric with the customer and measure its baseline before deployment. Step four is to get the whole flow running end to end early, even if at first a simple set of rules stands in for the model.

Step five is to own the full lifecycle: once in production, track the operational metric, not just accuracy.

Familiar traps for ML practitioners

The most common trap is to keep tuning the model while the integration still does not work. The sense of progress is real, but the customer cannot see it. The second trap is treating the exported data file as the whole truth, when part of the process lives outside the system, like the email flow in the example.

The third trap is using a notebook demo to prove value. Users judge a tool on the screen they open every morning, not on your charts. The last trap is avoiding customer conversations because they feel “non-technical”, when discovery is precisely where you find the real technical problem.

Putting these skills on your CV

The Pragmatic Engineer reports very strong demand for FDE hires at Google, OpenAI and Anthropic. Before editing your CV, open a few FDE job postings yourself and count how often words such as integration, customer-facing and data pipeline appear compared with modelling terms. The numbers will tell you which experience to put first.

Then rewrite each line of your CV in this order: the operational change, how you integrated, and only then the model. For example, “connected the classification model to the claims system and the assignment spreadsheet, cutting the time to assign a handler” says more than “raised F1 to 0.93”.

In interviews, describe a time you discovered that the real problem lay outside the model. Your ML skills remain a major advantage; what you need to add is the habit of following the problem all the way through, until the model’s predictions genuinely change the customer’s work.