# Why FDEs should learn evals before they learn fine-tuning

> In the FDE job ads compared here, only an AI infrastructure company asks for deep fine-tuning, and the decision to fine-tune usually starts with an eval set.

Original: https://fdetimes.net/en/analysis/fde-learn-evals-before-fine-tuning/

Together AI's job ad for an FDE, Post-Training lists a string of acronyms that could send a reader straight to a course sign-up page: SFT, LoRA, DPO, GRPO, PPO. The company wants candidates with expert-level hands-on experience in fine-tuning or RL for LLMs.

Read in a hurry, the ad suggests that fine-tuning is the ticket into FDE work in the AI era. Put it next to a few other FDE job descriptions, however, and that conclusion wobbles. Across the ads and first-hand accounts below, the demand for deep fine-tuning expertise comes from only one AI infrastructure company, and even there it does not stand alone.

For an engineer with a few years of experience preparing to move into an FDE role, the practical question is not whether to learn fine-tuning. It is when to learn it, and what to learn first so as not to spend six months on a skill your customers may never ask about.

## At Together AI, fine-tuning does not stand alone

Together AI has two FDE roles that touch fine-tuning, and both show it bundled with something else. The Post-Training ad, alongside its list of algorithms, also asks for experience leading complex technical projects directly with customers.

The second role, FDE Strategic Inference, describes the hire as a deep specialist in inference optimisation, fine-tuning pipelines and production deployment. The ad names the tools: inference engines such as vLLM, TensorRT-LLM and SGLang. The day-to-day work is choosing, configuring and optimising engines based on hardware, model architecture and workload characteristics.

The way both ads are written shows fine-tuning as one stage in a production line, not the destination. A model that has been fine-tuned still has to run on a specific GPU, under a specific traffic pattern, for a specific customer. The person who can handle the whole chain is the person Together AI needs.

## Infrastructure customers ask about latency, not DPO

An account from an FDE at Baseten, an inference platform, makes the point sharper. The engineer describes the main goal as helping customers put open-source models, fine-tuned models or custom models behind an API endpoint.

Note where the word "fine-tuned" sits in that sentence. The fine-tuned model is what the customer brings; the FDE's job is to make it run. The typical question cited is how to cut serving latency tenfold while keeping costs the same.

Treat that question as a problem to solve. If a request currently takes 2 seconds, the customer wants 200 milliseconds on the same GPU bill. DPO or GRPO will not easily solve that; the answer lies in choosing the engine, the configuration and hardware suited to the model architecture, exactly what the Strategic Inference ad describes.

## An application-layer example: the real work is retrieval and evals

On the side of companies selling models to enterprises, one example deserves a close look. On its project with Morgan Stanley, OpenAI's FDE team had to develop retrieval-tuning techniques so that research reports were found accurately and trusted by users. They tuned the search step, not the model.

The team also treated evals as mandatory: an LLM-based application is not complete without an evaluation set that proves it works.

OpenAI's guide to optimising LLM accuracy makes the same point as a sequence: prompt engineering is usually the best place to start, and from there you need a solid eval set as a baseline before considering RAG or fine-tuning.

From these two sides, two kinds of FDE emerge, with different skill sets despite sharing a job title. The right-hand column rests on a single project, Morgan Stanley, plus OpenAI's general guidance, so read it as an example for comparison, not a rule for the whole application layer.

| Aspect | AI infrastructure FDE | An application-layer example |
|---|---|---|
| Source of the example | Together AI, Baseten | OpenAI's FDE team with Morgan Stanley |
| Question the customer brings | Cut latency, hold costs, run a fine-tuned model | Are the reports found accurate and trustworthy? |
| Main work | Choosing and configuring inference engines, fine-tuning pipelines | Tuning retrieval, building evals |
| Where fine-tuning sits | An explicitly listed skill, paired with serving | Per OpenAI's guidance: considered after prompts, once evals provide a baseline |
| Tools named | vLLM, TensorRT-LLM, SGLang | An eval set as baseline |
| What must be proven | Having led technical projects with customers | An application with evals confirming it works |

What the two columns share is not fine-tuning. It is the ability to measure how good a system is and to take it into production for the customer.

## When fine-tuning finally gets its turn

Imagine you are an FDE at a bank that wants an internal assistant to answer questions about analyst reports. You build 100 questions with reference answers, run them with prompts alone and, say, get 62 right. That is your baseline.

Now read the 38 wrong answers carefully. Suppose 25 are wrong because the system pulled the wrong report, 8 because the answer had the right substance but the wrong format, and 5 because the model did not understand internal terminology. The first 25 are a retrieval problem, exactly the kind of work OpenAI's team did at Morgan Stanley; the next 8 can usually be fixed with prompts.

Only once those two groups are dealt with, and the last group of errors persists, does fine-tuning become a reasonable candidate. And at that point, according to OpenAI, the quality of training data matters more than its quantity. So the decisive skill is selecting and cleaning a few hundred good examples from the very questions the system got wrong, not memorising the DPO loss function.

**Key point:** For an FDE, fine-tuning is the last answer, not the first question.

## A learning order for those aiming to get in

From these descriptions a sensible order emerges. Start with evals and prompts, because they are the baseline every later decision rests on. Next come retrieval and data skills: retrieval is what OpenAI's team had to do at Morgan Stanley, and data quality is what OpenAI ranks above quantity when fine-tuning.

The third step is serving: run an open-source model on vLLM or SGLang, measure latency and cost, change the configuration and measure again.

This is the skill the Strategic Inference ad names, and the one the Baseten account revolves around. Only after that, if you are aiming at infrastructure companies, go deep on SFT and LoRA, then DPO and RL methods such as GRPO and PPO.

When reading job ads, use the table above as a rough filter. An ad that names inference engines and post-training algorithms is most likely an infrastructure-layer role; there, fine-tuning is worth investing in early.

An ad centred on accuracy and reliability for enterprise customers may sit closer to the Morgan Stanley example; there, a good eval project probably carries more weight than a fine-tuning certificate.

On your CV, do not write "experience with fine-tuning" as a bare skills line. Turn it into a story with numbers: what the baseline was, how many groups the errors fell into, how much fixing retrieval raised the score, and why you decided to fine-tune or not. If you are targeting the infrastructure layer, add a line on latency before and after you changed the serving configuration.

Do not forget the half of the requirement that Together AI places right beside its list of algorithms: leading technical projects with customers. The string of acronyms may be the most impressive line in the ad, but it is the eval set that tells you whether the customer needs any of them.

**Try this week:**

- Build a small eval set for an LLM task you are working on, run a prompt-only baseline and record the score.
- Classify each wrong answer by cause (missing document, wrong document retrieved, wrong format) before thinking about fine-tuning.
- Reread three FDE job ads you are interested in and mark which ones name an inference engine: those are most likely infrastructure-layer roles.

## Sources

- [Forward Deployed Engineer, Post-Training (Together AI)](https://jobs.generalcatalyst.com/companies/together-ai-2/jobs/97989008-forward-deployed-engineer-post-training)

- [Forward Deployed Engineer, Strategic Inference (Together AI)](https://refresh.cv/jobs/together-ai/45566a30-07a9-45b3-bb3e-ab5bda7e800e)

- [What I Learned As A Forward Deployed Engineer Working At An AI Startup](https://www.baseten.co/blog/what-i-learned-as-a-forward-deployed-engineer-working-at-an-ai-startup/)

- [Optimizing LLM Accuracy (OpenAI API docs)](https://developers.openai.com/api/docs/guides/optimizing-llm-accuracy)

- [Forward Deployed Engineering: Bringing Enterprise LLM Applications to Production (ZenML LLMOps Database)](https://www.zenml.io/llmops-database/forward-deployed-engineering-bringing-enterprise-llm-applications-to-production)