FDE Times dataFDE jobs open 924▲33 Change in open roles over 7 days, like-for-like: roles from companies we started tracking this week are left outNew in 7 days 80Companies hiring 196Remote-friendly 25%Median US pay $200kTop hirer Databricks 107
中文VI

The newspaper of the Forward Deployed Engineer

Guides

Three weeks of code lost to one question about where data lives

For FDEs building inside a customer's network, a one-page design doc written in two days surfaces disagreements while they are still cheap to fix.

ShareLinkedInFacebookX

In brief

  • For an FDE, code runs inside the customer's systems, so the design doc is the cheapest place to find constraints on data, security and operations.
  • The most valuable sections are usually the non-goals and the alternatives considered, because they show the boundaries and the cost of each choice.
  • Write the doc for the people who will run the system: the runbook and eval suite belong in the plan from day one.
The ten sections of a design doc, stacked from top to bottom: status quo, goals, non-goals, infrastructure constraints, design, options, SLA and errors, testing, monitoring, handover. The infrastructure constraints section is highlighted in orange and annotated: data, model, security, operations. It sits directly above the design section.
The design doc template in the article places the customer's infrastructure constraints ahead of the proposed design. Four questions to answer before writing code: where the data may live, where the model runs, who signs off on security, and who operates the system after handover.

Picture your first week at a bank. You have built a prototype assistant that looks up internal procedures. It runs smoothly on your laptop and calls a model through a public API. In week three the bank’s security team tells you internal documents may not leave their network. Nearly all the integration work has to be redone.

This kind of risk comes with the job, because an FDE works in someone else’s house. Anthropic’s FDE job posting names a core responsibility: working inside customer systems to build production applications with Claude models. Your code has to fit the customer’s network, approval processes and operations team.

So prompting and RAG are not the only skills worth learning early if you want to be an FDE. Writing a design doc before you code is another. The reason is simple: a page written over two days costs far less than three weeks of code that went the wrong way.

One page that exposes disagreement

A design doc does not specify every function. At Google, a design doc records the high-level implementation strategy and the key design decisions, with an emphasis on the trade-offs considered. It answers three questions: what are we going to build, how, and why?

Writing it down is the fastest way to find out whether everyone really agrees. Gergely Orosz of The Pragmatic Engineer observes that if the team already agrees on the approach, writing it down is a small task. Disagreements tend to surface during the writing itself. The page brings them out early, while changes are still cheap.

A shared document also creates accountability, and according to Orosz it almost always leads to better-considered decisions. For an FDE the benefit is practical. Once decisions about data or scope are written down and the customer has read them, there is less room later for each side to remember things differently.

Not every task needs a doc, though. Make the doc only as long as the work is complex. The companies Orosz surveyed tend to reserve RFCs for broad, complex projects, company-wide technical requirements and API changes.

Start from what happens today

Ask “what is the standard process?” and you get tidy documentation. Ask “the last time you needed to look up a procedure, what did you do?” and you get the truth. Pooya Golchian, who worked as an FDE from 2024 to 2026, sums up the principle: ask about what happens today, not what is supposed to happen.

In the bank example, the answer might be that staff dig through a shared folder and, if they cannot find what they need, message a long-serving colleague. That detail belongs in the first section of the doc.

It tells you that your real competitor is a colleague answering over chat. The system’s answers have to be fast, with enough citations that users trust them.

Only then do you write the goals. Straight after the goals comes the section newcomers skip most often.

Non-goals: where you draw the boundary

A non-goal is not a goal written in the negative. “The system must not be slow” is a goal in disguise. Google’s guidance defines non-goals as things that could reasonably be goals but are deliberately chosen not to be.

For the bank’s assistant, real non-goals might be: no answers for external customers yet, no handling of scanned image PDFs yet, no real-time index updates yet. All three are reasonable, someone will ask for each of them, and each is excluded on purpose.

You do not need to list everything left out. Write down the things that are reasonable, that have come up in review, or that would be costly to argue about again. That is how Joshka describes the “document intentional non-goals” pattern.

If you skip this, three months later someone at the customer will ask why the system cannot read scanned files, and a scoping decision will be treated as an oversight.

Where does the customer’s infrastructure enter the design?

Uber’s RFC template has sections on service SLAs, dependencies, load and performance testing, security, and metrics and monitoring. For an AI application in a customer’s infrastructure, each section becomes a set of questions you must ask before writing code.

For the bank in the example, those questions are: Where may the data live: only on the internal network, or in an approved cloud region? Does the model run on the customer’s on-prem GPUs or through an external service? How long does security review take, and who signs off? And the one most often forgotten: after handover, who operates the cluster?

The SLA section should include failure scenarios. For example, if the GPU cluster is overloaded, the system falls back to a smaller model and gives shorter, less detailed answers that still carry citations. If the model goes down entirely, the system returns only a list of relevant documents. Writing this down in advance shows the customer what they are signing up for.

Next comes “Alternatives considered”. List other designs that could achieve a similar result so reviewers can see the trade-offs. For the bank, the comparison might look like this:

Option Gains Costs
Model via cloud API Fast to deploy, high quality May conflict with rules that data must not leave the network
Open model on the customer’s on-prem GPUs Data stays in-house Customer needs people to operate the GPUs
Search only, no generated answers Simple, easy to clear security review Users still have to read the documents themselves

The table does not make the decision for you. It keeps the meeting with the customer on real trade-offs rather than technology preferences.

A design doc skeleton

Put these pieces together and the first draft for the bank’s assistant can come in under two pages:

# Internal procedure lookup assistant — design doc v0.1

## What happens today
- Staff dig through the shared folder; if they can't find it, they ask a long-serving colleague

## Goals
- Answer questions about internal procedures, always citing the source document

## Non-goals (with reasons)
- Not serving external customers yet: legal risk, needs separate approval
- Not reading scanned PDFs yet: needs OCR, infrastructure not in place

## Infrastructure constraints
- Where data may live / where the model runs
- Who approves security, and how long it takes
- Who operates the system after handover

## Proposed design
- Open model on on-prem GPUs, searching the shared folder,
  every answer links to the passage in the source document
- Pending security team confirmation: data does not leave the internal network

## Alternatives considered
- Cloud API / open model on-prem / search only (see trade-off table)

## SLA, dependencies and failure scenarios
- GPU overload: fall back to a smaller model, still with citations

## Testing and rollout
- Trial with 10 staff in one department for 2 weeks before expanding

## Metrics and monitoring
- Share of answers with correct citations, number of answers users report as wrong

## Handover: runbook + eval suite
- Eval suite: questions taken from times staff had to message
  a long-serving colleague, with correct answers and documents
- The bank's operations team reruns the evals whenever the model
  changes or documents are updated
- Runbook: GPU overload, stale index, answers missing citations

Write for the people who will run it, not for yourself

The final test of a design doc is whether the customer’s team can run the system on their own. Golchian defines the FDE by exactly that: someone who builds software inside a customer’s company until their team can run it themselves. In the handovers he describes, the customer’s engineers take over with a runbook and an eval suite.

So the last section of the doc should not sit empty until the final week. As in the skeleton above, write down from v0.1 which question sets make up the eval suite, who at the customer will run it, and which incidents the runbook covers.

Over a few projects, this skeleton thickens into an asset. Anthropic’s job posting also expects FDEs to identify and codify repeatable deployment patterns and feed them back to the product team. A design doc template refined across three customers is exactly such a pattern.

Common mistakes

The first mistake is writing the doc after coding, which turns it into an illustration of decisions already made. The second is non-goals that simply negate the goals, such as “not slow” or “not wrong”, which exclude nothing.

The third is an alternatives section filled with straw-man options that exist only to be rejected. Reviewers on the customer side see through this at once, and you lose credibility just when you need their sign-off. The fourth is writing a long doc for a small change, so nobody on the team wants to read it.

The last mistake is the most expensive: not asking who will operate the system after you leave. The answer shapes almost every infrastructure choice above.

If you want to demonstrate this skill when applying, put a design doc with sensitive details removed into your portfolio. When reading job descriptions, look for phrases such as “work within customer systems” in Anthropic’s posting. Hiring managers will see that you think about the customer’s systems before you think about the model.

A prototype on a laptop proves an idea can work. A design doc proves it will keep working after you have left the customer’s building.

Was this article useful?

Use with your AI assistantAsk Claude ↗Ask ChatGPT ↗
6 sources
Read next on the roadmap · Stage 2: Broad engineeringPick the wrong OAuth2 grant and it works today, then breaksAsk who is calling the API, keep the secret out of the repo, and rotate it without breaking the nightly job.