Building Systems with the ChatGPT API: three ideas worth keeping for FDEs

The most valuable lesson in this course lasts just 19 minutes. It sits inside a course whose landing page lists a total running time of 1 hour 55 minutes and asks only for basic Python.

Building Systems with the ChatGPT API is a short DeepLearning.AI course taught by Isa Fulford, Member of Technical Staff at OpenAI, and Andrew Ng. Aimed at beginners, it comprises 11 videos, 9 code examples and one graded quiz, and was updated on 14 October 2025.

If you have a few years of work behind you, the code will not trouble you. The reason to take the course is that it offers a way of thinking about LLM systems that an FDE can use straight away at a customer. Watch it in an evening, keep the three ideas below, and do not treat it as a ticket to production.

Break it down so you know where the system fails

The first idea is also what the course promises to teach: automating complex workflows by chaining model calls, splitting a large task into a sequence of subtasks.

The 19-minute Chaining Prompts lesson teaches exactly that technique. Anthropic describes the same design pattern in Building effective agents: each LLM call processes the output of the previous one.

Imagine a logistics company that wants to automate replies to complaint emails. One giant prompt could do it all, but when it goes wrong, you cannot tell whether it misread the order number or misapplied the policy. Split the work into four steps (classify, extract the order number, look up the information, draft the reply) and you can measure each one.

That measurement matters because errors compound quickly. Suppose each step is 95% accurate, errors are independent, and one wrong step ruins the whole email: the full chain is right only 0.95⁴ of the time, or about 81%. Out of 1,000 emails, about 185 will have a problem, even though each step looks fine on its own.

When is a prompt chain the wrong choice?

The course teaches the technique, but in the field the harder question is whether to use it at all. Anthropic states the condition fairly clearly: the pattern fits best when a task can be divided cleanly into fixed subtasks. Customers’ work is rarely so well behaved.

Back to the logistics company. If late-delivery complaints always pass through the four steps above, a fixed chain is enough. But if one email demands a refund, asks about the contract and attaches a photo of broken goods, the order of steps has to change case by case, and a rigid chain starts to break.

Before building, take 20 real emails and write out by hand the steps each one needs. If the lists are almost identical, go ahead with a prompt chain.

If every email is different, do not force them all into one chain. Keep the classification step at the front, then branch into short chains, each for a fixed group of tasks such as late delivery, refunds or contracts. Split an email with several requests into separate requests, then send each one down its own branch.

Whatever fits no branch should go to a member of staff and be logged, rather than leaving the model to guess. Suppose 15 of the 20 emails match three branches: you can automate 75%, each branch can still be measured separately, and the remaining 5 emails show which branch to add next. Give the customer that number before committing to a timeline.

Control both input and output, but do not copy the details

The second idea is built into the structure of the course: Moderation and Check Outputs are separate lessons. The message worth keeping for an engineer is that control has to be a step in the system, on both input and output, not a reminder tucked inside a prompt.

The technical details of the Moderation lesson have aged, so keep the idea and do not copy the method. Before writing code, read OpenAI’s current moderation documentation: the endpoint is free, the omni-moderation-latest model accepts both text and images, and image files can be up to 20 MB.

If the customer’s workflow includes images, such as the photo of broken goods in the example above, they must be taken into account when you design the moderation step.

Moderation also cannot replace business checks, because it knows nothing of the customer’s refund policy. At the first working session, ask two questions: what data may be sent to an external API, and what kind of wrong answer would do the most damage. The second question is, in effect, the brief for your output check.

Evaluation is where the course stops too early

The third idea is that evaluation must be treated as a piece of work in its own right. The course devotes a three-part block to Evaluation, and that is the right signal. But the block stops at the fixed test set of a short course: it does not reach the real output of a system after deployment at a customer.

Hamel Husain, in Your AI Product Needs Evals, recommends two further practices. The first is easy to start at once: unit tests for an LLM, in his account, are simply assertions like those you already write in pytest, and they are fast and cheap to run.

For the email system, the assertions could be these: any order number in the reply must exist in the data that was looked up, and refunds may be mentioned only when the email has been correctly classified into that group.

The second is harder, because it is a habit to sustain rather than code to write once. Husain writes that you must never stop looking at the data, and that there is no shortcut. At a customer, schedule a weekly read of real output for the first few weeks after deployment; each new error you find becomes a new assertion.

What order to study in, and what to skip

This course follows on from ChatGPT Prompt Engineering for Developers, but the earlier course is not required. If you have written prompts for a real product, go straight in; if you have never called an LLM API, taking the earlier course first will make this one easier.

The course suits engineers preparing to move into FDE work best. Once in, follow the table below: chain the steps first, then block errors at input and output, then evaluate.

You need the chain of steps to know where to put the blocks, and you need a complete system to have something to evaluate. If you already run LLM systems, you can skip the rest and watch only the Evaluation block.

Order	Part of the course	How to study it	Use at a customer
1	Chaining Prompts	Study closely, run the code yourself	Map the workflow into measurable steps
2	Moderation	Take the idea, read the current documentation	Set up an input moderation step
3	Check Outputs	Study closely	Write rules that block the costliest errors
4	Evaluation (three parts)	Study closely, add assertions	Build a test set from real data
5	Chain of Thought Reasoning	Skim	Try it when a step needs reasoning before it answers

On your CV, do not stop at the course title. Build a small project on public data and state clearly how many steps you split the workflow into, how accurate each step was, and which assertion caught which error.

When reading FDE job descriptions, look out for two words: “evals” and “production”. If a posting mentions them, this small project is the example to bring up at interview.

The course teaches you to link steps together. An FDE’s job begins with the first email that chain of steps did not anticipate.