# DeepLearning.AI's Post-training of LLMs: 76 minutes on when to use SFT, DPO or RL

> Banghua Zhu's short course will not teach you to run a GPU cluster. It does teach the question FDEs hear most often on site: should we fine-tune, and if so, how?

Original: https://fdetimes.net/en/books-courses/deeplearning-ai-post-training-llms-review/

On a customer site, an FDE is rarely asked "how do we run GRPO?" The far more common question is "should we fine-tune, and if so, how?" Get the answer wrong and a team can lose weeks training a model for a problem that a better prompt would have solved.

DeepLearning.AI's "Post-training of LLMs" is built around exactly that question. It covers three techniques: Supervised Fine-Tuning (SFT), DPO and online RL. According to the course page, the aim is to understand when and why to use each method, rather than simply learning to run one technique in isolation.

That is why the course is worth taking even if you have never had to train a model yourself. It is rated Intermediate, and the official page sums up its goal as learning to post-train and customise an LLM.

## Who teaches it, and who is it for?

The instructor is Banghua Zhu. According to his biography on the course page, he is an Assistant Professor at the University of Washington, a Principal Research Scientist at Nvidia and a co-founder of Nexusflow. DeepLearning.AI announced the course on its community forum on 9 July 2025, and the course page lists its most recent update as 17 July 2026.

It is compact. The course page gives a total of 1h26m, including a 10-minute graded quiz. Take away the quiz and the lessons come to about 76 minutes, which matches the 1h16m figure in an older listing of the same course.

The description says the course targets AI practitioners "ready to go beyond pre-training". In other words, you should know LLM basics before you start. For an FDE, the sensible order is to get comfortable with prompting, retrieval and evals first, then take this course. Fine-tuning is only worth discussing once you have measured that the cheaper options are not enough.

## Three methods, three kinds of signal

Each section of the course has a hands-on lab. In the SFT section, you load a pretrained model and build a pipeline that turns a base model into an instruct model. In the DPO section, the example is changing the "identity" of a chat assistant. In the RL section, you learn how PPO and GRPO work and how to design a reward function, then train with GRPO and a verifiable reward to make the model better at maths.

Set the three labs side by side and the first lesson worth keeping emerges: each method needs a different kind of signal. SFT learns from worked examples of an input and its correct output. DPO learns from comparisons between a good answer and a poor one. RL learns from a scoring function.

**Key point:** Before choosing how to fine-tune, ask what kind of signal the customer already has.

Picture a logistics customer that wants a model to read emails and extract waybill numbers. If they already have a few thousand emails labelled by staff, that is SFT-style data.

If all they have is a log of staff correcting the bot's answers, those are good/bad pairs, much closer to DPO. The first job on a customer site is to take stock of what data they have, not to start talking about models.

## DPO is for adjusting how a model behaves

The chat-assistant identity lab is small, but it shows that DPO changes a model's behaviour. That is the second lesson worth keeping.

Requirements such as "the bot must introduce itself with the company name", "the bot must decline out-of-scope questions" or "the tone must be concise" are all about behaviour, and can be expressed as pairs of preferred and rejected answers.

If the customer wants the model to "know" more internal documents, think carefully before reaching for DPO, because retrieval is usually the better first thing to try. Separating these two kinds of requirement during discovery will save both sides many rounds of pointless training.

## Verifiable rewards are the part closest to FDE work

The third lesson, and the main reason to take the course, is the section on GRPO with verifiable rewards. A maths problem has a right or wrong answer, so the reward can be computed automatically without a human grader. Many customer tasks share that property.

Go back to the logistics customer. The reward can be as simple as this: 1 point if the output is valid JSON and the waybill number matches the system, 0 otherwise.

If the task is reading invoices, you can add the condition that the total must equal the sum of the line items. Writing a reward like this forces you to define "correct" in code. That is also the skill you need to build evals, even if in the end you train nothing at all.

The downside is that a loosely written reward lets the model find ways to score points without actually doing the job. So show the customer the reward function as a specification before running even a single training step.

## What to skip

Do not wait for the course to teach you infrastructure. The labs run on a small pretrained model downloaded from Hugging Face. NVIDIA's NeMo RL post-training library, by contrast, is designed for models from very small to more than 100B parameters, running on anything from 1 GPU to hundreds.

The gap between the two is a whole skill set of its own, and 76 minutes cannot close it.

For most FDEs, that gap is acceptable. Your job is usually to pick the right method and prove that it works. When a real project needs training at scale, the NeMo RL documentation is the next step, not a second pass through this course.

If the job descriptions you are targeting mention fine-tuning in the context of customer work, do not write a vague "studied post-training" on your CV. Describe a specific decision: which task, which signal, why you chose SFT or dropped fine-tuning, and how much the eval score moved.

What to take away from these 76 minutes is not how to run GRPO. It is the habit of asking "what does correct mean, and how do we verify it?" before touching a single GPU.

**Try this week:**

- Start with the lesson on PPO and GRPO, then write out on paper a reward function for a task with a verifiable answer that one of your customers is facing.
- Take the most recent "we want to fine-tune" request you received and note whether the customer has worked examples, good/bad comparison pairs or an automatic way to score right and wrong.
- Add a line to your CV that describes, with numbers, a time you chose or ruled out fine-tuning, with the reason and the result on the eval set.

## Sources

- [Post-training of LLMs – DeepLearning.AI](https://www.deeplearning.ai/courses/post-training-of-llms)

- [✨ New course! Enroll in Post-training of LLMs](https://community.deeplearning.ai/t/new-course-enroll-in-post-training-of-llms/848544)

- [Post-training of LLMs – DeepLearning.AI Short Courses](https://www.deeplearning.ai/short-courses/post-training-of-llms/)

- [NVIDIA NeMo RL documentation](https://docs.nvidia.com/nemo/rl)