Andrew Ng's Machine Learning in Production: 11 hours on the work notebooks don't teach

A model can be right on handover day and then grow steadily worse from the following week, even if nobody changes a line of code. Machine Learning in Production, the DeepLearning.AI course taught by Andrew Ng, goes straight to that paradox: unlike ordinary modelling, a production system has to cope with data that never stops changing.

For an FDE, this can happen while you are on site with the client, and the first question will not be about model architecture. It will be “why, and since when?”

The course is worth your 11 hours because it teaches end-to-end system design, building a baseline and handling concept drift. The specific measurement for each client is still yours to build.

Who should take it, and in what order?

Ng is the founder of DeepLearning.AI and a co-founder of Coursera. The course grew out of the MLOps Specialization launched in 2021, which he taught with Robert Crowe and Laurence Moroney of Google, on the grounds that building and deploying production systems still demanded a great deal of manual work.

According to the official page, the course is intermediate level and runs to about 11 hours, typically over 3 weeks at 5 hours a week.

You need intermediate Python, and the course is written for people new to ML or for software engineers. Coursera lists it as a standalone course, with MLOps and model deployment among its skills and a rating of 4.8.

Week 1 covers the project lifecycle and deployment, week 2 the challenges of modelling, and week 3 data definition and baselines. If you have the time, take them in order.

If you are meeting a client next week, watch week 1, skip to week 3, and only then return to week 2.

Week 2 centres on the difficulties of building the model itself, so it is less urgent: at the first meeting you have no model yet, whereas you need the baseline straight away to know what mark the eventual model has to beat.

This is a postponement, not a cut, so do week 2 before you start training.

It also helps to know what the course does not teach. None of the three weeks is devoted to LLMs or agents, so anyone building RAG for a client will come away with a way of thinking about operations rather than a recipe to use immediately.

The baseline is your first answer to the client

The course teaches end-to-end design, from project scope and data needs to modelling strategy and deployment requirements, and it teaches how to establish a model baseline. This is the idea most worth keeping, because it changes the whole conversation with the client.

Imagine a company that wants to automate the routing of support tickets. You take 200 labelled tickets: the current keyword rules get 156 right (78%), staff get 182 right (91%), and your model gets 176 right (88%). On its own, 88% means nothing. Set beside the other two marks, it beats the rules by 20 tickets but still trails people by 6.

Now the meeting has a question worth discussing: for every 200 tickets, 6 more go wrong, so how much staff time does that buy, and will the client accept the trade? So when you arrive at a client, ask for a sample of real data and measure how they work today before you open a notebook.

Drift is routine, not an incident

In his 2021 introduction, Ng placed detecting and fixing data errors, and recognising data drift and concept drift, among the tasks of MLOps. The two kinds of drift are easily lumped together, but they are handled differently.

Go back to the ticket example. The client launches a new product, and tickets start to contain words the model has never seen: the input has changed, and that is data drift. The fix is to label a batch of tickets about the new product, add them to the training set and retrain; the old labels are still correct, so they can stay.

Then the client reorganises its departments, and billing complaints move to a different team. The tickets look exactly as they did, but the right answer has changed, and that is concept drift. More data of the old kind will not help, because the old labels themselves are now wrong. You have to correct the labels to match the new organisation chart, or drop the outdated examples, before retraining.

To catch both early, set up a weekly measurement. Each week, take a random sample of 200 new tickets, ask the client’s staff to label them without seeing the model’s answers, and count how many the model got right.

Record that number beside the two marks you already have: 176 tickets at handover and 156 for the keyword rules. You can set thresholds in advance: below 168 (84%), investigate; below 156 (78%), discuss with the client a temporary return to the old rules.

Suppose that by week six the sample shows only 148 tickets right, or 74%, lower even than the keyword rules the model was bought to replace. Read the 52 wrong tickets: many new words point towards data drift, while familiar tickets whose answer has changed point towards concept drift.

Without this measurement, you will find out from the client’s complaint call. So agree from the start who labels the sample and who receives the alert.

If you cannot trace it, you cannot defend it

The course’s stated goal is an ML project that is reproducible, traceable and verifiable for production. That sounds dry until a client asks why their ticket was sent to the wrong team on Tuesday and you do not know which model was running then, or on what data.

Starting is cheap: at every deployment, record the data version, the code commit and the evaluation results. Those three lines turn an argument into an investigation with leads to follow.

The course also sets a constraint that FDEs should read closely: the system must run continuously, at minimum cost and with maximum performance. In practice these three goals pull against one another, so the deployment requirements are only truly finished once the client has said clearly what they prioritise and what they are willing to give up.

Turn 11 hours into evidence for job applications

Do not stop at the certificate. Apply the framework to a problem with measurements, even one you choose yourself using public data, so that you have something concrete to talk about.

Write one CV line to this pattern: the baseline you measured, the result against that baseline, and how you detected drift. When reading FDE job descriptions, watch for terms such as model deployment, MLOps and drift, and prepare a story for each one.

If you can tell an interviewer about the moment a model fell below its baseline and what you did next, they will understand that you can do more than train models.

The course takes only three weeks, but its real test comes later: in week six at the client, when the data has changed and you still have to answer with numbers.