FDE Times dataFDE jobs open 882▼2 Change in open roles over 7 days, like-for-like: roles from companies we started tracking this week are left outNew in 7 days 52Companies hiring 185Remote-friendly 25%Median US pay $200kTop hirer Databricks 98
中文VI

The newspaper of the Forward Deployed Engineer

Books & courses

Two coded lessons make this 65-minute browser agent course worth it

Skim the AgentQ and MCTS theory; the failure rates behind the headline numbers are the real lesson for forward deployed engineers.

ShareLinkedInFacebookX

In brief

  • Building AI Browser Agents is a 1 hour 5 minute DeepLearning.AI course taught by Div Garg and Naman Garg, co-founders of AGI Inc
  • For FDEs, the value lies in the two coded lessons on a simple web agent and an autonomous web agent; the AgentQ and MCTS section is theory
  • The Agent Q paper reports success rising from 18.6% to 81.7% on a booking task, but 81.7% still means nearly one task in five fails
Slope chart: Llama-3 70B's booking success rate rises from 18.6% (zero-shot) to 81.7% with the Agent Q method; the remaining gap to 100% is shaded orange, marked as about 18 failures per 100 runs.
Even at the best figure the paper reports, about 18 in every 100 booking attempts still need a person to redo them. The result comes from a single task and is self-reported by the authors. Source: Agent Q paper on arXiv (2024).

Success rose from 18.6% to 81.7%. The Agent Q paper reports this result from applying the team’s method to Llama-3 70B in zero-shot mode, on a real-world booking task.

That is a result on one task, alongside a separate experiment on WebShop, a simulated e-commerce environment. It is not an average across many benchmarks.

Still, the number explains why a course only 1 hour 5 minutes long, built on this very research, is worth an evening of an FDE’s time.

The course is Building AI Browser Agents from DeepLearning.AI, produced with AGI Inc. It is taught by Div Garg and Naman Garg, co-founders of AGI Inc. It was announced on 17 April 2025, with the aim of helping learners understand how autonomous web agents work, their current limits and how AgentQ helps agents correct their own mistakes.

Why should FDEs care? At many clients, not every piece of data sits behind an API. Much of it still lives on internal web portals, partner sites and forms that staff click through by hand every day. When a client asks “can an agent do this on its own?”, you need to know both how to build it and where it will break.

One hour, but not every minute is worth the same

The course is rated intermediate, but the only prerequisite is basic Python. Of its 9 sections, two are coded lessons: one builds a simple web agent, the other an autonomous web agent. There is also a deep dive into AgentQ and MCTS.

The two coded lessons are the part to work through carefully, typing the code yourself rather than just watching the video. Picture a situation that comes up early in many engagements: the client needs an agent to visit a site they use every day and pull information out of it. These two lessons are where you practise exactly that kind of job.

The deep dive is a different matter. AgentQ and MCTS are theory about how agents improve themselves, not a deployment skill. The advice for working FDEs: grasp the main idea, and don’t try to master every detail of the algorithm unless your project needs it.

First idea: an agent needs room to try several branches

The Agent Q paper, submitted to arXiv on 13 August 2024, proposes combining guided Monte Carlo Tree Search (MCTS) with a self-critique mechanism, so that an agent learns from both successful and failed runs. The course page describes AgentQ as a framework that combines MCTS, self-critique and DPO so that agents can correct their own mistakes.

Put in a programmer’s terms: instead of walking a straight line from the home page to the “Book” button, the agent weighs several possible sequences of actions and scores for itself which steps look promising.

The idea worth keeping is not in the MCTS details but in the starting point: before the method was applied, the zero-shot model succeeded less than one time in five. That is reason enough to be wary of an agent that runs straight through, never backtracks and never checks itself.

Second idea: read success rates as failed tasks

At 81.7%, roughly 18 of every 100 bookings still fail.

Suppose a client wants an agent to handle 500 booking requests a week. Even at exactly that level, more than 90 requests would need a person to redo them. So the first thing to settle with the client is who handles the failures, and how they will be detected.

The wider picture is harsher still. The REAL benchmark, built by AGI Inc with its co-authors, uses deterministic replicas of 11 websites with 112 tasks. On it, frontier models reach at most 41%; AGI Inc’s blog records a score of 41.1% for Claude-3.7-Sonnet-Thinking. Out of every 100 tasks, nearly 60 fail.

Third idea: measure before you promise

AGI Inc presents its research site as a handbook for people who build, measure and review agent systems. For FDEs, the order of those three tasks is worth remembering: building the agent is only the first step, not the whole job.

Imagine writing 20 tasks for an agent you have just built on a public website, then putting each run into exactly one bucket. A hypothetical results table might look like this:

Outcome Tasks Next fix
Success 13 Keep as a regression test suite
Element not found 3 Check selectors, wait for the page to finish loading
Misread the page 2 Review what content the agent actually sees
Incomplete result 2 Fix the prompt, add grading criteria

Reading the table: 13/20 is 65%, so for every 20 requests, 7 need a person to review them. The last column tells you what to fix first, here the element-not-found errors, because they are the most common. Rerun the same 20 tasks after each fix, and bring that table to the client meeting instead of a demo that worked once.

If you want a shared yardstick rather than only your own task set, the open-source AGI SDK is the logical next step. It installs with pip and runs a harness on the REAL tasks, so you can compare your agent against the same “mini-Internet” on which the large models are being scored.

Who should take it, and in what order

If you are a software engineer looking to move into FDE work and have never built an agent, this is a gentle place to start: the course is short and needs only basic Python. If you already know agents, you can go straight to the two coded lessons and then on to the AGI SDK.

A sensible order for newcomers: read the abstract of the Agent Q paper on arXiv for the big picture, work through the two coded lessons carefully, skim the AgentQ and MCTS deep dive, then measure your agent on your own task set.

On a CV, a short-course certificate says little. A small repo is different: a web agent running on a public website, with a table of success rates and an error breakdown on a task set you wrote yourself, plus scores from a few REAL tasks.

When reading FDE job ads, watch for terms such as “agent”, “automation” and “evaluation”: that repo answers all three at once.

The course teaches you to build an agent in an evening. The real FDE work starts when you count how many times it fails out of 100 tasks.

Was this article useful?

Use with your AI assistantAsk Claude ↗Ask ChatGPT ↗
7 sources
Read next on the roadmap · Stage 3: Applied AIThe skill format travels between platforms. Deployment does notThe two-hour DeepLearning.AI and Anthropic course is worth taking, as long as you also read each platform's own documentation.