# Building and Evaluating Data Agents: grade the process, not just the answer > An answer grounded in the data can still be wrong. A short course from DeepLearning.AI and Snowflake teaches you to find that error inside the agent's plan. Original: https://fdetimes.net/en/books-courses/building-evaluating-data-agents-course-review/ The longest lesson in DeepLearning.AI's Building and Evaluating Data Agents course does not teach you how to build an agent. It is called "Measure Agent's GPA", runs for 29 minutes, and teaches you how to check whether the agent did what it set out to do. That allocation of time says a lot about the course's priorities. For an FDE, building a data Q&A agent that works in a demo is not hard. The hard part is explaining to the customer why it gave a wrong answer on Tuesday, and proving that you have fixed it. ## Who teaches it, and why that matters This is an Intermediate-level short course lasting 1h59m, made by DeepLearning.AI with Snowflake. The course page lists its last update as 31 August 2026. It contains 8 videos, 5 code lessons and 1 graded quiz. The two instructors are Anupam Datta, AI Research Lead, and Josh Reini, Developer Advocate. Both work at Snowflake. In other words, a data platform vendor is teaching you how to build agents on data, so you need to separate the method from the product. The entry requirements are low: basic Python and object-oriented programming. That makes it a sensible starting point for software engineers who want to move into FDE work but have never built evals. The course page states that the course is free for a limited time during the beta of the new learning platform, while a certificate of completion requires a PRO plan. ## Idea one: keep the planner separate from the data fetchers The agent in the course consists of several specialised sub-agents, each responsible for one data source, plus one sub-agent that handles summarising or charting. For each question, a planner decides which sub-agents to call and in what order. Imagine a retail customer asking: "Why did revenue in the northern region fall in Q3?" The planner calls the SQL sub-agent to pull figures by region, calls the document sub-agent to read promotion notes, then hands everything to the summarising sub-agent. Every step is a place where things can break, and also a place you can measure. That is why this separation is useful on site with a customer. When the agent gets something wrong, you can pin down whether the planner picked the wrong source or a sub-agent wrote the wrong query, instead of staring at one long block of prompt. ## Idea two: a grounded answer can still be wrong The output evaluation uses LLM-as-a-judge with two questions: is the answer relevant to the question, and is it grounded in the data that was retrieved? But go back to the example above. Suppose the SQL sub-agent mistakenly filters for Q2, and the agent faithfully summarises what it retrieved. That answer is both relevant and grounded, yet still wrong. The GPA framework (goal, plan, action) exists to catch exactly this kind of error. It grades the whole process by checking whether the goal, the plan and the actions line up with one another. Snowflake's engineering blog published the Agent GPA framework on 4 November 2025 and groups its metrics in pairs. The goal-action pair covers answer correctness, answer relevance and groundedness. The goal-plan pair covers plan quality and tool selection. The plan-action pair covers plan adherence and tool calling. If you have to deliver a data Q&A agent next week, start with the three goal-action metrics. **Key point:** An answer that is faithful to the wrong data is still a wrong answer. ## Idea three: let the agent grade itself while it runs The final part of the course teaches inline evaluation. At each retrieval step, the agent scores whether the data it has just fetched is relevant, and uses that score to decide whether to change its plan. In the retail example, a low score could lead the agent to re-run the query rather than hand the customer Q2 figures. Every self-check adds another judge call, so this technique trades latency for reliability. For customers who make decisions based on the numbers, the trade is usually worth it, but you should measure it before switching it on in production. A simple way to measure it: run the same set of real questions twice, once with inline evaluation off and once with it on. For each question, record total run time, the number of judge calls, and whether the agent changed its plan. Then compare the median run time and the slowest run for both configurations, set them beside the number of questions that a plan change rescued, and take that table to the customer to decide. ## What to skip, and in what order to study The most impressive GPA figures all come from the people who built the framework. Snowflake's blog says annotators applying GPA to TRAIL/GAIA caught all 570 internal agent errors. The arXiv paper 2510.08847 by Allison Sihan Jia and colleagues, including Anupam Datta, reports that the judges found 95% of human-labelled errors and localised 86% of them. These figures show the method has a basis. They do not prove it will perform similarly on your customer's data. Likewise, treat the parts of the course that are tightly bound to the Snowflake platform as illustrations, not requirements. A sensible order is to take the course first, then read the blog for the metric table, and only then read the paper if you need arguments to defend your evaluation approach to the customer's engineering team. As for your job application: when an FDE posting mentions "evals" or "agents", a small repo with a data agent running on a public dataset, plus an error table classified by goal, plan and action, will be far more convincing than a line about a certificate. The demo shows the customer that the agent can answer. The eval suite shows them that you know why it answers wrongly, and that is what gets them to renew. **Try this week:** - Take the 'Measure Agent's GPA' lesson (29 minutes), then write down the goal, plan and actions of one run of an agent you are working on - Write 20 real customer questions about their own data and grade your agent on three metrics: answer relevance, groundedness and answer correctness - Read Snowflake's Agent GPA blog post and the abstract of arXiv 2510.08847, and compare the method with how you grade today ## Sources - [Building and Evaluating Data Agents - DeepLearning.AI](https://www.deeplearning.ai/short-courses/building-and-evaluating-data-agents/) - [What's Your Agent's GPA? A Framework for Evaluating AI Agent Reliability](https://www.snowflake.com/en/blog/engineering/ai-agent-evaluation-gpa-framework/) - [What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment](https://arxiv.org/abs/2510.08847)