# A wrong agent answer may be a data fault, not a prompt fault

> Tracing data through five stages, from generation to serving, is the data engineering an FDE needs, short of becoming a data engineer.

Original: https://fdetimes.net/en/guides/data-lifecycle-five-stages-fde/

Picture your second week at a customer site, a delivery company. The agent you built says the Da Nang branch had no late orders in September. The operations manager looks at the screen and says straight away that it is wrong: last week they received complaints about late deliveries.

Many engineers reach first for the prompt or a different model. But the number may have been wrong before the model ever saw it. It passed through a chain of systems, and any stage in that chain can drop data.

What you need to do is trace that chain backwards. The skill will not make you a data engineer. It does decide whether you stand in front of the customer guessing, or say "the fault is here, and it can be fixed today".

## What data engineering is, and where the FDE fits

IBM defines data engineering as designing and building systems to collect, store and analyse data at scale. Data engineers, in IBM's description, are software engineers who build and run an organisation's data infrastructure.

FDE Academy treats data engineering as central to FDE work, but points to a key difference: data engineers usually work internally, while FDEs work directly with customers. It also says a data engineer's debugging mindset carries over, but the range of systems they know has to grow.

In practice, a data engineer knows one stack in depth, usually one their own team built. An FDE meets a different stack at every customer, built by someone else, and has to understand it quickly enough not to slow the deployment down.

## Five stages, one question each

According to Johnny Winter's review of *Fundamentals of Data Engineering*, the book splits the data lifecycle into five stages: generation, storage, ingestion, transformation and serving. Those five stages are also your map for tracing a fault.

Storage underpins every other stage, so the table below does not follow the book's order. It follows the path data actually takes when you debug: generation, ingestion, storage, transformation, serving.

Generation happens in the source systems: the ERP, the mobile app, the warehouse software. This is where an FDE first meets customer data, and it is almost always owned by someone else.

Ingestion, as IBM describes it, brings data from many sources into one shared environment. An ETL pipeline automates extracting the data, standardising its format and loading it into a database.

Storage is the hardware and software that write, organise and protect data. IBM lists three types: file, block and object. Transformation relies heavily on SQL. Serving is the point where data reaches its users: a dashboard, an API, or your agent.

| Stage | What the FDE should ask on site |
|---|---|
| Generation | Which system creates this data, who owns it, and has anything changed recently? |
| Ingestion | How does the data arrive, how often, and did the last run succeed? |
| Storage | Where does the data live: a database table, an exported file or object storage? |
| Transformation | Which SQL query or job turns the raw data into the number on screen? |
| Serving | Which table, view or API does the user or agent read from? |

## Tracing the "no late orders" figure

Back to the Da Nang branch. You start at the last stage, serving: the agent reads from a view called `late_orders_by_branch`. Open the view definition and you see it joins `orders` to `branches` on `branch_code`, then filters for orders where `delivered_at > promised_at`.

Two suspects appear at once. First, the join drops orders whose branch code has no match. Second, orders not yet delivered have an empty `delivered_at`, and a comparison against an empty value removes them from the result, even though they are the latest orders of all.

The first check goes after suspect number one. (The column aliases are Vietnamese: `so_don` is the order count, `so_don_khop_chi_nhanh` the count of orders that matched a branch.)

```sql
SELECT o.branch_code,
       COUNT(*)              AS so_don,
       COUNT(b.branch_code)  AS so_don_khop_chi_nhanh
FROM orders o
LEFT JOIN branches b ON o.branch_code = b.branch_code
WHERE o.created_at >= '2026-09-01'
GROUP BY o.branch_code
ORDER BY so_don DESC;
```

Suppose the result shows an unfamiliar code with a few hundred orders and zero in the matched column. That suggests the source system has changed how it writes branch codes while the `branches` table still holds the old ones. The fault is not in the agent, nor in the view's SQL, but at the generation stage.

Before you conclude that, rule out ingestion by counting rows per load date, to make sure last night's export job really ran (assuming the table has a `loaded_at` column; `ngay_nap` is the load date, `so_dong` the row count):

```sql
SELECT CAST(loaded_at AS DATE) AS ngay_nap,
       COUNT(*)                AS so_dong
FROM orders
GROUP BY CAST(loaded_at AS DATE)
ORDER BY ngay_nap DESC;
```

The third query deals with the remaining suspect. Undelivered orders that are already past their deadline must also count as late, so it compares `promised_at` with the current time (`so_don_tre` is the late-order count).

```sql
SELECT o.branch_code,
       COUNT(*) AS so_don_tre
FROM orders o
WHERE o.created_at >= '2026-09-01'
  AND (
        o.delivered_at > o.promised_at
     OR (o.delivered_at IS NULL AND o.promised_at < CURRENT_TIMESTAMP)
  )
GROUP BY o.branch_code
ORDER BY so_don_tre DESC;
```

If the Da Nang figure from this query is clearly higher than the one in the view, you have found a second fault, this time in transformation. Three SQL queries and less than an hour give you a concrete answer for the operations manager, plus the name of the person to contact in the IT team that owns the ERP.

**Key point:** When a number is wrong, trace it back from the screen to the source before you touch the model.

## How deep is deep enough?

Simplilearn calls SQL a foundational skill for data engineers. For an FDE it matters even more, because it is the common language for reading any customer's transformation logic. You need to write JOINs, GROUP BYs and window functions fluently, and be able to read a hundred-line view that someone else wrote.

You do not need depth in building and running infrastructure: tuning clusters or designing an enterprise-wide data warehouse. That belongs to the customer's data engineers. Your job is to talk to them in their own terms and point to exactly what is broken.

The book also lists six "undercurrents" that run through the whole lifecycle: security, data management, DataOps, data architecture, orchestration and software engineering. Use them as a scoping checklist: who may see this data, which job orchestrates the pipeline, and who gets alerted when it fails.

## What to do when you arrive on site

In your first week, map the five stages for the specific data flow your use case depends on, not for the whole company. Write the owner's name next to each stage. On a customer site, knowing whom to ask matters as much as knowing what to ask.

Next, request read access to the transformation and serving stages first, because that is where you can check things fastest. Prepare a set of SQL checks in advance: row counts by load date, counts of records with unmatched keys, counts of empty values in key columns. Run them whenever the data looks odd.

When you find a fault in someone else's stage, do not quietly patch it in your own code. Report it to the owner with the SQL that proves it, and only then agree on a temporary workaround.

## Common mistakes

The most common mistake is treating the data in the final table as the truth and pouring effort into fixing the AI. The second is patching dirty data inside the agent's code, so that every time the source system changes you must fix a place nobody else knows about.

The third gets less attention: understanding only one type of storage. If you only know relational databases, you will struggle when a customer's data sits in thousands of files in object storage. That is exactly the kind of systems range that people moving from data engineering to FDE work need to build up.

If you are aiming for an FDE role, look in job descriptions for phrases such as "integration", "data pipeline" and "SQL" appearing alongside "customer-facing". On a CV, one line describing how you traced a fault from a dashboard all the way back to the source system is more convincing than a list of tools.

## This week's exercise

Take a report you use regularly at work, pick one number on it, and trace it back through all five stages to the system that created it. Any stage you cannot explain is where your learning should go next.

Models will keep getting better. Customer data will still pass through five stages owned by many different people, and the FDE who can follow that whole path will always be the first one called.

**Try this week:**

- Pick a dashboard or report at your company, map its five data stages and write down the owner of each stage.
- Write a SQL query using LEFT JOIN to count records with unmatched keys between two tables you use every day.
- Add a line to your CV describing a time you traced a data fault from the user's screen back to the source system.

## Sources

- [What is data engineering? (IBM Think)](https://www.ibm.com/think/topics/data-engineering)

- [Book review: Fundamentals of Data Engineering (Greyskull Analytics, Johnny Winter)](https://greyskullanalytics.substack.com/p/book-review-fundamentals-of-data)

- [What is data storage? (IBM Think)](https://www.ibm.com/think/topics/data-storage)

- [Data Engineer Skills: Roles and Responsibilities (Simplilearn)](https://www.simplilearn.com/data-engineer-role-article)

- [Data Engineer to Forward Deployed Engineer: Skills, Gaps, and Transition Guide (FDE Academy)](https://fde.academy/blog/data-engineer-to-forward-deployed-engineer)
