# Auditors don't read your code. They trace one AI decision

> If a single log line cannot lead an auditor to the test suite, the human reviewer and the drift alert, the go-live date will slip.

Original: https://fdetimes.net/en/guides/audit-evidence-trace-ai-decision/

Picture this: two weeks before go-live, the AI system you built to help a bank assess loan applications is running smoothly on staging. Then an email arrives from internal audit. They want to know how the system was tested, who oversees it and where the logs are kept.

Many FDEs treat that email as paperwork, hand it to the client's compliance team and go back to writing code. That is a mistake. For AI systems in regulated sectors, most of the audit evidence required can only be produced by the people who built the system, and producing it once the questions arrive is too late.

This guide shows how to read an auditor's questions as a technical specification. Design for them from the start and the audit meeting becomes a matter of opening the right documents. Leave it to the last minute and every gap in the evidence can push back the go-live date.

## Auditors follow a trail, they don't read code

To prepare well, you need to know which lens the questioner is using. The Institute of Internal Auditors (IIA) has an AI Auditing Framework, which it describes as practical guidance for internal auditors on the governance, management and auditing of AI.

The framework rests on the Three Lines Model, so the first question is usually who owns the system at each line: the business, then risk management and compliance, then audit.

Their working method is just as concrete. An ISO 42001 audit checklist from ISMS.online advises sampling two AI systems in scope, then following each system's impact assessment from drafting through approval, along with every corrective action closed.

Auditors don't read everything. They pick a thread and pull.

Most of the evidence they expect is generated by the system itself. The same checklist lists access logs, monitoring outputs, model performance dashboards, drift alerts and incident tickets. All of these fall within your work.

## What does the law say about logs and documentation?

If the system counts as high-risk under the EU AI Act, some of the questions are already set by law. Article 12 requires high-risk systems to technically allow events to be recorded automatically throughout their lifetime.

Those logs must serve three purposes: identifying risk situations, post-market monitoring and operational monitoring.

Article 26 places obligations on the deployer, which is your client. They must keep the logs the system generates automatically, to the extent those logs are under their control, for at least six months unless other law provides otherwise.

They must also assign human oversight to people who have the necessary competence, training and authority.

The phrase "under their control" is where an FDE needs to pay attention. If the logs live only in the vendor's cloud account, the client cannot retain them.

Article 11 further requires technical documentation to be drawn up before the system is put into service and kept up to date. According to the law's summary page, the application dates are 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems.

Outside Europe, the NIST AI RMF Playbook is a common reference. NIST states clearly that both the framework and the Playbook are voluntary, so treat it as a menu of evidence to choose from, not a mandatory checklist.

## The map: every question has its evidence

The most useful tool you can bring to the meeting is a table that pairs each audit question with a specific piece of evidence, a storage location and an owner.

| Audit question | Evidence | Where it lives | Owner |
|---|---|---|---|
| How was the system tested? | Test sets, metrics and tools used for test, evaluation, verification and validation (TEVV, per NIST MEASURE 2.1) | Project documentation repository, tied to each release | FDE, engineering team |
| How is quality monitored after go-live? | Performance dashboards, thresholds and drift alerts | Monitoring system on the client's infrastructure | FDE hands over, client operations take over |
| Where should the system not be used? | A "Known limitations" section stating the conditions the system has not been tested under | Technical documentation, updated per version | FDE writes, business signs off |
| Who oversees it, and do they have enough authority? | List of reviewers, training records, override rights | Client HR and training records | Client's first line |
| Can a decision be traced? | Event logs linked to incident tickets | Client's logging infrastructure, kept at least six months | FDE designs, client retains |

Rows one and three come from the NIST Playbook: the MEASURE function calls for documenting test sets, metrics and tools, as well as limitations when the system is used outside the conditions it was developed under. Row two combines NIST's question about how metrics such as accuracy are monitored after deployment with the dashboards and drift alerts listed in the ISO 42001 checklist.

## Tracing one decision back

Back to the bank. The auditor picks an application where the system recommended "refer to an underwriter for review" and asks why. You can only answer if the log record looks something like this:

```json
{
  "event_id": "evt_000123",
  "system": "credit-assist",
  "model_version": "credit-assist-v1.4",
  "prompt_template": "tpl-appraisal-v3",
  "eval_suite": "eval-v7",
  "input_ref": "application/8812",
  "output": { "recommendation": "refer", "score": 0.62 },
  "reviewer_id": "u_lan",
  "reviewer_action": "override_approve",
  "reason_code": "income_verified"
}
```

From `model_version` and `eval_suite`, you can open the eval-v7 report to show which tests this version passed and what metrics it achieved.

From `reviewer_id`, the client can show who the reviewer was and what training they had. `input_ref` is only a pointer to the application, not a copy of personal data in the log, so the log can be kept for a long time without becoming a second store of sensitive data.

Now imagine the record lacks `eval_suite`. You know which model ran but cannot prove what testing it passed, and the thread breaks at the second step. The auditor records a finding, and go-live waits until that finding is closed.

**Key point:** Audit evidence must be designed in the first week, not gathered in the last.

## Five things to do before anyone asks

First, during discovery week, ask the client who owns the system at each line. The question often reveals that nobody has been assigned oversight. Found in week one, there is still time to fix it; found in the final week, it becomes the reason the schedule slips.

Next, design the log schema before writing the first endpoint, so that every decision carries the model version, prompt version, eval suite and reviewer. Ship the logs to the client's infrastructure so they can keep them for at least six months. Treat the eval suite as a versioned artifact tied to each release, not a notebook run once and abandoned.

Then write the known limitations in business language, for example "not tested on corporate applications, only on individual applications". Finally, before the real meeting, play the auditor yourself: pick two decisions at random and trace each one back step by step using the table above.

## Where things usually break

One common failure is an accuracy figure on a slide with no test set or tooling behind it, so nobody can reproduce the number.

Another is a dashboard that exists but has no alert thresholds, and no ticket showing that an alert was ever handled.

A subtler failure is treating the NIST Playbook as a checklist and trying to fill in every item. The result is a thick stack of documents with no thread connecting them. Auditors need fewer documents, but ones they can trace from end to end.

## Turning the skill into a career advantage

If an FDE job ad mentions finance, healthcare, insurance or "regulated industries", you will very likely be working with internal audit. Practise the exercise of tracing a decision back before you go to interview.

On your CV, don't just write "deployed AI systems". Say that you designed the traceable logging and evidence pack that took a system through internal audit before go-live.

In interviews, describe which fields a log record needs and why each one exists, rather than talking about "compliance" in general terms.

With these clients, a system that works correctly is not enough to go live. You also have to prove it to people who will never read the code, and that is part of the handover too.

**Try this week:**

- Pick a real decision made by the system you are working on, and time how long it takes to find its model version, eval suite and reviewer.
- Fill in the 'audit question → evidence → where it lives → owner' table for your current project and mark every empty cell in red.
- Write half a page of 'Known limitations' for your system and send it to the business owner on the client side to read.

## Sources

- [Article 12: Record-keeping | AI Act Service Desk](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-12)

- [Article 26: Obligations of deployers of high-risk AI systems | AI Act Service Desk](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-26)

- [Article 11: Technical Documentation](https://artificialintelligenceact.eu/article/11/)

- [The IIA's Artificial Intelligence Auditing Framework](https://www.theiia.org/en/content/tools/professional/2023/the-iias-updated-ai-auditing-framework/)

- [Measure - AIRC (NIST AI Resource Center)](https://airc.nist.gov/airmf-resources/playbook/measure/)

- [NIST AI RMF Playbook](https://www.nist.gov/itl/ai-risk-management-framework/ai-rmf-playbook)

- [AI Audit Checklist: A Practical Guide to Auditing AI Systems (ISMS.online)](https://www.isms.online/iso-42001/ai-audit-checklist/)