How agents earn autonomy, and who keeps watch after the FDE leaves
Users past 750 sessions turn on auto-approve in over 40% of them, twice the rate of newcomers; a traffic-light permission table turns that trust into rules.
In brief
- Anthropic's data shows trust grows with experience: new users enable full auto-approve in about 20% of sessions, users with 750 sessions in over 40%.
- Classify each action by risk, run red actions in propose-only mode at first, and promote a tier only when monitoring data shows the previous one is stable.
- Handover means passing the governance role (policy, auditing) to a named person on the client side. It starts on day one and is tested with an outsider.
Three weeks into a project, the customer-support ticket agent you built has been running cleanly in a sandbox. The client’s head of operations asks: “When can it work on its own, without someone clicking approve?” Then a harder question: “When you leave the project, who on my side looks after it?”
The two questions belong to one skill. The first is raising an agent’s autonomy on evidence rather than gut feeling. The second is moving oversight of the agent to the client’s team so that it survives your departure.
Get either wrong and the agent meets one of two fates: stuck forever in manual-approval mode, or released too early and stripped of its permissions after the first incident.
Trust accumulates, and it can be measured
Data Anthropic has published on how people actually use agents shows a simple pattern. New users, with fewer than 50 sessions, enable full auto-approve in about 20% of sessions. Users past 750 sessions enable it in more than 40%.
On the agent side, Anthropic’s internal figures show the average number of human interventions per session falling from 5.4 to 3.3 as the agent performed better. Anthropic calls the gap between the autonomy a model can handle and the autonomy it is actually granted the “deployment overhang,” and considers that gap significant.
For an FDE, this is a job description. The model is probably good enough already; what is missing is a trail of evidence that gives the client the confidence to grant more permissions. Anthropic also stresses that understanding how agents are used requires post-deployment monitoring. Without logs and metrics, there is nothing to persuade anyone with.
Traffic lights: permissions per action, not per agent
The traffic-light model described by Data Science Dojo works the other way round from granting permissions to an agent wholesale: it classifies each action by risk and assigns it a matching permission tier. High-risk actions run in propose-only mode, with a human approving.
The accompanying principle is that an agent must “earn” autonomy through demonstrated capability, one tier at a time.
MIT Media Lab’s A1 to A5 scale adds two ideas worth remembering. At A3, the agent must know when to stop and escalate to a human. At A4, humans no longer supervise each action and move instead to governing the system: setting policy, confirming the business scope the agent may operate in, and auditing.
At the top of the scale, MIT Media Lab draws one more line that FDEs should know by heart. A5 means the agent executes on its own; it does not mean the agent has authority of its own. It remains bound by the authority of the organization it represents.
An example: an agent that handles refund tickets
Back to the ticket agent from the opening. The first job is to list every tool it can call and color-code them. The table below is illustrative. Promotion thresholds are numbers you agree with the client; there is no common standard.
| Action | Color | Starting tier | Promotion criterion (example) |
|---|---|---|---|
| Read order history, check delivery status | Green | Automatic, logged | Not needed |
| Tag and route tickets | Yellow | Automatic, a human spot-checks samples daily | Low rate of corrected tags for several consecutive weeks |
| Send email replies to customers | Yellow | Drafts only, a human clicks send | Steady decline in the share of drafts whose content is edited |
| Issue refunds | Red | Propose only, a human approves | Promote only for small amounts, with sign-off from the client’s policy owner |
The last row is where the distinction between execution and authority becomes concrete. The agent may be accurate enough to issue refunds itself. But who may refund how much is the client’s policy, and that threshold must be set by someone on their side, not by you.
Policy belongs in config, readable and reviewable, rather than scattered across prompts:
actions:
lookup_order: { tier: green, mode: auto }
tag_ticket: { tier: yellow, mode: auto, sample_review: daily }
send_reply: { tier: yellow, mode: draft_only }
issue_refund: { tier: red, mode: propose_only, owner: "customer support lead" }
escalate_when:
- confidence_below_threshold
- customer_mentions_legal_action
promotion_review: weekly # review interventions, corrections, incidents
The owner field is a detail many people skip. Every red action needs a named person accountable for promotion decisions, and that is the seed of the handover.
Doing it: five steps in order
Start in a sandbox with guardrails, as Anthropic recommends, and design checkpoints where the agent stops to ask for feedback. Then color-code every action as in the table above, defaulting anything uncertain to red.
Next, build monitoring before going to production: count human interventions per session, corrected outputs and escalations. This replaces the sentence “it looks fine to me.”
Each week, sit down with the client to review the numbers, and promote one group of actions by one tier only when the current tier is stable. Finally, once most actions are green or yellow, hand chairing of that review meeting to someone on the client side. At that point they are doing exactly the governance role described at A4.
Handover starts on day one
Tandem’s advice sounds backwards but is right: start the handover before you know you need it. Static documents written in the final week go stale fast. So point readers to the live state of the system, such as the policy config file, the intervention dashboard and the promotion history, instead of copying them into a Word file.
PostHog sets a clear bar for done: a handover deliverable is finished only when an outsider on the client team can pick it up without anyone explaining it first. Next steps must be specific, ordered and assigned to named people. “The ops team will monitor it” is not a next step.
Runbooks should be organized by symptom, not by architecture. The person on call at midnight starts from “the agent is sending emails in the wrong tone” or “the escalation rate has spiked,” never from a service diagram. AI Codex suggests a test: give the runbook to someone who did not help build the system and ask them to read aloud how they would handle a symptom.
Common mistakes
The first mistake is promoting on a schedule (“after a month, let it run on its own”) instead of on data. The second is conflating capability with authority: an agent that is right 99% of the time is not thereby entitled to set refund policy.
The third is having no post-deployment monitoring, so the promotion meeting becomes an argument about impressions. The last is leaving handover to the final week, when the only person who understands why action X is still red is the one about to leave.
If you are applying for FDE roles, put this skill on your CV in numbers rather than adjectives. For example: designed an autonomy roadmap for an agent, cut interventions per session from X to Y, and handed the review process to the client’s operations team.
This week’s exercise
Pick an automated process you run: an agent, a cron job or an internal bot. Write a policy file like the YAML sample above, assign an owner to every red action, then write a runbook entry for exactly one symptom. Hand it to a colleague who has never touched the system and listen, silently, as they read it aloud.
Wherever they hesitate is where your system still depends on you. A good FDE makes those spots disappear before the last day on the project.
Was this article useful?
Thanks for the feedback!
7 sources
- Measuring AI agent autonomy in practice (Anthropic) · 2026-02-18
- Graduated autonomy for AI agents: the traffic-light model · 2026-08-25
- From Autonomous Cars to Autonomous Agents: The Five Levels of AI Agent Autonomy · 2026-09-28
- Building effective agents · 2024-12-19
- Working with customers - Handbook (PostHog)
- The handoff that survives your departure (AI Codex)
- FDE handoffs: keeping an engagement alive when the builder changes | Tandem · 2026-08-28