FDE PulseFDE jobs open 618New in 7 days 33Companies hiring 94Remote-friendly 24%Median US pay $209kTop hirer Databricks 121
VI

The newspaper of the Forward Deployed Engineer

Analysis

When an agent gets it wrong, courts look for someone to blame. Contracts push that risk onto the customer

From Air Canada's chatbot to the Workday lawsuit, the message is clear: nobody gets to blame the AI. Much of the risk therefore lands on approval points and logs, which are usually built by FDEs.

When an agent gets it wrong, courts look for someone to blame. Contracts push that risk onto the customer
Photo: Michael D Beckwith / Unsplash
ShareLinkedInFacebookX

In brief

  • A tribunal in British Columbia and a federal court in California both refused to let AI serve as a shield: the deployer, and even the vendor, can still be held liable.
  • Under standard contracts, the customer bears the risk when an agent misbehaves, and struggles to prove the vendor was in breach.
  • Bristows advises customers to require vendors to provide approval points for high-risk actions; a new EU directive turns version and update records into evidence.
Diagram of a refund-handling agent: a refund request enters the agent; small amounts are handled automatically and passed to payment; amounts over the threshold (shown in orange) stop and wait for a human approver before payment, and each approval is logged with who approved, when, and which agent version.
Sample set-up from the article: the agent handles small refunds on its own, pauses for a human approver above a threshold the customer chooses, and logs every approval along with the agent version running at the time. It is the kind of approval gate for high-risk actions that Bristows advises customers to demand from vendors. Source: Bristows, as analysed in the article.

Air Canada once tried an argument that many businesses using AI have probably considered: that its chatbot was a separate legal entity. In February 2024 a tribunal in British Columbia rejected that argument and held the airline responsible for the wrong information the chatbot gave a customer.

The tribunal’s reasoning was simple. According to an analysis by McCarthy Tétrault, the tribunal found it made no difference whether information came from a static page or from a chatbot, and Air Canada was responsible for all the information on its website.

For an FDE, the ruling goes straight to the daily work. You are the one who connects the agent to the customer’s real systems, sets thresholds and decides which actions need human approval. Law and contracts are shifting in a direction that turns those technical decisions into evidence when a dispute arises.

Courts will not let AI serve as a shield

The law firm Torkin Manes draws a general principle from Moffatt v. Air Canada: a business remains liable if wrong information reaches consumers through any AI tool. The agent does not take the blame; the deployer does.

The engineering consequence is fairly clear. If a chatbot is treated like a page on the website, its answers need to be controlled like official content: an eval suite that checks them against the source policy, and a way to detect when answers drift. That work usually sits in an FDE’s backlog.

In the US, liability can also reach the vendor. In Mobley v. Workday, a federal court in California denied a motion to dismiss and held that Workday could be liable under an agency theory, because customers had allegedly delegated hiring functions to it.

Duane Morris sums up the lesson for vendors: a vendor that uses AI to screen candidates may be liable for discrimination claims. Put differently, handing a task to software does not make the legal liability attached to that task disappear.

Contracts are pulling the other way

Reading the case law, you might expect vendors to carry more of the burden. The familiar contract templates allocate risk differently.

Mayer Brown describes the logic of traditional SaaS terms: the vendor looks after the platform, and the user is responsible for every action taken on it.

This usually comes with the familiar warranty disclaimer that the service is provided as is, with all faults. According to the firm, that logic is being stretched as agents are sold as a service.

The law firm Bristows is more blunt: under standard contracts, the customer bears the risk when an agent behaves unexpectedly or wrongly. Worse, the customer finds it very hard to prove a breach of contract when an agent causes an error, and must also prove causation.

Put the two pieces together and a gap appears. Courts say the deployer cannot blame the AI, while contracts leave the deployer to carry the agent’s actions alone. When something goes wrong, the key question will be whether human oversight actually existed and worked.

That is why Bristows advises customers not to take on oversight alone where that is inappropriate or technically unfeasible, and to require vendors to provide approval points for high-risk actions. Those approval points do not appear by themselves; someone has to configure them in the customer’s system.

In Europe, version records become evidence

The EU’s new Product Liability Directive extends the notion of “product” to software and digital services. Baker McKenzie confirms that this scope includes AI applications and systems.

Two accompanying mechanisms make logging critical. Courts can order the parties to disclose relevant evidence, and according to HLC, if the defendant fails to provide technical documentation, the product is presumed defective. Manufacturers are also liable if a defect stems from a software update within their control.

Apply those two mechanisms to a specific agent and you need to be able to answer: at the moment the damage occurred, which model, which prompt and which configuration were running, and what was the most recent update? If you cannot answer, you not only cannot defend yourself, you also risk being presumed at fault simply for lacking records.

The table below lines up the sources above against the same questions: who is liable, on what grounds, and what trail the deploying engineer should leave.

Context Who is held liable Grounds What the FDE should leave a trail of
Moffatt v. Air Canada, British Columbia The business deploying the chatbot Responsible for all information on its website; the chatbot is not a separate entity An eval suite checking answers against the source policy
Mobley v. Workday, California Possibly the AI vendor too Agency theory: the customer delegated hiring functions Clear documentation of what the agent decides, and at which step
Traditional SaaS contracts Mainly the user, i.e. the customer The vendor looks after the platform; the service is provided as is A real oversight workflow, with logs of who approved what
Bristows’ recommendation The customer should not carry oversight alone The customer should require the vendor to provide approval points for high-risk actions Approval points that are configured and tested
EU Product Liability Directive Software manufacturers, including AI systems Missing technical documentation means the product is presumed defective; defects from updates within their control The agent’s versions, configuration and update history

Approval points are FDE work

Picture an agent handling refund requests for an e-commerce company. The contract puts oversight on the customer. But in real operation the support team is overloaded, so every refund is approved automatically.

On paper, the risk already sits with the customer. If the agent issues wrong refunds at scale, the system cannot demonstrate any oversight, and following Bristows’ reasoning, the customer will also struggle to shift the blame back to the vendor. The two sides will end up arguing over a control that was never built.

Now try a different configuration. The agent handles small refunds itself, stops and waits for human approval above a threshold chosen by the customer, and every approval is logged together with the agent version that was running. That is exactly the kind of approval point for high-risk actions that Bristows advises customers to demand.

Three traps to avoid

The most common trap is oversight that exists only on paper: the contract and design documents say “human review”, but in real operation nobody opens the approval queue. Check it with data, for example what percentage of above-threshold actions actually had someone click approve last week.

The next trap is a threshold that has never been tested. An escalation threshold written into the config, with no test case proving the agent stops in the right place, is not yet a control. Add a few above-threshold scenarios to the eval suite and rerun them after every update.

The last trap is logs without context. A log that says “approved” without recording who approved, when, and with which agent version and configuration is hard to use when a court orders disclosure of technical documentation.

On every deployment, ask early in discovery: where does the contract assume oversight happens, and which of the agent’s actions could cause harm to end customers, candidates or buyers?

This skill belongs on your CV

Many engineers write a CV line such as “Deployed an AI agent to handle refunds for an e-commerce client”. That line only says what the agent can do, and skips the hardest part of the deployment.

A stronger line states which actions must wait for approval, who the threshold was agreed with, what the logs store, and which eval checks accuracy against the official source.

When reading FDE job descriptions, look for words such as governance, human review, approval workflow or audit trail, and have a concrete example of that work from your own projects ready. In interviews, if asked about a deployment, explain why you put the stopping point where you did.

A software engineer can treat liability clauses as a lawyer’s job. An FDE cannot: an approval point written into a contract only protects anyone when it actually runs in the customer’s system.

Was this article useful?

Use with your AI assistantAsk Claude ↗Ask ChatGPT ↗
7 sources
Read next on the roadmap · Stage 7: LeadershipOne candidate's account: Cognition AI's final FDE round includes a pitch to executivesA pitch to a panel of executives does not grade how well you talk. It tests whether you really understand why the product is worth paying for.