Engineering interviews now grade how you doubt AI-written code

Tristan Hume writes on Anthropic’s engineering blog that each new Claude model forces the company to redesign its engineering hiring test.

The reason is that the model performs as well as the strongest candidates. Even so, Anthropic does not ban AI. The take-home on performance engineering states explicitly that candidates may use AI tools, just as they would on the job.

This is not confined to one lab. Meta and Google also run coding rounds that allow AI. According to one interview guide, so do some Forward Deployed Engineer rounds at OpenAI. In these rounds, the weight of the grading is shifting from the ability to write code to the ability to judge it.

The evidence from Anthropic and Meta concerns engineering hiring in general, not FDEs specifically. But for people who work as FDEs, or want to, it matters even more, because their job is to put code into customers’ real systems and run it on real data.

There, a piece of agent-written code that looks right but is wrong has real consequences, not just a lost interview point.

When the machine can write the code, what is left to grade?

An analysis on techinterview.org explains why companies are changing how they score. Interviewers have moved the bar to the one thing the model cannot do for you: decide whether its output is any good.

A Meta speaker, in a piece published on LeadDev, called the company’s new interview an “AI-native” coding round that measures adaptability rather than memory. In this speaker’s account, the skill being tested has moved from knowing frameworks to catching the AI’s hallucinations.

Knowing a lot is therefore no longer the biggest advantage. The advantage lies in noticing when what you are looking at is wrong, even when it looks highly convincing.

The behaviour being graded is simpler than you think

According to techinterview.org, Meta calls this behaviour “critical verification”. The way to observe it is simple: do you read AI-generated code before you run it?

Discussing Meta’s and Google’s AI-enabled coding rounds, the same site argues that verification makes up most of the score. The moment you paste an AI suggestion into the editor, the interviewer watches what you do next.

The article describes strong candidates as those who treat the AI like a quick, confident intern who is sometimes wrong. Catching its mistakes during the interview counts as a good signal. The senior-level signal these rounds look for is a candidate who grows more suspicious precisely when the answer looks fine.

Each company describes it differently, but put the rounds side by side and they grade the same skill from different angles:

Interview round	AI allowed?	What is graded	What the interviewer watches
Anthropic’s performance engineering take-home	Yes, stated in the brief, as on the job	Beyond the submitted code, how you decide what to invest in tooling	Which tools you put effort into, and why
Meta’s “AI-native” coding round (per LeadDev) and Meta’s and Google’s AI-enabled rounds (per techinterview.org)	Yes	Adaptability, catching hallucinations; verification is most of the score	Whether you read code before running it, what you do after pasting a suggestion, whether you doubt an answer that looks neat
OpenAI’s FDE rounds (per Aced’s guide)	Some rounds, including with Codex	Reasoning about and verifying code written with AI	Screen sharing, reasoning out loud, explaining time and memory costs

OpenAI’s FDE rounds want to hear you state the costs

A guide to OpenAI’s FDE interviews on Aced (formerly Exponent) says some coding rounds allow AI. What is assessed is whether you can reason about and verify the code you write with it.

In screening rounds with Codex, candidates share their screen, reason out loud and explain the time and memory costs of everything the tool writes.

To see what this means in practice, imagine an exercise. A logistics customer hands you an orders file with 2 million rows and asks you to merge duplicate customer records by email. The agent immediately writes a function with two nested for loops that compares every pair. You run it on 10 sample rows and the result is entirely correct.

A candidate who only knows how to press run stops here. A candidate with judgement reads the code and names two problems.

The first is that comparing every pair across 2 million rows takes roughly two trillion comparisons. With O(n²) complexity, the function will not finish in an acceptable time.

The second is that the email comparison is case-sensitive, so “[email protected]” and “[email protected]” are treated as two different customers.

The fix is not hard. Normalise each email by stripping whitespace and lowercasing it, then group records with a hash map. Time drops to O(n), at the price of O(n) memory, because every key has to be held in memory.

That is exactly the “explain the costs” part the guide describes. This hypothetical example shows something else too: 10 sample rows are usually clean, customer data rarely is, so green tests on a sample prove nothing.

Anthropic also grades how you choose your tools

According to Hume’s post, Anthropic treats how candidates decide to invest in tooling as part of the hiring signal. Do you stop to write a benchmarking script or a small test suite before letting the agent optimise, or do you let it make blind edits and hope the result improves?

FDEs face exactly this question in their first week on a customer site. Time is limited, and you must choose between doing the work by hand now and building a tool that makes verification faster for the month ahead. Seen this way, Anthropic’s take-home is far closer to the real job than a LeetCode problem.

The fact that the test has to change with every new model is also a hint: skills the machine can perform are likely to drop off the grading list. Investing in the ability to tell right from wrong is a safer bet than memorising another framework.

How to practise before your next interview

The most effective practice is to rehearse exactly what the interviewer watches.

Each time an agent produces code, go through five steps before trusting it: read every line before running it, say out loud what the code is meant to do, actively look for inputs that break it, state the time and memory costs, then decide to keep, fix or discard it based on evidence from tests or measurements.

Do not press run until you have completed the first four steps. The fifth is where you show the interviewer what your decision rests on.

When reading FDE job descriptions or interview prep material, look for phrases such as AI-enabled coding round, Copilot or Codex allowed, or share your screen and explain your reasoning. They signal that the company will grade judgement, and then you should spend more time practising coding while talking out loud than grinding problems.

On a CV, the line “used AI to boost productivity” tells a recruiter almost nothing. A more valuable line describes one specific occasion: what code the agent produced, what bug you found before it reached production, and which test or measurement you used to verify it.

The new interview rounds are not looking for people who trust AI, nor for people who reject it. They want people who know to doubt the agent’s code exactly when it looks most perfect, and on a customer site, that is also when you are most valuable.