Klarna is hiring humans again: AI use cases need operational evidence, not vendor blogs

In February 2024, Klarna announced that its AI assistant, built on OpenAI, had handled 2.3 million conversations in its first month, or two-thirds of the company’s customer service chats. Fifteen months later, the same company’s CEO said that investing in the quality of human support was the way forward.

The two statements do not contradict each other. They show a pattern that anyone deploying AI in a business runs into sooner or later: a demo is measured by the share of work automated, while a system that runs for the long term is measured by who fixes things when the machine gets them wrong.

If you are trying to move into forward deployed engineering, this is the most valuable lesson available. Clients will arrive with a list of use cases taken from vendor blogs. Your job is to say which ones have operational evidence behind them, and what each one needs so it does not fall apart when real users arrive.

Vendor lists are long, but not every item comes with numbers

Read enough use-case roundups and it is easy to conclude that AI is already present in every industry. Glean groups RAG by industry: customer service uses RAG to pull product information and customer history and personalise replies, while in legal work the system searches case law and statutes to help lawyers research and draft.

Codica lists agents in finance that can detect anomalies and signs of fraud. All of this is plausible, but it describes capability, not operational evidence.

A vendor blog post tells you what kind of problem can be tackled. It does not tell you the rate of wrong answers, whether users trust the system, or who is accountable when something goes wrong.

So when you read about a use case, ask five questions: where does the data come from, does the answer point to its source, who reviews the result before it reaches the end user, when must the system stop and hand over to a person, and if nobody reviews it, what is measured instead.

Answer those five questions for each of the examples below and they fall into a few clear groups.

RAG works best when someone reads the output

Two use cases described by theblue.ai deserve a close look, because they share a structure that is easy to control. The first is internal Q&A: the system answers within seconds and names the documents it used to answer.

The second is repetitive compliance questionnaires or tender documents: RAG drafts answers from policies and technical documentation, then an expert reviews and approves them.

What the two have in common is that the machine does not have the final say. An answer with citations can be checked by the reader. A draft reviewed by an expert means a wrong sentence is caught before it goes out, and case-law research follows the same pattern: the lawyer still does the research and the drafting.

Consider a hypothetical: a B2B software company receives about 40 security questionnaires a quarter, each with several hundred questions, most of them repeated across questionnaires. If RAG can draft most of the answers and the expert only has to fix the hard ones, the time saved is real, and the risk stays in the hands of someone who knows the work.

That is exactly the shape of a use case that gets its budget approved.

Klarna shows where customer service agents are strong and where they break

According to figures the company published itself, customer satisfaction with the AI assistant was on a par with human agents and repeat enquiries fell by 25%, although these are the company’s own numbers rather than those of an independent auditor.

In May 2025 the Klarna CEO changed tone: according to Entrepreneur, the company was rehiring customer service staff and promising that customers who wanted to speak to a real person would always be able to. The lesson is not that AI failed. It can be read as a shift from a design that treats automation as the goal to one that treats the handoff to a human as a mandatory feature.

Picture a customer asking about a refund, but the refund details sit in the payments system and the complaint history sits in the ticketing system, so the bot sees only part of the picture and answers confidently, yet wrongly.

That is why, for an FDE, the first question on a customer service project is which systems the agent can read, not which model to choose. The second is what signal the agent relies on to know it lacks enough information and must hand over to a person.

Product recommendations are the AI with the clearest metrics

On recommender systems, Algolia cites figures suggesting recommendations account for as much as 31% of e-commerce revenue, and repeats the claim that 80% of what is watched on Netflix comes from its recommendation system. Both are second-hand figures in a post by a vendor that sells search and recommendation tools, so treat them as indicative only.

What is more instructive is how this problem works in practice. Nobody reviews each recommendation, but a bad one usually just makes the user scroll past, and the outcome is measured through behaviour: whether users click, buy or keep watching. The system can run fully automatically because of this feedback loop, not because the model is smarter.

Put the three groups in one table and a pattern emerges that no use-case roundup states outright.

Use case	Industry	Data needed	Who signs off	Evidence available
Internal Q&A with citations	All industries	Internal documents	The person asking checks the source	Vendor description
Drafting compliance questionnaires, tenders	B2B	Policies, technical documentation	Expert review	Vendor description
Case law and statute research	Legal	Case law, legislation	Lawyer	Vendor description
Customer service agent	Fintech, retail	Products, customer history, multiple systems	Agent, handing off to a human when needed	Self-reported by Klarna, then adjusted in 2025
Anomaly and fraud detection	Finance	Transactions	Must be designed case by case	Still only a list of capabilities
Product recommendations	E-commerce, streaming	User behaviour	Automatic, measured by behaviour	Second-hand figures, measurable

The pattern is this: the higher the cost of being wrong, the more a person needs to sign off, and where nobody signs off, there must be a metric measured every day. A use case with neither, such as an agent answering every complaint on its own with no path to a human, is exactly where a project is likely to have to reverse course.

An FDE portfolio should show the handoff to a human, not a chatbot

If you are preparing to move into an FDE role, do not build your portfolio around an “ask me anything” chatbot.

Build a narrow RAG system on real data, such as public legal texts, a university’s regulations or the product documentation of an open-source project, require every answer to cite its source, then publish the citation accuracy rate you measured yourself.

On a CV, a line such as “designed a handoff-to-human flow for low-confidence answers, tracked weekly handoff rate” carries far more weight than “built a chatbot using GPT”.

When reading job descriptions, look for mentions of evaluation, human-in-the-loop or integration across multiple source systems, because they signal a company that has moved past the demo stage and is dealing with real operations.

And in your first working session with a client, ask before you propose anything: who signs off on the answer today, and how will they check the machine? The answers to those two questions all but decide what the first use case should be.

The Klarna case does not prove that customer service agents are a bad idea. It shows that even when AI is already handling two-thirds of chats, customers still need a person on the other side when they want one.