Tool calling: the model proposes, your code executes

The demo is going well. The customer-service agent receives the message “my order arrived in the wrong colour, please refund me” and replies that a refund request has been created. The client’s head of operations frowns and asks the question every FDE eventually hears: “So this AI goes into our payment system on its own?”

The correct answer is no, and an FDE has to be able to explain why. Say only that “the model calls the API” and the client will picture a machine freely pressing buttons inside their systems. Understand the mechanism properly and you can show them exactly where the checkpoints sit, and the security conversation goes in a very different direction.

This skill underpins every deployment that involves an agent. Engineers who misunderstand tool calling write loose integrations. Those who understand it can design a tool layer that both the security team and the agent can trust.

The model presses no buttons

Anthropic’s documentation states the contract plainly: the model never executes anything on its own. It emits a structured request, your code runs the operation, and the result goes back into the conversation.

Google says almost exactly the same for the Gemini API: the model does not run functions itself; the application extracts the function name and arguments and executes them.

OpenAI describes the same flow step by step. The application receives a tool call, runs application-side code with the input taken from that call, then sends a second request containing the output. Three major providers, three different APIs, one architecture. The model acts only as the proposer. The power to execute belongs to your code.

There is one exception worth knowing. For some tools, such as web_search and code_execution, Anthropic executes on its own servers, and you never build a tool_result block for them. Every tool that touches a customer’s systems, whether a database, a CRM or a payment gateway, is a tool you define and your code runs.

Every tool call is a round trip

A post from Anthropic’s engineering team calls tools a new kind of software: a contract between deterministic systems and a non-deterministic agent. Because the model cannot run your code, each tool call is a round trip: the model asks, you execute, you report back, the model continues.

When it encounters a tool you have defined, the API returns a tool_use block containing the tool name and JSON arguments, and you send a tool_result in the next request.

In practice the loop is very short: while stop_reason == "tool_use", execute the tool and continue the conversation; on any other stop reason, exit. Below is a hypothetical example for the refund scenario above, written in Python with Anthropic’s SDK:

TOOLS = [{
    "name": "issue_refund",
    "description": "Create a PENDING-APPROVAL refund request for a delivered order.",
    "input_schema": {
        "type": "object",
        "properties": {
            "order_id": {"type": "string"},
            "amount": {"type": "number"},
            "reason": {"type": "string",
                       "enum": ["damaged", "late", "wrong_item"]}
        },
        "required": ["order_id", "amount", "reason"]
    }
}]

def run_tool(name, args):
    if name != "issue_refund":
        return f"Tool '{name}' does not exist. Valid tools: issue_refund."
    order = db.get_order(args["order_id"])
    if order is None:
        return (f"No order {args['order_id']}. "
                "Ask the customer again for an order ID like DH-xxxxxx.")
    if args["amount"] > order.total:
        return (f"Amount {args['amount']} exceeds the order total {order.total}.

"
                f"Propose an amount <= {order.total}.")
    ticket = refunds.create_pending(order.id, args["amount"], args["reason"])
    return f"Created request {ticket.id}, status: awaiting staff approval."

messages = [{"role": "user",
             "content": "Order DH-104233 arrived in the wrong color, please refund me."}]

for _ in range(8):  # cap the number of rounds so the agent can't run forever
    resp = client.messages.create(model=MODEL, max_tokens=1024,
                                  tools=TOOLS, messages=messages)
    messages.append({"role": "assistant", "content": resp.content})
    if resp.stop_reason != "tool_use":
        break
    results = [{"type": "tool_result", "tool_use_id": b.id,
                "content": run_tool(b.name, b.input)}
               for b in resp.content if b.type == "tool_use"]
    messages.append({"role": "user", "content": results})

(The strings in the example are in Vietnamese. The tool creates a refund request pending approval; the error messages tell the agent the order doesn’t exist and to ask the customer for an ID in the form DH-xxxxxx, or that the amount exceeds the order total and to propose an amount no greater than it. The loop comment reads: cap the number of rounds, never let the agent run forever.)

Now read the code through the eyes of that head of operations. The model supplies just three values: an order ID, an amount and a reason. Looking up the order, comparing the amount with the order total, and creating a request pending approval rather than moving money immediately are all decisions made in run_tool. You write them, you test them, you are accountable for them.

Three layers of defence, none a substitute for another

The first layer is the schema. OpenAI recommends using enums and object structure to make invalid states unrepresentable in the schema itself. The reason field above accepts only three values, so the agent cannot invent a reason such as “loyal customer”.

OpenAI also offers strict mode: in an OpenAI function definition, setting strict to true makes function calls reliably follow the schema, rather than on a best-effort basis.

If you use a different API, check its own documentation for an equivalent mechanism.

But JSON that matches the schema can still be wrong for the business. The schema does not know the total of order DH-104233, nor that this customer was refunded last week. That is why the second layer, the checks in code, cannot be dropped, even with strict mode switched on.

The third layer is the one most often overlooked: error messages. Anthropic’s engineering team advises that tool error responses should state specifically what needs to be improved, rather than return an opaque error code or a traceback. “Please propose an amount <= 450000” gives the agent a way to correct itself on the next turn. ValueError at line 42 just leaves it guessing.

Applying it to your own project

Start by listing every action the agent needs to take and splitting them into two groups: read-only and write. For the write group, ask the client during customer discovery which actions require human approval, then design those tools to return a “pending approval” status, as in the example above, instead of executing directly.

Next, write the narrowest schema you can. Use enums for any field with a fixed set of values, and mark as required only what is genuinely needed. Then write run_tool so that every error branch returns a sentence that tells the agent what to do next.

Finally, watch the size of results. A lookup tool that returns 5,000 lines of logs will fill the context very quickly. Anthropic suggests combining pagination, range selection, filtering and truncation for responses like these, and when you truncate, adding a line telling the agent how to request the next page.

Mistakes that break deployments

The most common mistake is believing the model “knows” the rules because you wrote them in the system prompt. A prompt is advice; code is law. Every constraint involving money, access rights or personal data must live in run_tool.

The second is an unbounded loop. If the tool keeps returning errors and the agent keeps retrying, while True will burn tokens forever. A cap on rounds plus a log of every call costs very little compared with an incident at midnight.

The third is dumping raw exceptions into tool_result. They are useless to the agent and can leak file paths or the structure of the client’s database into the conversation.

If you are aiming for an FDE role, this is a skill to make visible when you apply. When a job description asks for experience integrating LLMs or building agents, treat it as the place to describe the tool layer you have built in detail. On your CV, don’t just write “LLM integration”.

Spell out where you put validation, which tools required human approval and how you paginated results. An interviewer will see at once that you know which side holds the power to execute.

The next time a client asks “Can the AI do this on its own?”, don’t rush to reassure them. Open run_tool and walk them through each line of code that decides what is allowed to happen.