LLM Tool Calling: A Practical Guide to Function Calling in 2026

Who this is for: developers who have called an LLM API a few times and now need to wire it into real tools - a database, an internal API, a payment provider - and want the request/response shapes, the failure modes, and the provider differences without wading through three vendors' docs.

LLM tool calling is the mechanism that turns a model from a text generator into something that can look things up, take actions, and chain multi-step work. You describe a function with a JSON schema, the model decides when to call it and with what arguments, your code executes it, and you feed the result back. Everything from customer-support bots that check order status to coding agents that run tests depends on this loop working correctly under real conditions - malformed arguments, five tools firing at once, a downstream API returning a 500.

Most of the pain in production tool-calling systems isn't the happy path. It's the edges: what happens when the model hallucinates a parameter, when you need it to call a tool no matter what, or when three providers disagree on how to report a failed call. This guide covers the mechanics that matter for shipping something reliable.

How tool calling actually works

The request/response cycle is the same shape across every major provider, even though field names differ:

1. You send a request with a tools array (name, description, JSON Schema for inputs) alongside the normal messages.
2. The model returns either a normal text response or one or more "tool use" blocks containing a tool name and a set of arguments.
3. Your code executes the matching function with those arguments.
4. You send the result back as a new message, tagged with the same tool-call ID.
5. The model reads the result and either responds to the user or calls another tool.

On Anthropic's Messages API, step 2 comes back as a tool_use content block with an id, name, and parsed input object; step 4 is a tool_result block in a user message referencing that same tool_use_id. OpenAI's Chat Completions API uses tool_calls on the assistant message and a role: "tool" message keyed by tool_call_id. Gemini calls it functionCall / functionResponse. The concepts map 1:1; only the JSON keys change.

The critical implementation detail people get wrong: tool arguments arrive as a string that must be JSON-parsed, and that string is not guaranteed to be safe for naive string matching. Never do if input.includes('"admin": true') against the raw payload - always parse it into an object first. Recent model generations can escape Unicode or forward slashes differently than you'd expect, and code that string-matches the serialized input breaks silently when that happens.

Defining a tool schema that the model actually uses correctly

The schema is not decoration - it's the interface contract, and the model's accuracy at filling it correctly is proportional to how much work you put into the description.

{
  "name": "get_shipping_status",
  "description": "Look up the current shipping status for an order. Call this when the user asks where their order is, whether it has shipped, or for a tracking number. Do not call this for questions about return policy or pricing.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "The order ID, format ORD-XXXXXX. Ask the user for this if not already provided."
      },
      "include_tracking_events": {
        "type": "boolean",
        "description": "Whether to include the full tracking event history or just the current status."
      }
    },
    "required": ["order_id"],
    "additionalProperties": false
  }
}

Three things matter more than people expect:

• State the trigger condition, not just the function. "Call this when the user asks about X" gets meaningfully better trigger accuracy than a description that only explains what the tool does. This matters more on newer models, which have gotten more conservative about reaching for tools by default - they'll answer from context rather than call a tool unless the description makes the "when" explicit.
• Use enum for closed value sets. Don't leave a status field as a free string if there are only five valid values - enumerate them. This eliminates an entire class of downstream validation errors.
• Keep the tool count focused. Past roughly 15-20 tools in a single request, selection accuracy degrades. If you're past that, look at MCP servers for dynamic tool discovery, or a tool-search pattern that only loads schemas relevant to the current turn.

Strict schema validation

Anthropic supports strict: true as a top-level field on the tool definition (schema must set additionalProperties: false and required), which guarantees the returned input validates against your schema exactly - no more defensively parsing loosely-typed output. OpenAI has an equivalent strict flag on function definitions. If your downstream code assumes a shape, turn this on; it removes a whole category of runtime KeyError/undefined bugs.

Parallel tool calls

Modern models can request several tools in a single turn - for example, checking inventory for three different SKUs at once instead of three sequential round trips. This is on by default; a single assistant message can contain multiple tool_use blocks.

The rule that trips people up: execute all the calls, but return every result in a single message. Splitting tool results across multiple follow-up messages silently trains the model to stop batching its calls - it starts making them one at a time again because it's learned that parallel requests don't get handled together.

# Pseudocode - works the same shape across SDKs
tool_uses = [b for b in response.content if b.type == "tool_use"]

results = await asyncio.gather(*(execute_tool(t.name, t.input) for t in tool_uses))

# One user message, one tool_result block per call
next_message = {
    "role": "user",
    "content": [
        {"type": "tool_result", "tool_use_id": t.id, "content": r}
        for t, r in zip(tool_uses, results)
    ],
}

If you need the opposite behavior - at most one tool call per turn, for a UI that can only render one action at a time - pass disable_parallel_tool_use: true alongside your tool_choice.

Forcing tool use with tool_choice

There are four states, and picking the right one avoids a lot of prompt-engineering theater:

If you've ever tried to get "reliable JSON output" out of a model by begging in the system prompt ("You MUST always respond with valid JSON and nothing else"), forced tool choice replaces that entirely. Define a tool whose schema is your desired output shape, force it with {"type": "tool", "name": "extract_data"}, and read the arguments - no more stripping markdown code fences off a text response that occasionally has a stray sentence before the JSON. This is also the pattern to use instead of assistant-message prefilling, which several current models reject outright as a way to steer output format.

For a more general take on this problem, see structured outputs - some providers now offer a dedicated response-format parameter that constrains the model's raw text output to a schema, separate from tool calling proper, and it's often the better tool for pure extraction tasks with no side effects.

Error handling that doesn't leak into the model's context wrong

A tool call can fail for reasons that have nothing to do with the model: the downstream API times out, the argument the model gave you doesn't correspond to a real record, your own code throws. How you report that failure back matters as much as catching it.

Always return a tool_result, even on failure - never drop it. If a tool call fails and you simply don't send a result, the model's context is left with a dangling call it can't reason about, and most providers will error on the next request because a tool_use block has no matching result. Set is_error: true and put a human-readable explanation in content:

try:
    data = lookup_order(order_id)
    result = {"type": "tool_result", "tool_use_id": tool_use.id, "content": json.dumps(data)}
except OrderNotFoundError:
    result = {
        "type": "tool_result",
        "tool_use_id": tool_use.id,
        "content": f"No order found with ID {order_id}. Ask the user to double-check it.",
        "is_error": True,
    }
except UpstreamTimeoutError:
    result = {
        "type": "tool_result",
        "tool_use_id": tool_use.id,
        "content": "The order lookup service timed out. Do not retry automatically - tell the user to try again shortly.",
        "is_error": True,
    }

The model generally handles is_error: true gracefully - it'll acknowledge the failure, try an alternate approach, or ask the user for clarification instead of hallucinating a plausible-looking answer. What breaks things is silence: a missing result, or a raw stack trace dumped into content that the model then quotes back to the user verbatim, exposing internal error details it shouldn't. Translate exceptions into a sentence a non-technical user could read before they hit the model's context - this is the same principle behind never exposing internal errors to the client in any AI-facing product.

Common mistake: validating the tool's arguments only for type-correctness and skipping business-logic validation. A model can produce a syntactically valid order_id that simply doesn't exist, or a quantity: -5 that passes JSON Schema but is nonsensical. The schema catches shape errors; your handler still needs to catch semantic ones.

Provider differences that actually matter

The practical implication: if you're building a multi-provider abstraction layer (see Vercel AI Gateway or a comparable router), don't assume you can serialize one tools array and fan it out unmodified - the wrapper keys differ enough that a thin adapter layer per provider is worth writing rather than fighting a lowest-common-denominator abstraction that hides the differences you actually need to know about, like error-result shape and strict-mode support.

Building the agentic loop

Most SDKs now ship a "tool runner" helper that automates the call-execute-loop cycle so you don't hand-write a while loop. Anthropic's Python and TypeScript SDKs expose client.beta.messages.tool_runner(...); you define your functions with a decorator or schema wrapper, and it handles calling the API, detecting tool-use blocks, executing your functions, feeding results back, and stopping when the model has no more calls to make.

Reach for the manual loop only when you need something the runner doesn't expose - a custom transport, non-standard streaming, or control flow that doesn't fit a per-turn hook. Otherwise the runner covers the common needs people assume require a manual loop: human-in-the-loop approval gates (return a "user declined" result from inside the tool function instead of executing), retry logic, and result post-processing before it goes back to the model.

from anthropic import Anthropic, beta_tool

client = Anthropic()

@beta_tool
def get_shipping_status(order_id: str) -> str:
    """Look up shipping status for an order. Call when the user asks where their order is."""
    return lookup_order(order_id)  # your implementation

runner = client.beta.messages.tool_runner(
    model="claude-opus-4-8",
    max_tokens=1024,
    tools=[get_shipping_status],
    messages=[{"role": "user", "content": "Where is order ORD-88213?"}],
)
final = runner.until_done()

If you're deciding whether you need this level of agentic wiring at all versus a single structured call, see vibe coding: when to use it and agentic workflow patterns for the decision criteria - complexity, value, viability, and whether errors are recoverable. A lot of "I need an agent" requests are actually a single tool call with forced tool_choice.

Security: tool calls execute untrusted model output

Treat every argument a model hands you as untrusted input, the same way you'd treat a value from an HTTP request body. A bash or file-path style tool is the highest-risk case: validate that a file path resolves within your intended root before touching the filesystem, and never pass a model-generated string directly into a shell command without an allowlist of permitted operations. This is the same discipline covered in more depth in vibe coding security risks - the model is a co-author of the input your backend receives, not a trusted caller.

For tools with real side effects - sending money, deleting records, sending an email - add a confirmation gate rather than executing automatically. Several providers support an explicit "ask before running" permission mode per tool for exactly this reason; use it for anything irreversible.

FAQ

What's the difference between tool calling and function calling?

They're the same concept under two names. "Function calling" is the older, more literal term (the model calls a function you defined); "tool calling" is the broader term that now also covers server-executed capabilities like web search or code execution that aren't strictly user-defined functions. Most current documentation uses "tool calling" as the umbrella term and reserves "function" for the specific case of a custom, client-executed callback.

Can I force an LLM to always use a specific tool?

Yes - set tool_choice to the forced-tool variant ({"type": "tool", "name": "..."} on Claude, {"type": "function", "function": {"name": "..."}} on OpenAI). This is the standard pattern for using tool calling to get reliably structured output, rather than for letting the model decide when to act.

Why did my tool calls stop happening in parallel?

The most common cause is returning tool results across multiple separate messages instead of one message containing all of them. Batch every tool_result for a given turn into a single user message - if you split them, the model tends to relearn a one-call-at-a-time pattern within a few turns.

How do I handle a tool that fails or times out?

Always send back a tool_result (never silently drop it), mark it is_error: true, and put a plain-language explanation in the content - not a raw exception or stack trace. The model uses that signal to retry differently or ask the user for more information instead of guessing at an answer.

Do I need a JSON Schema validator library, or is the model's output already valid?

Use strict: true where the provider supports it - it guarantees schema-valid output at the API level, so a separate validation pass is largely redundant for providers that support it. On providers without strict mode, or for business-logic checks the schema can't express (does this order ID actually exist), you still need your own validation layer.

What's the maximum number of tools I can give a model?

There's no hard API cap in most cases (Anthropic's Managed Agents surface caps at 128 per agent, for instance), but selection accuracy degrades well before you hit any technical limit - past roughly 15-20 tools, expect more wrong-tool picks. Use dynamic tool discovery (MCP, or a tool-search pattern) once you're past that range rather than stuffing every schema into every request.

Getting this right in production

Tool calling is the foundation most useful AI features are built on - lead capture, order lookups, internal automations - and the gap between a demo and a production system is almost entirely in the error handling and schema discipline covered above, not in the happy-path wiring.

If you want a second pair of eyes on a tool-calling integration you're building - or want it built for you - book a free 30-minute call through the contact form or message us directly on WhatsApp.

Sources

• Anthropic - Tool Use Overview
• Anthropic - Messages API Reference
• OpenAI - Function Calling Guide
• Google - Gemini Function Calling
• Model Context Protocol Specification