Agent API Design: Designing Tool APIs That Survive Production

Agent API design is how you design the tools, functions, and response payloads a language model calls, so an agent can use them without breaking. Most guides on the topic cover function-calling schema syntax: how to name a function, write its description, and constrain its parameters. That advice is correct and necessary, and it is also the easy half. The hard half is what happens after the schema is correct and real traffic hits it.

This page assumes you can already write a clean tool definition. It covers what the best-practice guides skip: the production failure modes that show up once your well-formed schema meets multiple providers, streaming responses, and inputs you did not anticipate.

Why Is Designing a Tool API for an Agent Different?

Your API has always had one consumer: a developer who reads the docs, tests against staging, and files a ticket when something is unclear. An agent is a second consumer with none of those traits. It cannot ask for clarification, it reads your tool description as the input to a routing decision rather than as documentation, and it runs unattended at 3am on an input no human reviewed.

That changes what “good design” means. A human consumer recovers from an ambiguous error by reading it. An agent acts on it. So the consensus best practices are worth stating once, because they are the floor:

• Write tool descriptions as classifiers, not labels. “Search the web for current events and news” routes better than “Search the web.”
• Include negative guidance: when not to use a tool, and which tool to use instead.
• Minimize required parameters. Every required field is a value the model can get wrong.
• Keep the active toolset small. Selection accuracy drops as the tool count climbs, which is why teams narrow to a handful of tools per workflow.

Get those right and the model will usually call the correct tool with plausible arguments. The rest of this page is about the failures that survive that floor.

JSON Schema Drift: One Tool Schema, Three Providers, Three Behaviors

The first surprise in production is that a schema which works on one provider is not portable. JSON Schema support is implemented differently across model APIs, so the same constraint can produce three different outcomes.

The dangerous case is the middle one. A provider that rejects your schema gives you a loud error you fix before shipping. A provider that silently ignores a constraint, then invents a value that violates it, gives you a tool call that looks successful and is wrong. Mastra’s engineering team measured this directly: constraints like format: uri and nullable were rejected by some models and silently dropped by others, producing tool-calling error rates of 15 to 27% depending on the model. Moving the constraint into the property’s plain-language description cut that to about 3%.

The design rule follows from that result: do not rely on schema keywords alone to enforce a constraint. State the constraint in the property’s description in plain language too, and test your schema across every provider you target before you trust it on any of them. The same caution applies when a provider changes its tool-calling contract under you, which happens often enough that a working integration can break on a model update you did not make.

How Should a Tool Return Errors to an AI Agent?

The second failure mode is the quietest, and it is the most common cluster of tool-calling complaints in Mutagent’s community-research corpus of 7,797 developer pain quotes. A tool call returns, the model reads the return as truth, and the return was never a success.

“Agent reads status: \"ok\" off an API call, marks the step complete, downstream batch reconciliation surfaces the actual failure 30 minutes later.” — r/aiagents

An API that returns HTTP 200 with {"status": "ok", "data": null} is encoding a failure as a success-shaped payload. A human reads the null and investigates. The model reads the 200 and the ok and proceeds. The agent cannot recover from an error it cannot detect.

// Ambiguous: a failure wearing a success shape
{ "status": "ok", "data": null }

// Actionable: the model can branch on this
{
  "status": "error",
  "error_type": "account_not_found",
  "retriable": false,
  "message": "No account matches id 8812"
}

Design the response so failure is structurally distinct from success. Return an explicit error type the model can branch on, not a generic string buried in a message field. Mark whether the error is retriable, so the model knows the difference between “try again” and “stop.” And never return an empty success when the real answer is “not found” or “partial.” If an agent cannot tell, from the shape of the response alone, whether the call worked, the shape is the bug.

Design the Return Payload for a Token Budget

The third failure mode is about cost and context, not correctness, until it becomes correctness. Every tool result is fed back into the model along with the full conversation history. A verbose raw API payload, returned on every call, eats the token budget and pushes the agent toward context-window overflow.

“every tool invocation is a separate inference with the full conversation context, so a 10-step agent pipeline with 3 tool calls per step is 30+ inferences, each one carrying the entire context.” — r/LLMDevs

Context-window overflow is a top cluster in the corpus for a reason: teams design the tool to return what the API returns, not what the agent needs. A list endpoint returns fifty fields per row when the agent uses three. Pagination metadata, internal IDs, and timestamps all ride along, and the model pays for them on every subsequent step.

The fix is to treat the tool’s return schema as a separate design surface from the upstream API. Project only the fields the agent will act on. Summarize or paginate large results inside the tool before they reach the model. The upstream API can stay verbose; the tool contract should not.

Streaming Adds a Timing Contract

If you stream tool calls, you inherit a fourth class of failure that has nothing to do with the schema and everything to do with timing. A tool call assembled from stream chunks is not valid JSON until the last chunk arrives, and parsing it early is a trap:

“Standard JSON.parse() throws an error if you try to parse a chunk mid-stream. If the stream gets truncated due to a timeout or token limit…” — r/nestjs

Three rules keep streaming tool calls safe. Use a tolerant partial parser that accumulates chunks and never executes on an incomplete object. Wait for the provider’s explicit finalization signal before firing any action, because providers stream tool calls with different event models and “looks complete” is not “is complete.” And require an idempotency key as a schema-level parameter on every write tool, so a retry after a mid-stream interruption cannot double-charge, double-send, or double-write.

Why 97% Per Call Is Not Enough

Each fix above raises the odds that a single tool call behaves. The reason none of them is optional is the same compounding math that governs every multi-step agent. End-to-end reliability is per-call reliability raised to the number of calls. At 97% per call, a ten-step agent completes cleanly only about 74% of the time, a twenty-step agent drops to about 54%, and a twenty-five-step agent falls below half. This is the same curve that makes agent patterns a reliability bet rather than an architecture choice.

A layered validation stack is what pushes per-call reliability up. Most teams ship with only the first layer.

Layer	What runs it	What it catches
Syntax	Provider strict mode / constrained decoding	Malformed JSON, missing required fields
Structure	A validator such as Pydantic or Zod	Wrong types, extra or absent keys in the return value
Semantics	Domain checks in your tool code	Valid-but-wrong values, like an end date before a start date
Drift	Monitoring across runs	The slow regression after a provider quietly updates a model

Next Steps: A Tool API the Agent Can Trust

Schema syntax gets the model to call the right tool. The four production failure modes here, cross-provider drift, success-shaped errors, payload bloat, and streaming timing, decide whether that call can be trusted once it ships. Designing for them is the difference between a tool that demos well and one that holds at scale.

The same return payloads you design here become the traces that tell you when a tool starts failing, which is why agent API design sits in the build hub right next to observability as two ends of the same loop. You cannot fix a tool contract you cannot see breaking.

Mutagent’s autonomous AI Engineer closes that loop: it reads the traces your tools produce, finds the call where reliability dropped, whether it is a provider that started ignoring a constraint or a payload that grew past the budget, and proposes the targeted fix before your users hit it. Meet the autonomous AI Engineer to see it work on your own tool calls.