Context-budget-aware tools: generated tools that respect your token window

The scarcest resource in your agent stack isn't compute — it's the context window. Tool design has to treat it that way.

The scarcest resource in your agent stack isn't GPU. It isn't API rate limits. It's the context window — and almost everything in the tooling ecosystem is designed as if it were infinite.

Here's the failure mode, and if you've built agents against real APIs you've lived it:

You take a serious internal API — say, 300 endpoints, enterprise-grade OpenAPI spec — and you do the "obvious" thing: generate one tool per endpoint, schemas verbatim, and hand the pile to the model.

Two things go wrong immediately.

The specs eat the window before work starts. Every tool definition — name, description, parameter schema — is tokens the model carries in every single request. Three hundred tools with fat schemas can burn a five- or six-figure token count as pure overhead. Your agent is exhausted before it reads the user's question.

The responses finish the job. One innocent listCustomers call returns 200 records × 40 fields of JSON. That single tool result can outweigh your entire conversation history. Two or three calls like that and the context is full of unread payload — the model starts forgetting its own plan, earlier results fall off the cliff, and quality collapses in that maddening, hard-to-debug way where nothing errored but everything got worse.

Nothing failed loudly. The agent just got dumber with every call. That's what an unbudgeted tool surface does.

Tokens are a budget. Spend them like one.

The fix isn't a bigger window — every window fills; bigger windows just fail later and cost more per failure. The fix is treating tokens as a first-class budget in tool design, on both sides of the call:

Spec-side: what does it cost to carry this tool? A tool's definition is rent paid on every request. That forces real decisions: not every endpoint deserves to be a tool; descriptions should be written for a model deciding when to call, not copied from human reference docs; schemas should carry what the model needs to call correctly, not every nested corner of the response type.

Response-side: what may a result inject? A tool result is context you can never un-spend. Unbounded responses are a design bug — results need caps, pagination as the default posture, and field surfaces chosen for the task rather than dumped wholesale.

This is the same discipline rysh applies to agent loops — budgets measured in pages, books, and shelves, hard ceilings instead of vibes. Tools are where that philosophy has to reach next, because tools are where most of the tokens actually go.

What Forge does about it

Rysh Forge generates agent tools from an OpenAPI spec — and "context-budget-aware" is the design constraint the generation is built around, not a feature bullet appended afterward. Generating from a spec is exactly where budget-awareness belongs, because the generator sees the whole picture at once — every endpoint, every schema, every response shape — and can make the economic decisions before any tokens are spent:

• The tool surface is a decision, not a dump. A spec describes an API; a good toolset describes a job. Mapping 300 endpoints to 300 always-loaded tools is the naive move Forge exists to avoid.
• Definitions are written for the payer. The model pays rent on every schema; generated descriptions and parameter surfaces are shaped for call-time decision-making, at model-appropriate size.
• Responses are treated as injections. Caps and pagination-first defaults, because "the tool returned 4,000 lines of JSON" should be an explicit choice, never a surprise.

You get the integration in minutes and an agent that's still smart after ten calls. Those are usually a trade-off; the entire point of budget-aware generation is refusing the trade.

This lesson is bigger than rysh

If you take one thing from this post into whatever stack you run, take the audit. For every tool you expose to an agent, answer three questions:

1. Carry cost — how many tokens does this definition add to every request, and does its usefulness pay that rent?
2. Worst-case injection — what's the largest result this tool can return, and have you actually capped it?
3. Decision quality — could a model reading only your description reliably decide when to call this and when not to?

Most tool surfaces — hand-written or generated, MCP or bespoke — fail at least two of the three. It's also why the MCP ecosystem, for all its genuine promise, is uneven: the protocol standardizes how tools are exposed, but not the economics of exposing them well. A context-bomb tool served over MCP is still a context bomb. (rysh consumes MCP servers — visibly and gated — but no client can rewrite a wasteful server into a frugal one.)

Token discipline is a tool-design property. It has to be built in at the source — which is exactly why Forge generates it in.

Honest scope

• rysh runs on Claude — bring your own Anthropic key. The provider layer is pluggable; Claude is what's wired today. No secret better model.
• rysh is self-hostable — your machine, your keys, your data. It is not open source.
• Forge is OpenAPI-only today; GraphQL and gRPC are roadmap, not live.
• Budget-aware generation manages the economics of a tool surface; it can't conjure semantics a spec doesn't have. Empty descriptions and type: object everywhere will produce frugal-but-vague tools. Garbage in, cheaper garbage out.
• No benchmark numbers in this post on purpose: token overhead varies enormously with spec size and usage shape. Run the three-question audit on your own surface — that's the measurement that matters.

Fighting the context window in your own agent stack? That's a design-partner conversation we'd genuinely enjoy — bring your ugliest spec. → rysh.ai/design-partner

Companions: Rysh Forge: point it at an OpenAPI spec · Pages, books, and shelves: budget your agents like a library