Tool Budget Audit

Tool Budget Audit

Tools cost context in two ways, plus one side effect:

• Definitions. Every tool's name, description and parameter schema sits in context whether or not it is used this turn.
• Intermediate results. Every call puts its full output back into the conversation, and every round trip is another full model request.
• Side effect: wrong picks. Similar names and vague descriptions make the model choose the wrong tool or pass wrong arguments.

Tool search mainly saves the first cost. Programmatic tool calling (the model writes a small program that calls tools) mainly saves the second.

Companion to MCode Engineering Notes 02, "Too many tools will eat your context". Plain-text checklist: checklists/tool-budget.md.

1. Measure first

Work from the target project's code and real requests, not from guesses:

1. List every tool exposed to the model, grouped by source (built-in, each MCP server, plugins).
2. Estimate the token size of each tool definition with a tokenizer and sum it. Say which tokenizer you used; it is a relative measure.
3. For one or two representative tasks, record how many tool calls were made, how many model requests that caused, and how large the tool results were.
4. Note which tools are used almost every turn and which are rarely used.

Report the numbers you measured. If you cannot measure something, say so instead of estimating.

2. Step 0 - subtract

• Merge by user intent. Do not map APIs one to one. "List issues", "filter issues by label" and "filter by author" can be one tool with parameters.
• Make names and descriptions distinguishable. Two tools whose descriptions read alike will be confused. State the return format.
• Group. Put tools into namespaces by domain, keeping each group small.
• Decide what stays resident. Read, edit, search and run-command style tools used nearly every turn should always be loaded.

3. Step 1 - deferred loading and tool search

Consider it when there are many tools and most are rarely used.

• Keep core high-frequency tools resident; make low-frequency tools searchable.
• If a search miss means a tool "does not exist", check descriptions again: vague descriptions break recall.
• Test tool search recall on your own tool set. Results on clean benchmarks do not transfer to messy real catalogs.
• Tools whose catalog must be fetched over the network every turn can slow the first token. Consider keeping them resident instead.
• When debugging, check what the model had actually seen at that moment, not just what the request listed.

4. Step 2 - programmatic tool calling

Consider it when there is a lot of data to filter or aggregate, three or more dependent calls, or batch operations that could run in parallel.

• Actually move tools into the program. Offering a code tool next to all the normal tools tends to go unused.
• Keep at top level: tools that interact with the user, change session state, need their result read before deciding the next step (tool search, loading a skill), browser and publishing tools, and spawning subagents.
• Calls from inside the program must go through the same validation, hooks, permission prompts and UI as direct calls.
• Tool failures should return an error result, not throw, so one failure does not discard the whole program.
• Prefer structured tool output so the program can pick the fields it needs. A program that returns raw output saves nothing.
• Search the codebase for logic keyed on tool names (reminders, counters, analytics). It silently stops firing once a tool moves into the program.

5. Decide

The steps stack. A clean catalog makes tool search work; working tool search makes programmatic calling work. Run an end-to-end eval before and after each step and look at score, request count and tokens together (see harness-experiment).

Output

A short report: measured numbers, the main bottleneck, the recommended next step with concrete tool names, and what to measure afterwards.