DIRTY STATIC / DISPATCH Tools20 NOV 2026 • 14 MIN READ
A vast wall of grey tool cabinets in a brutalist concrete warehouse, with one drawer open and glowing caution yellow and a robotic hand pulling out a single glowing schema document
ToolsINTELLIGENCE FOR THE MACHINE AGE

Gemini Tool Retrieval: Load Tools Only When Needed

Google launched Tool Retrieval in preview in the Gemini Interactions API on November 19, 2026. Agents keep a large tool library on standby, show the model only a name and a short description for each deferred tool, and load the full parameter schema only when a task needs that tool. This profile covers how the server side and client side modes work, how remote MCP servers and caching fit in, what the billing really counts, and the limits Google documents and the figures Google does not publish.

Start reading ↓

An agent with six tools has no problem. An agent with sixty or six hundred tools has a tax on every single request. Each tool a developer exposes, such as looking up an order or issuing a refund, travels with a parameter schema that lists its arguments, their types and which ones are required. Send every schema on every turn and the prompt grows, the bill grows and the model has to pick the right tool from a longer and longer list.

Google names those three costs plainly in its launch post: inflated input token counts, added latency and lower tool selection accuracy as the candidate list grows. Tool Retrieval, launched on November 19, 2026 in preview in the Gemini Interactions API, is its answer. This article explains what the feature does, how its two modes differ, what it costs in practice and where its documented limits sit.

Gemini Tool Retrieval

Official site

A preview feature of the Gemini API for agents with large tool libraries. Primary use case: keeping dozens or hundreds of tools available to an agent without sending every full parameter schema on every request.

Key features: a tool_search tool plus a defer_loading: true flag on each tool held back; server side search run by Gemini with no index for the developer to build or host; a client side mode for teams with their own registry, embedding index or permission filter; deferred loading for a whole remote MCP server in one setting; and a visible step record in interaction.steps that shows which tools the model searched for and loaded.

Advantages: frequent tools can stay loaded up front while the long tail waits on standby; loaded schemas are added later in the conversation, so the cached prefix at the start of the prompt stays stable from turn to turn, per Google. Limitations: the feature is in preview and Interactions API only; Google publishes no savings, latency or accuracy figures of its own; and a schema once loaded stays loaded for the rest of the conversation.

Platforms
Gemini Interactions API, version v1beta; Gemini 3.6 and later models, including all Gemini 4 models per Google
Pricing
No separate charge. Input and output tokens are billed at the standard rate for the model called, checked November 20, 2026

What it is

A tool declaration has two jobs. It tells the model that a tool exists, and it tells the model exactly how to call that tool. Tool Retrieval separates those jobs in time. The existence part, a name and a short description, is present from the start. The calling part, the full parameter schema, arrives only when the model decides it needs that tool for the task in front of it.

That distinction is the whole feature. The model is never blind to its library. It can see what is available and reason about which tool fits. What it does not carry, turn after turn, is the detailed argument list for every tool it will probably never touch in this conversation.

Google advises the obvious split. Keep the tools you use most often loaded up front in the normal way. Defer the long tail, the tools that matter once in a while and cost tokens on every other turn while they wait.

How server side retrieval works

Turning the feature on takes two changes, and Google stresses that there is no index to build or host because Gemini runs the search on the server. Add a tool_search tool to the request. Then set defer_loading: true on each tool you want held back. The documentation allows exactly one tool_search tool per request and requires the v1beta version of the API, per coverage of that documentation.

from google import genai

client = genai.Client()

tools = [
    {"type": "tool_search"},
    {
        "type": "function",
        "name": "issue_refund",
        "description": "Refund an order to the original payment method.",
        "defer_loading": True,
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount": {"type": "number"},
            },
            "required": ["order_id", "amount"],
        },
    },
    # ... 50+ other deferred functions (leave frequent tools undeferred)
]

interaction = client.interactions.create(
    model="gemini-4-hydrogen",
    input="Order A-1042 arrived damaged. Please refund 40 dollars.",
    tools=tools,
)

The example above is Google own refund example, shortened only in the comment line count it carries. A customer reports that order A-1042 arrived damaged and asks for a refund of 40 dollars. Inside that single API call, three steps happen in order. The model emits a tool_search_call asking for issue_refund. The API answers with a tool_search_result that carries the full schema. The model then emits a normal function_call with the order ID and the amount filled in. From there it is ordinary function calling: your code runs the refund and sends back a function_result.

All three steps appear in interaction.steps. That record matters more than it first looks. When an agent picks the wrong tool or fails to pick the right one, the steps show which tools the model considered, which makes weak names and weak descriptions visible as the bugs they often are. Tool naming was already a quiet craft in agent building. This feature turns it into a logged one.

A glowing yellow schema document on a conveyor passing through a concrete search gate, with closed grey toolboxes waiting on standby shelves
One schema moves when it is needed. The rest of the library waits on standby and costs only its listing line. Illustration: Dirty Static

The client side mode, for your own registry

Not every team wants Google to run the search. A team may already keep its own tool registry, its own embedding index or a permission filter that decides which tools a given user may reach. For those teams Google offers client side retrieval. Declare the tool_search tool with "execution": "client" and define your own search parameters, such as a single query string.

The trade is clear. Your app runs the lookup and returns matching function declarations in a function_result, in a follow up request that points back to the earlier one. That costs one extra round trip per search. In return, tools do not need to be declared up front at all, so a new tool can appear in the middle of a conversation when a user connects a new integration. The two modes cannot be mixed in a single request.

That second point connects to work this magazine has covered all month. Large MCP servers pose the same problem at a larger scale, whether the server carries close to twenty analytics tools with write access or 84 app store tools in a single Mac app. A library that grows by connection, rather than by code release, needs a loading rule that grows with it.

MCP servers and the cached prefix

Remote MCP servers get the simplest version of the feature. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand. There is no need to list the tools of a large server one by one just to defer them.

Caching gets a quieter benefit, and Google explains the mechanism rather than just claiming the result. Deferred schemas stay out of the start of the prompt. Schemas the model does load are added later in the conversation. The opening stretch of the prompt, the prefix that caching depends on, therefore stays the same from turn to turn. A design that inserted each loaded schema near the top would break that cache and could cost more than it saved.

What it costs

There is no separate charge for Tool Retrieval, and that sentence needs the rest of this section to mean anything. Input and output tokens are billed at the standard rates for the model you call. What changes is which tokens exist.

Billing partWhat counts
The tool_search tool itselfA small fixed block of input tokens; Google does not give its size in the launch post
Each deferred tool listingAbout 26 tokens per tool in early testing, reported from the documentation
Each schema the model loadsBilled as input for the rest of the conversation, the same as a tool declared up front
Each server side searchA further model pass over the context with the loaded schemas, reported as tool use tokens and billed at the input rate, per the documentation
Each client side searchOne extra round trip between your app and the API

Do the rough maths on the listing line alone. At about 26 tokens a line, a library of 100 deferred tools adds roughly 2,600 input tokens to a request before any schema loads. That is a real cost, and it is also plainly smaller than 100 full schemas would be, though Google gives no average schema size that would let anyone state the gap as a number. Google own closing advice is the honest one: you save tokens when the schemas you never load outweigh the ones you do, which it calls the usual case for large libraries, and you should measure the effect against your own traffic.

The limits in the documentation

The launch post is short. The documentation, as reported in detailed coverage of it, carries rules that shape real use and deserve space here.

A loaded schema cannot be unloaded. Once the model loads a tool in a long conversation, that schema stays in context and keeps billing for the rest of the conversation. The full tool list also travels every time: the API does not carry tool declarations from one request to the next, so the complete tools array goes out with every request in a conversation.

A tool named in your system instructions, or in the output of another tool, should not be deferred, per the documentation. The model may try to call such a tool without searching first, and the next request can then fail. A search that matches no deferred tool is also quiet: it produces a text response rather than a structured retrieval error, so code that waits for a tool_search_result step to detect a miss will wait in vain.

Support needs care too. The launch post says the feature works with Gemini 3.6 and later models, including all Gemini 4 models. The documentation lists four supported model codes, gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash, and says other models return an error when defer_loading is set. Whether the post or the documentation is ahead of the other, Google does not say. Pin a listed model if you build on this in preview, and expect the surface to change before general availability. The older generateContent method does not support the feature at all.

Third to the same idea

Google does not mention any rival in its post, but the design has a history worth recording, because it tells you this is now the standard shape for large tool libraries rather than one vendor experiment.

ProviderLaunchedNamePublished result
AnthropicNovember 24, 2025Tool Search Tool85 percent token reduction in its own testing, from about 77,000 to about 8,700 tokens
OpenAIMarch 5, 2026Tool search47 percent reduction in total token use across 250 tasks on 36 MCP servers, in its own testing
GoogleNovember 19, 2026Tool RetrievalNo figure published

All three use the same flag name, defer_loading: true. The figures in the first two rows come from each company tests on its own models and are not comparable with each other, let alone with a Gemini result that does not exist yet. What the table does support is narrower and still useful: three large providers looked at the same cost, reached for the same split between a listing and a schema, and shipped it. If you build agents on more than one platform, your tool descriptions now have to serve as searchable listings on all of them.

What Works

  • No index to build or host in server side mode; two changes turn the feature on
  • Frequent tools stay loaded while the long tail waits at about 26 tokens a listing
  • Whole remote MCP servers defer in one setting, with no tool by tool listing
  • Every search, load and call is visible in interaction.steps for debugging
  • Loaded schemas sit late in the prompt, so the cached prefix stays stable

What Does Not

  • Preview only, in the Interactions API only, with a documented model list narrower than the launch post wording
  • No savings, latency or accuracy figures from Google to plan around
  • Loaded schemas cannot be unloaded and keep billing for the rest of a conversation
  • Tools named in system instructions should not be deferred, which rules out a common prompting habit
  • A missed search returns plain text, not a structured error your code can catch cleanly

Which should you choose

Choose server side retrieval if your library is large and your registry is simple. If your agent already declares dozens of tools in code and the long tail is the problem, this is the shortest path: one new tool, one flag on the tools you defer, and no search infrastructure of your own to run or maintain.

Choose client side retrieval if permissions or discovery are yours to govern. Teams with a per user permission filter, their own embedding index or tools that appear when a user connects an integration need the lookup in their own hands. The extra round trip per search is the price of that control, and for those teams it is plainly worth paying.

Stay with full loading if your library is small. An agent with a handful of tools gains nothing from a search step, a listing line per tool and a further model pass per lookup. Full schemas up front are simpler, and simple wins until the count makes it lose. The same caution applies to very long conversations where early loads never unload: count your likely loads before you defer.

Look at the wider pattern if you test browsing and tool agents. Google has been building places for agents to fail safely, as our profile of its sealed mirror web for browsing agents shows, and narrow scoring tools such as the one in our Gemini Decisions API tutorial show the same instinct from another side: give the model less to carry, and make each step easier to inspect.

Final thoughts

Tool Retrieval is not a flashy launch, and that is part of why it matters. It takes a cost that every serious agent builder already pays in silence, the schemas of tools the current task will never use, and makes that cost optional for the tools that stay on standby. The mechanism is small, the step record is genuinely useful, and the MCP and caching details show the design was thought through as a system rather than as a demo.

The missing numbers are the honest gap. A feature sold on saving tokens and protecting accuracy should eventually show its work, and Google has not yet. Until it does, treat Tool Retrieval as a sound design in preview, verify the model list against the documentation rather than the post, and let your own traffic tell you what it saves. Libraries of hundreds of tools are not going away. Paying full price to describe all of them on every turn should.

Sources

■

END OF TRANSMISSION

Stay curious. Question everything.

CONTINUE THE DESCENT

MORE FROM DIRTY STATIC ↘