Tool Search with Embeddings: Scaling Claude to Thousands of Tools

Why it matters

Built-in tool search can miss the vocabulary of your business. 3.1 asks whether a custom retriever earns its upkeep, and this notebook shows what owning one involves.

What to practise

Embed eight weather and finance tool definitions with a SentenceTransformer model, rank them by cosine similarity, and expose the ranking as a tool_search tool. Its handler returns {"type": "tool_reference", "tool_name": ...} blocks in a tool_result, under the advanced-tool-use-2025-11-20 beta header. Watch which tools each query surfaces and which one Claude then calls. Notice that the conversation loop sends the whole library in tools without deferring any of it.

How it fits

The exercise practises 3.1.3’s custom client-side search. Compare its tools list with 3.1.3’s configuration: deferral needs defer_loading: true on the discoverable definitions, so measure context savings only after adding it. Score discovery recall and selection separately, as 3.1.3 does.