Select retrieval that preserves answer correctness

Learning objective

Match retrieval methods to the pattern: lexical or metadata filters for exact identifiers, semantic search for paraphrase, generated SQL for aggregates, graph traversal for relationships.

Core idea

Choose a retrieval method whose operation matches the answer’s correctness requirement. An exact identifier needs exact eligibility, a paraphrase needs meaning-based discovery, an aggregate needs computation over the defined population, and a relationship needs connected entities. A more capable model cannot reconstruct a record that the retriever excluded.

The architect owns the route and its limits, while source owners approve keys, measures, and relationships. Claude can select or parameterize retrieval tools, but your executor must enforce the scope in which those tools operate. The classification in 3.6.1 becomes an executable retrieval choice here; relevance tuning follows in 3.6.3.

How it works

Preserve identity with lexical search and filters

Use lexical retrieval for identifiers or rare technical terms inside text. Anthropic documents BM25 as a lexical ranking method and contrasts its exact error-code matching with semantic retrieval that can miss the code. It is appropriate when the sought identifier occurs in manuals or narrative records rather than in a dedicated keyed service.

Distinguish ranking from equality. BM25 orders candidates by lexical relevance; the application should still verify that the returned passage refers to the requested identifier. Tokenization and searchable field configuration belong to your search implementation. If the source has an authoritative identifier field, an equality predicate or direct keyed lookup makes eligibility explicit before relevance ranking.

For example, an application-defined lookup_product tool can accept product_id, and its executor can apply WHERE product_id = ? with a bound value. A document search can filter by the same validated identifier in metadata. These names are your interface, not built-in Anthropic retrieval fields. Tenant and access scope must come from the authenticated request, not from an arbitrary model-generated filter; see 3.2.2.

Missing and ambiguous identifiers need distinct outcomes. Do not silently replace an absent key with its nearest semantic neighbor. Return a missing-record result or request clarification if several records satisfy an incomplete key. Broader retrieval can help find candidates, but it must not silently change which identity the answer claims to describe.

Use semantic search for paraphrases

Embedding-based retrieval represents the question and source passages as vectors and retrieves similar content. It suits a request such as “customers lose access after changing their payment card” when the relevant documentation says “subscription entitlement reconciliation.” Shared meaning can recover evidence that exact word overlap misses.

Keep hard constraints outside that semantic judgment. If the question concerns a particular product, customer scope, or approved document set, filter eligibility in the retrieval implementation and rank within it. A strong similarity score does not establish that the passage applies to the requested population.

Use semantic results as evidence for explanation, not as a complete database result. If a conceptual question also contains an exact code, combine methods deliberately; see 3.6.3. Neither replacing semantic retrieval entirely with keywords nor using semantic proximity as identity serves mixed traffic well.

Generate SQL when the answer is a computation

For aggregates, Claude can translate a natural-language question into SQL using a supplied database schema. Anthropic’s text-to-SQL cookbook demonstrates joins, aggregations, and examples that teach the mapping from business wording to database operations. The database computes the result; Claude explains it.

Provide the relevant schema and business definitions before accepting generated SQL. A syntactically valid query can count duplicate join rows, use the wrong date column, or average the wrong population. Review the generated operation against the counting unit and filters defined in 3.6.1. Limiting the number of returned rows is not a substitute for defining the population over which an aggregate runs.

When the schema is too large to supply whole, the text-to-SQL cookbook retrieves query-relevant schema information from a vector store before generating SQL. Assign the database integration owner responsibility for that schema index, including updates when tables, columns, or relationships change. Evaluate required-table, column, and join coverage alongside executed-query correctness: an omitted schema element leaves Claude without that part of the database definition.

Separate value binding from query validation. Python’s sqlite3 documentation specifies placeholders rather than string formatting for binding values. That protects the bound values; it does not make an arbitrary generated statement safe. For a SQLite prototype, opening the URI with mode=ro and uri=True prevents writes, and Connection.set_authorizer() can deny operations through SQLITE_DENY. These are Python/SQLite controls, not Claude options. Apply source-specific authorization in the deployed database integration.

Choose fixed, parameterized queries when the supported business questions are bounded. Generated SQL earns its flexibility when legitimate analytical questions vary beyond that catalog. Assign an owner to its schema context, approved operations, query validation, and workload limits. Do not give every natural-language request the same unrestricted database execution path.

Traverse relationships instead of assuming them

Use graph traversal when the answer depends on typed connections: which suppliers connect to a recalled component, which systems depend on a service, or which people connect through a project. Anthropic’s knowledge-graph cookbook resolves entity aliases, creates directed relationships with source documents, and serializes a bounded neighborhood for Claude.

Its serialize_subgraph(center, hops=2) example expands through predecessor and successor nodes and returns relationship triples. The hop count is a cookbook choice, not a universal retrieval default. For a production dependency query, specify direction, permitted relationship types, and traversal depth according to the question. An undirected neighborhood may include facts that do not establish the required causal or dependency path.

Entity resolution is part of correctness. Under-merging aliases disconnects related facts; over-merging distinct entities invents paths. The cookbook explicitly identifies dropped names and over-merging as resolver failure modes. Preserve relationship provenance so an answer can be checked against the underlying records. A graph built from extracted text is not automatically authoritative merely because traversal is deterministic.

Relational joins can also answer known relationships. Choose graph representation when the connected structure and traversal requirements justify it; do not migrate a well-defined two-table join solely because an exam stem says “multi-hop.”

Key distinctions and trade-offs

The method choices differ in what they establish:

Flexibility adds ownership. Generated queries require a governed execution surface; extracted graphs require relationship and resolver quality checks. Keep each route narrow enough that its failures can be explained and repaired without rebuilding every retrieval path.

Architecture / decision scenario

A retailer’s internal product assistant must answer within five seconds and serve buyers across separate business units. A proposal sends product codes, paraphrased complaints, sales totals, and supplier-impact questions to the same semantic index. A trial yields 95% correct complaint explanations but 72% correct sales totals; these are hypothetical measurements.

The architect chooses lexical retrieval for product-code mentions, backed by exact product_id eligibility, and semantic retrieval for paraphrased complaints. Generated SQL handles sales aggregation against the approved sales schema, with database enforcement of business-unit scope. The finance owner approves the definition of net sales and the date basis before the result is treated as a business metric.

Supplier-impact questions traverse the component-to-product-to-supplier relationships. The catalog owner validates aliases and edge provenance. If a required edge is missing, the response identifies the unresolved relationship instead of substituting a similarly named supplier. A known direct relationship can use a relational join; an exploratory dependency path can use the graph.

The tempting alternative is to raise retrieval depth and let Claude count and connect the passages. That preserves the same missing guarantees. The architect instead accepts separate integrations because each makes the required identity, computation, or relationship inspectable within the five-second budget.

Common exam traps

• Treating BM25’s highest-ranked hit as an equality guarantee.
• Using semantic proximity to resolve exact identity without verification.
• Counting or summing a retrieved sample as the complete population.
• Treating placeholders as validation of the whole generated SQL statement.
• Giving generated SQL unrestricted execution because it came from Claude.
• Assuming deterministic traversal repairs incorrect entity resolution.
• Replacing ordinary joins with a graph without a relationship requirement that warrants it.

What to remember

Match the operation to the answer: exact eligibility for identifiers, semantic discovery for paraphrases, database computation for aggregates, and verified paths for relationships. Preserve source ownership and executor controls; model-generated retrieval arguments do not supply either.

Knowledge check

References

• Anthropic, Contextual Retrieval
• Anthropic, Text to SQL with Claude
• Anthropic, Knowledge Graph Construction with Claude
• Anthropic, Define tools
• Python Software Foundation, sqlite3 — DB-API 2.0 interface for SQLite databases