Grounding agents with IDs instead of names

An agent asked about a company will answer. Whether the answer is about the company you meant depends entirely on what the retrieval step was keyed on.

The eval

1,200 questions about group structure, status and register facts, over companies chosen to include renames, same-name pairs and cross-border groups — the cases that separate the setups rather than the ones they all pass.

Why the middle row fails the way it does

Name-matched retrieval fails silently. Ask about “Nordwind GmbH” and it retrieves filings for several of the 41 companies with that name, merges them into one context, and produces a fluent answer describing a company that does not exist. The failure is not a missing document; it is a confident synthesis of documents about different entities.

Retrieval that cannot tell two companies apart does not return less information. It returns a company that never existed.

What the ID changes

• Resolution happens once, in a tool with a calibrated confidence and an explicit no_match, rather than implicitly in a similarity search that always returns something.
• Ambiguity becomes visible: review with 10 alternatives is a state the agent can surface, and a name-keyed index has no way to express it.
• Every downstream call is keyed on an ID, so nothing later in the chain can drift to a different company.
• lineage gives a citation the grader can verify — the source document either contains the claimed fact or it does not.

Four things to put in your own eval

1. Grade on entity_id, not on the name string.
2. Include renamed companies. This is 8% of real traffic and close to 0% of most eval sets.
3. Include same-name pairs in different communes. This is where a name-keyed system produces its most confident errors.
4. Include questions with no answer, and check that the agent says so. An agent that cannot abstain will invent.

Wiring, key scoping and the tool call order are in MCP & agents.