Recall wide, score narrow
Semantic search is good at recall and bad at exact names; BM25 is the opposite. Running BM25 only over the semantic candidates gets both without scoring 2.67M documents lexically on every query.
Company Synonym Agent
Entity resolution for company names across 2.67M company and subsidiary records: semantic recall, lexical rescoring, rule boosts, and an LLM plus a person only for the hard cases.
- Role
- Built
- Where
- Zinnov (Draup)
- When
- Oct 2025
- Access
- Work · private repo
Problem
The same company shows up as an acronym, a spelling variant, a legal-suffix variant, or one of its subsidiaries, depending on the dataset. That breaks joins and every SQL query built on them. Exact matching misses real matches; loose fuzzy matching merges companies that are different.
01
Clean the name, keep the acronym
Names are normalized and legal suffixes stripped. Multi-word names also get an acronym token, so a short form can match the full name.
02
Semantic recall over 2.67M records
All queries in a batch are embedded at once and searched against a FAISS index of about 2.67M company and subsidiary records. Candidates are over-fetched at ten times top-k, at least 50, so the lexical step has room to work.
03
Lexical rescoring, only where it matters
BM25 runs over the semantic candidates rather than the whole corpus, which keeps lexical scoring cheap at this scale. Ineligible records drop out, and the score is 0.6 semantic plus 0.4 normalized BM25.
04
Rules for what embeddings miss
Fixed boosts cover the cases embeddings get wrong: an exact clean match (+0.3), a prefix match (+0.15), the query equal to the candidate's acronym (+0.5), and the parent name contained in the candidate (+0.2).
05
Hierarchy-aware selection
A master record within 0.20 of the top score is promoted to first place, and a winning subsidiary is rolled up to its parent company.
06
Escalate only the hard cases
A score of 0.85 or more is accepted without any model call. Below that, an LLM decision agent chooses among the candidates, and when the top two stay within 0.05 of each other the pipeline pauses for a person.
Measured
- Company and subsidiary records indexed
- 2.67M
- Score accepted without an LLM call
- 0.85
- Semantic / BM25 score fusion
- 0.6 / 0.4
- Candidate over-fetch before rescoring
- 10×
- Tests in the extracted service
- 40+
How it's verified
- Harness
- Runs as a normalization tool inside the DRI-MAS graph, with an LLM decision agent and a human step behind it.
- Guardrails
- Scores of 0.85 or more skip the model entirely; the LLM can only choose among retrieved candidates, so it cannot invent a company; ties go to a person.
- Testing
- 14 unit tests for the hybrid normalizer and 40+ tests in the extracted service.
In depth
An escalation ladder
Cost and auditability go together: most names resolve with arithmetic, some need a model, a few need a person.
- Score ≥ 0.85: accepted, no model call
- Fuzzy or no match: an LLM chooses among the retrieved candidates
- Top two within 0.05: paused for a human decision
From library to service
The engine was later extracted into a standalone service (dense HNSW vectors plus sparse BM25 behind FastAPI), so the multi-agent app no longer loads the embedding stack and degrades to a stub when the service is down.
Decisions
- 01Semantic recall over the full corpus, BM25 only on the candidates: cheap lexical precision at 2.67M scale.
- 02Explicit acronym and parent boosts instead of hoping embeddings learn them.
- 03The LLM sees only below-threshold cases and only chooses among retrieved candidates, so it cannot invent a company.
Stack
Python · sentence-transformers (MiniLM) · FAISS · BM25 · RapidFuzz · LangGraph · Pydantic · Milvus · FastAPI