Case study 02
Cross-provider entity resolution
Link confidently, ask when unsure, and explain every decision.
Matched the same business across inconsistent names and addresses from multiple commercial data providers, with conflict safeguards, a review path for ambiguous cases, and readable decision trails.
- Role
- Data Science Co-op
- Period
- 2026
- Status
- Shipped matcher · research prototype
01 — My contribution
I designed and built the shared normalizer and matcher, wired them into search and resolution, added a secondary search source with merged ranking, and built the review flow for ambiguous matches. Separately, I built and blind-tested a probabilistic record-linkage prototype.
02 — Problem and context
A customer searches for a business by name and location. Each provider returns its own candidates in its own format: legal suffixes come and go, streets are abbreviated differently, suites appear in one record and not another, and a branch can share an address with its parent.
Linking the wrong record is worse than linking none, because the customer would see information about a different business. The design goal was to link when the evidence is clear, ask when it is not, and leave a readable explanation either way.
From one search to one confident answer
Search
Name + location
Candidates
From several providers
Normalize
One neutral record shape
Compare
Field similarity + conflict rules
Link
Clear evidence
Review
Close or partial
No link
Conflict or weak name
User confirms
Server accepts only offered candidates
03 — Approach and key decisions
Decision 01
One normalizer for every provider
Each provider's record is projected into one neutral shape: name, street, city, state, and postal code. Text gets a readable form for searching and a collapsed key for comparison. Streets are parsed into components such as house number, road, and unit. The vocabulary of abbreviations and suffixes was generated from sample data and then reviewed by hand.
Decision 02
Standard similarity, field by field
Names are compared with edit-distance-based similarity, street names with character n-gram overlap, and location fields exactly. A different house number is a disagreement, not a near miss.
Decision 03
Some conflicts are logic, not similarity
A clear geographic conflict removes a candidate instead of lowering its score. A weak name match cannot be rescued by a matching address, which stops a different business at the same location from linking.
Decision 04
Ambiguity is a first-class outcome
Every candidate set ends in one of three outcomes: link, review, or no link. When two strong candidates are too close to call, the case goes to review instead of being decided by a hair.
Decision 05
Ask the user, validate on the server
When a match needs review, the user sees a short table of candidate records and confirms the right one. The server accepts only candidates it actually offered, exposes only opaque identifiers, and records the confirmation when the user proceeds.
Decision 06
Readable decision trails
Each decision carries a plain-language trail of what was compared and why it ended where it did. An internal debug view shows the same trail for diagnosing hard cases.
Decision 07
A secondary search source
When the primary source finds nothing, search falls back to a secondary provider. A “search again” path merges results from both and re-ranks them by closeness to the query.
Illustrative example — synthetic data
| Field | Search | Candidate A | Candidate B |
|---|---|---|---|
| Name | Juniper Ridge Bakery LLC | JUNIPER RIDGE BAKERY Agrees · suffix ignored | Juniper Ridge Catering Co Partial |
| Street | 120 N Example St Ste 4 | 120 North Example Street Agrees · unit absent | 120 N Example St Agrees |
| City | Sampleton | Sampleton Agrees | Sampleton Agrees |
| State | AZ | AZ Agrees | AZ Agrees |
| Outcome | — | Link | No link Same address, different business: an address cannot rescue a weak name. |
Normalized forms compared: juniper ridge bakery · {120, N, example, st, unit 4}
04 — Evaluation and evidence
The shipped matcher and the research prototype were evaluated differently, and the results of one do not carry over to the other.
- Prototype: blind evaluation
- For the probabilistic prototype I sampled businesses at random, collected candidates from several providers, and labelled the true links by hand after the model's predictions were locked, so labelling could not drift toward the model.
- Prototype: decision-level measurement
- Measured whether the final decision for each business was right, including whether it correctly declined to link, rather than only how well candidate pairs were ranked.
- Shipped matcher: tests and review
- Unit tests cover normalization, comparison, conflict handling, and result merging. Thresholds are hand-set starting values; no statistical accuracy result has been established for this implementation yet.
05 — What shipped
The shared normalizer, matcher, secondary-source search, and server-validated confirmation flow shipped in the product.
The probabilistic prototype remains unmerged research. Its evaluation informed the design but does not validate the shipped matcher.
- Status: Shipped
Deterministic matcher and search integration
Shared normalization and comparison across providers, with merged multi-source search.
- Status: Shipped
User confirmation for ambiguous matches
Review table with server-side validation of the user's choice.
- Status: Research · not merged
Probabilistic linkage prototype
Fellegi–Sunter-style weight table evaluated blind against hand-labelled links.
06 — Limitations and lessons
Limitations
- Hand-set thresholds are a starting point and need tuning against labelled outcomes.
- User confirmations will create data for future evaluation; a large labelled set does not exist yet.
- This is a deterministic matcher, not a trained model.
- Providers disagree about facts as well as formatting, so some cases genuinely need a person.
Lessons
- A “no link” answer needs as much design as a “link” answer.
- Locking predictions before labelling is cheap and removes a whole category of self-deception.
- Readable decision trails made hard cases much quicker to diagnose.
07 — Tools
- TypeScript
- React
- SQL
- Python
- String similarity
- Record linkage