How to Evaluate a Third-Party Business Identity Graph Before You Use It

The question usually comes up after a vendor demo. The match rate looked great, the sample file looked clean, and someone on the marketing ops team wants to know how to check before importing 200,000 records into the automation platform. That instinct is right. Bad identity data in a marketing automation tool is expensive to undo: bounced sends, damaged sender reputation, wrong-person outreach, and records that get merged with the wrong accounts.

The same steps apply to HCP identity graphs that link NPIs to emails, devices, or practice locations. The healthcare-specific version is in how to evaluate an identity graph for healthcare media.

Best practices for evaluating identity graph accuracy

Here is the sequence I would follow. Each step is cheap compared to cleaning up a bad import.

1. Write down what you need the graph to do (find emails, link people to companies, resolve duplicates, add devices).
2. Build a truth set from records you can verify.
3. Send the vendor inputs only, blind to the answers.
4. Score match rate and accuracy separately, by segment.
5. Check recency and how the vendor handles job changes.
6. Review how links are built: deterministic or probabilistic, and from what sources.
7. Pilot a small segment with a holdout before full load.

Build a truth set first

A truth set is a sample where you know the correct answer. Good sources for a business graph include:

• CRM contacts verified in the last few months through a sales call, meeting, or reply
• Event registrants who used a work email and confirmed attendance
• Customers whose employer and title you confirmed in a contract
• For HCPs: NPIs with practice addresses confirmed against the NPPES registry and recent field rep contact

Keep the truth set separate from anything the vendor has seen. If you send them the full CRM and then test on part of it, they may simply give your own data back.

Size matters. If you care about results by company size, industry, or specialty, you need enough records in each group to see a difference. A few hundred per key segment is a sensible starting point.

Match rate vs accuracy

This is the distinction most evaluations miss. Match rate counts how many input records came back with an answer. Accuracy counts how many of those answers were right.

A hypothetical comparison: Vendor A matches 80 percent of 1,000 truth-set records, and 600 of those 800 links are correct. Precision is 75 percent, correct coverage is 60 percent. Vendor B matches 65 percent, and 598 of those 650 links are correct. Precision is 92 percent, correct coverage is about 60 percent. Same useful yield, but Vendor A would load 200 wrong records into your system for every 1,000 sent, against 52 for Vendor B.

The existing article on why HCP audience match rates differ across partners explains how vendors reach such different numbers.

Check recency and coverage by segment

Business identity decays. People change jobs, companies rebrand, emails get deactivated. Physicians move practices or join health systems. Ask the vendor:

• When was each record last observed or verified, and can you see that date per record?
• How do you detect a job change, and how fast does the graph update?
• What share of records have not been seen in 12 months?

Then split your accuracy results by segment. Graphs are often strong for large companies and weak for small ones, or strong in some specialties and thin in others. An overall precision of 85 percent can hide a segment at 60 percent, and that segment may be the one you care about most. See data freshness in healthcare audiences for more on how old is too old.

Ask how the links are built

Deterministic links come from a direct shared identifier, like a verified email login. Probabilistic links are inferred from patterns such as IP address, device behavior, or co-occurrence. Both have uses. You should know which you are getting, at the record level if possible, and be able to filter to deterministic only. The piece on deterministic vs. probabilistic HCP identity covers the tradeoffs.

Ask about sources and permissions too. Where did the data come from, what notice or consent applies, and can you use it for email outreach under your own policies and applicable law? A graph that is accurate but cannot be used for your purpose is not useful. This is not legal advice; involve your privacy team.

Design a pilot before full load

A file test tells you about accuracy. A pilot tells you about consequences. Keep it small and measurable:

• Load one segment, a few thousand records, into a separate list or program
• Hold out a random portion so you can compare against a group that did not receive outreach
• Track hard bounces, spam complaints, "wrong person" replies, unsubscribes, and engagement
• Do not merge pilot records with existing CRM records until the results are in

Set pass and fail thresholds before the pilot starts. If bounces or wrong-person replies run well above your current baseline, stop and go back to the vendor.

Practical takeaway

Pull 1,000 contacts from your CRM that were verified in the last six months, strip the answers, and send the inputs to each vendor you are considering. Score precision and correct coverage, not match rate, and do it by segment. That one test will separate vendors faster than any demo.

Editorial note. Analysis and frameworks are the author's own and do not represent Acxiom or any current or former employer, client, or named platform. Examples labeled hypothetical or illustrative are not results from real campaigns. Nothing here is legal, regulatory, or medical advice.