Clean Your CRM Before 2027: An AI Dedupe Day

Every CRM drifts. Someone books a call from their work email, then fills in a web form from Gmail six months later. A receptionist types "Bob Chan" and a sales rep types "Robert Chan, ABC Plumbing". By December you've got two, three, sometimes four records for the same person, and each one holds only part of the history.

If you run a small business in Burnaby or New Westminster, one focused "dedupe day" before 2027 is about the cheapest cleanup you can do. AI can help. It just isn't the first tool you should reach for, and it should never be the one pressing the merge button.

Why bother before the new year

The practical reason is simple. Duplicates split your history. Reps miss the last quote, reminders fire twice, and your reports count one lead as three.

The legal reason is quieter, but it's real. BC's Personal Information Protection Act says an organization "must make a reasonable effort to ensure that personal information collected by or on behalf of the organization is accurate and complete" when that information is likely to be used to make a decision about the person or disclosed to another organization (BC PIPA, s. 33). Federally, PIPEDA's accuracy principle asks you to keep personal information "as accurate, complete and up to date as necessary, taking into account its use and the interests of the individual" (OPC, PIPEDA Principle 6).

The OPC's practical steps include documenting where related information is stored and recording when it was obtained or last updated. A dedupe day is a natural moment to do exactly that. Which statute governs you depends on what you do and where your data goes, so check with your privacy advisor if you're not sure.

Start with what your CRM already does

Before you pay for anything with AI in the name, look at the tools you already have. Here's what the vendor docs said when I checked in October 2026:

• HubSpot, all plans: contacts are automatically deduplicated on the Email property and companies on the Company domain name, according to HubSpot's deduplication article. That's exact matching, nothing clever.
• HubSpot Professional and Enterprise: the manage duplicates tool (Marketing, Sales, Service, Data Hub and Smart CRM tiers) compares property values daily. For contacts it looks at fields like first and last name, email, phone, zip code and company name (HubSpot Knowledge Base, updated September 25, 2026). It shows up to 10,000 duplicate pairs as standard, and Data Hub raises that to 30,000 on Professional and 100,000 on Enterprise.
• Microsoft Dynamics 365 and Dataverse: model driven apps come with default duplicate detection rules for accounts and contacts. Rules use operators such as Exact Match, Same First Characters and Same Last Characters, and you can only publish five rules per base record type at a time (Microsoft Learn).

One line in HubSpot's docs matters more than the rest: merges can't be undone. You can roll back a rejected duplicate pair within 14 days if no properties changed, but a merge is permanent. Plan around that.

Where AI actually earns its place

Rules are great at "same email" and bad at "same human". A rule won't connect "Bob Chan, [email protected]" with "Robert Chan, ABC Plumbing and Heating, [email protected]". That fuzzy middle ground is where a language model helps.

There's decent research behind this. In Entity Matching using Large Language Models (Peeters, Steiner and Bizer, arXiv, latest revision October 2024), the authors report that the best LLMs "require no or only a few training examples to perform comparably to PLMs that were fine-tuned using thousands of examples." They also found LLM matchers held up better on entities they hadn't seen before, and that GPT-4 could explain its match decisions in a structured way.

The same paper also gives you a reason for caution. The authors found no single best prompt. Each model and dataset combination needed its own prompting. And those results come from benchmark datasets, not your customer list. So test any AI matcher on a sample of your own records before you trust it.

You'll notice I'm not recommending a specific model or quoting a per token price. Those change every few months. Pick the model when you set the job up, using the vendor's current pricing page.

A one day plan

Here's an illustrative schedule for a CRM with a few thousand contacts. Adjust to taste.

Morning: back everything up. Export contacts, companies and deals in full. Since HubSpot merges are permanent, this export is your only real undo.

Late morning: run the built in tools. Merge the obvious exact matches, like identical emails. Most of the easy wins live here, and they cost nothing extra.

Midday: an AI pass on the leftovers. Narrow the candidates with something cheap first, like a shared phone number or a similar company name. Then send each candidate pair to a model and ask for one of three labels: match, no match or unsure, with a one line reason. Don't ship your whole database. Send only the fields the model needs. That lines up with the OPC's generative AI principles, which ask organizations to limit personal information to what's necessary for the purpose and to put safeguards in place that match its sensitivity (OPC, 2023). Read your AI vendor's data handling terms before you start.

Afternoon: a human reviews every match. Someone who actually knows your customers approves each merge. The same OPC principles are clear that accountability for decisions stays with the organization, not with the automated system supporting it. Spot check a handful of the "no match" labels too.

End of day: write it down. Record what you merged, which rule or label triggered it, and the date. Then add prevention: make key fields unique, and have your web forms check for an existing email before creating a new record.

To show the shape of the workload, here's a purely illustrative example. Say the built in tools flag 400 pairs and the AI pass marks another 150 as likely matches. Your reviewer's afternoon is those 150 judgment calls, while the 400 exact matches mostly take care of themselves. These numbers are made up. Yours will differ.

Where this doesn't apply

AI dedupe isn't always worth it. Skip it or slow down if:

You have a few hundred records. Do it by hand over coffee. Setting up an AI pass would cost more time than it saves.

Your intake is broken. If nobody owns the CRM and fields aren't standardized, duplicates will be back by spring. Fix your forms and data entry first.

The data is sensitive. Health details or personal financial information deserve real thought before any record goes to a third party AI service. Get advice first.

Records look alike but aren't. Two family members sharing one email, or franchise locations sharing a domain. Rules and models both get these wrong. A person has to decide.

You're tempted to bulk delete. Merging and deleting are different decisions. BC PIPA s. 35 says you should destroy personal information once it no longer serves its purpose and isn't needed for legal or business reasons, but if you used it to make a decision that directly affects someone, you must keep it for at least a year (BC PIPA).

You expect a turnkey tool. LLM entity matching works well in research, but the prompting is still fragile. Treat it as an assistant for your reviewer, not a replacement.

Sources

• Province of British Columbia, Personal Information Protection Act, SBC 2003, c. 63, ss. 33 and 35. bclaws.gov.bc.ca
• Office of the Privacy Commissioner of Canada, PIPEDA Fair Information Principle 6: Accuracy. priv.gc.ca
• Office of the Privacy Commissioner of Canada, Principles for Responsible, Trustworthy and Privacy-Protective Generative AI Technologies, 2023. priv.gc.ca
• HubSpot, Deduplication of records, HubSpot Knowledge Base, accessed October 2026. knowledge.hubspot.com
• HubSpot, Manage duplicate records, HubSpot Knowledge Base, updated 2026. knowledge.hubspot.com
• Microsoft, Set up duplicate detection rules to keep your data clean, Microsoft Learn, Power Platform documentation, 2025. learn.microsoft.com
• Peeters, R., Steiner, A. and Bizer, C., Entity Matching using Large Language Models, arXiv:2310.11244, 2023 (revised 2024). arxiv.org

Want a hand planning your own dedupe day? Autana Solutions works with businesses across Metro Vancouver to set up practical AI that keeps your CRM clean after the cleanup, too. Book a free call and we'll look at your data together.