With DataTrim Dupe Alert, Matching and Merging records is NOT limited to being a one-man’s Job.
By intelligently segmenting your data into logical parts, you can match and review these parts separately.
By involving multiple data-stewards and even end-users, you will improve the effectiveness of your cleaning process and increase your visibility to the cleaning initiative.

In this and 2 related blogs we will talk about:

1. Segmentation: How to break your data into logical segments, and how to create views for more effective reviewing and matching.
2. Permissions: Sharing Rules, Record Ownership, and how you provide review and merge processes respecting this.
3. Collaboration: Engaging the End-users in the Review and Merging process, or just allowing them to provide feedback on the duplicates.

Segmentation

If your database is large (100.000’s or millions), you cover multiple countries and regions, business units or in any other way have a database where the data, the quality and the way it is being used may vary from segment to segment, -applying the same matching and merging rules across the entire database will NEVER work.

Certain parts of your company may have had different rules for data entry, or no rules at all. Data from certain source system may have enforced mandatory fields to populated, and you have ended up with a lot of nonsense in these fields, or you may be sitting on a pile of contacts called: Accounts Payable etc.

Splitting the database into logical segments, and managing these with the respect for the local particularities is critical for any data quality initiative and also for the deduplication initiative.

But segmenting the data in the matching process may not be the only key to an effective cleaning process.
Will your team in the US be able to decide whether 2 German companies are duplicates?

Should the Marketing be doing the cleaning of leads only, or should they also cleaning up accounts and contacts, or should this be done by Sales – because you don’t think of having someone in IT doing it all, do you?

Engaging multiple users in your org, improves the overall end-user engagement, their ownership of the problem and at the end: it increases the quality of your data.

Using Filters to segment your data

In DataTrim Dupe Alerts, we separate the Deduplication process in 2 steps: a) Matching records and b) Reviewing and Merging

Matching Records is simple, because we have already built in best practice matching rules and algorithms from more than 12 years of doing this on a consulting basis for companies and organizations all over the world, but segmenting can be rather complex, but it doesn’t have to be.

Once you have your data segment defined and matched, you will then of cause need to orchestrate your reviewing process.

All potential duplicates should not necessarily be reviewed by the same person, so you will need to be able to provide each of these users (even if it is an end-user) with simple-to-use Views, where they have the necessary options to take appropriate action on the duplicates found during the matching process.

How To …

In DataTrim Dupe Alerts we operate with 2 sets of records in the matching process, a Secondary Data Set and a Master Data Set. During the matching these 2 sets are compared to each other in order to identify the potential duplicates, e.g. like when you match Leads against Contacts.

This also allows you to define 2 sets of records within an Object for your matching, e.g. matching your newly imported leads against the active leads already in the database.

And of cause it allows you to perform the simple process of matching one set of records against itself, just with a single click (Master = Secondary).

In DataTrim Dupe Alerts we support the entire querying language of the underlying salesforce database. This gives great flexibility, but it also imposes a strict syntax which need to be followed.

Simple filters and more complex filters can be created to identify subsets of records:

Filter	Description
Owner.Name = ‘Frank Scaramanga’	All records owned by a particular user
RecordType.Name = ‘Prospect’	All Records of a particular record type
Country = ‘USA’	By Geography
Country = ‘USA’ or Country = ‘Canada’	By Geography (multiple)
Name > ‘A’ AND Name < ‘F’	By Alphabetic grouping
Type IN (‘Prospect’, ‘Lead’, ‘Active’)	By Type (multiple)
LastModifiedDate = LAST_N_DAYS:1	By time period

The built in: Include New and Changed data since and Campaign Field, can be used in addition to the main filter field to select a specific set of records:

Image text: All records with Country = USA AND linked to the campaign: Dreamforce AND has been updated or added since 01/01/2016

Reviewing and Merging

Once an Alert has been processed it will generate an Alert Result Record, which is like a log record, with static data, providing you with details on what happened at the time of the matching.

The Potential Duplicates are all stored in the Matched Record Object, and it is from the DataTrim Matched Records Tab, that anyone who will be reviewing and merging the duplicates should initiate the process.

The package will install a number of default views. These are convenient to get started, but like anywhere else in salesforce the List Views are a basic way of selecting the set of records which you will work on, and naturally it also applies here.

Edit Filter (Lightning)	Edit Filter (Classic)

Make sure that your views include the 2 filters for Resolves and False Dupes, as you do not want to waste time browsing through potential duplicates which already have been reviewed in the past.

Edit Filter (Lightning)	Edit Filter (Classic)

Sharing is caring!