Deduplication

Data deduplication occurs when a similar piece of data is recorded into the system. MDO has an inbuilt tool which helps in checking the duplication of data. The tool requires the rules to be defined for when duplication needs to be checked. MDO manages the duplicate data by highlighting the duplicated records and eliminating them. Duplication check can be configured for an exact match or through fuzzy logic. MDO also allows the end user to define the percentage of duplication in the record and what action can be done, whether it would permit creating the record with duplicate values or stop the user from creating the record. 

For example:  

The field Name can be added to be checked for duplicates. There are options of adding exclusions, selecting whether it is an exact match or an approximate match check and checking inverse as well 

After selecting the field(s), a selection criterion can be set. For example, if the name “John Cleese” exists in countries USA and Australia and we do not require the same name in different countries to be identified as duplicates, we can add country to the selection criteria. Custom countries can also be added as desired.

After the Selection Criteria, Master Settings can be configured. These ensure that a master record can be selected from a set of duplicates through a rule and the remainder of the duplicate records discarded. 

In the example below, for customer module Name 1 is configured for identifying duplicates. 

When a user tries to create a customer record with the name ‘Duke Energy’ which already exists, it shows a pop-up window highlighting the duplicate record and the fields which have duplicate values.  

For more information on setting up duplicacy checksplease refer to the link below. 

Duplicacy Check