MDO Enterprise Data Platform / Setting up Data Quality Workbench

Setting up Data Quality Workbench

Data quality is the most essential attribute that contributes to the success of the business as it defines the accuracy of the data for the organization. Within MDO, data quality workbench provides a health check for the organization master data. It checks the data for duplicity, completeness, accuracy and validness. This work bench displays all the data rules running for the user data. Admin can set-up data quality checks from a pre-defined set of rules or create their own rules and stay on the top of its important data.

Schema Creation

Data Quality Workbench can be defined as MDO’s Data Health Analytics framework. This framework also provides the functionality of transformation to standardize the data. MDO provides the ease of creating a schema, which is a consolidation of the rules and validations for a data object. Any data set associated to the object can be checked for quality by running the schema(s) for the object. MDO provides an easy interface to create a schema and define rules within it.

To create a new schema:

  1. Login to MDO with valid credentials.
  2. Click on Settings icon in the top right corner à Click on the desired object à Click on Business Rules tab at the top.
  3. Click on Add New.
  4. Fill in the details according to the table below.
  5. Click Save.
Field name Details
Category Select a valid category that seems the most appropriate for the schema.
Name define the name of the schema
Owner Type Choose the owner type based on
User – Schema updates would be conveyed to a user
(If selection is user, then select the user(s) in the Select User field)
Role – Schema updates would be conveyed to all users assigned to a role
(If selection is Role, then select the role(s) in the Select Role field)
Group – Schema updates would be conveyed to a group
(If selection is Group, then select the group(s) in the Select Group field)
Plantcode Select the company code
Success Enter the desired label for Successfully processed records
Error Enter the desired label for unsuccessful records
Total Enter the desired label for Total records
Business Rules Setup

To configure business rules within the schema:

  1. Select “Detail” against the schema.
  2. Select “Add Business Rule”.
  3. Select the Business Rule type and select Next.
  4. Based on the Rule type follow the instructions below:
Rule Type Instructions
Dependency 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In reference field, select the existing dependency (refer to section 1.3) between source and target field.

4.      Select Finish.

API 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In reference field, select the existing API.

4.      In the message field, add a message that would appear if the record is not in accordance with the API.

5.      Select Finish.

Duplicacy Check 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In reference field, select the existing Duplicacy Check (refer to section 1.4) Rule.

4.      In the message field, add a message that would appear if the record is a duplicate.

5.      Select Finish.

Metadata Rule 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In Field, select the field(s) that need to be assigned metadata rule.

4.      Select Finish.

Missing Rule 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In Field, select the field(s) that need to be assigned missing rule.

4.      In the message field, add a message that would appear if the record is missing.

5.      Select Finish.

Regex 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In Field, select the field(s) that need to be assigned regex rule.

4.      In the Regular Expressions, add the desired Regular expression to define the rule.

5.      In the Message field, add the message that should appear if the Regex rule fails.

6.      Select Finish.

User Defined Rule 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In Field, select the field(s) that need to be assigned the user defined rule.

4.      In Select Pseudo Code (refer to section 1.5), select a user defined rule (if already exists)

5.      If the pseudo code already exists, then the custom script would get auto-populated. If not, then enter the custom script.

6.      In the Message field, add the message that should appear if the User defined rule fails.

7.      Select Finish.

Lookup Table 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      In Table, select the table that need to be assigned lookup rule.

4.      In the message field, add a message that would appear if the Lookup table rule fails.

5.      Select Finish.

Query Rule 1.      Enter the name of the rule in Description.

2.      Select the preferred category for the rule.

3.      Select the field(s) that need to be assigned the query rule.

4.      In Query Expressions, input the SQL query.

5.      In the message field, add a message that would appear if the query rule fails.

6.      Select Finish.

Create a Dependency

To configure a dependency within an object:

  1. Login to MDO with valid credentials.
  2. Click on Settings icon in the top right corner à Click on the desired object à Click on Business Rules tab at the top.
  3. Select Dependency tab
  4. Select Add dependency
  5. Enter the description of the combination
  6. Enter the company code.
  7. Drag the fields at the source fields to define the source fields and drag the fields at the target fields to define the target fields for the dependency.
  8. Select Save.
 Create a Duplicacy Check

To configure a dependency within an object:

  1. Login to MDO with valid credentials.
  2. Click on Settings icon in the top right corner à Click on the desired object à Click on Business Rules tab at the top.
  3. Select Duplicacy Check tab
  4. Select Add Rule
  5. Select the field(s) for which Duplicacy check needs to be created.
  6. Select the Duplicacy criteria – Fuzzy Match or Exact Match.
  7. If the criterion is a Fuzzy Match, then check inverse if the inverse of the record also needs to be considered a match.
  8. If there are no Exclusions to the rule, go to step 15
  9. Check Exclusions, if there needs to be exclusions for the rule.
  10. Select Show library
  11. Enter word(s) that need to be ignored in “Ignore Words” and select add for each word.
  12. Select “Similar Words” tab and enter the synonyms for specific words. Select Add for each pair.
  13. Check “Add to Standard” if a pair needs to be added in the standard library.
  14. Select Finish. The Exclusions window will close.
  15. Enter the weightage in such a manner that the total weightage for all the fields equals 100.
  16. Select Next to go to Selection Criteria
  17. Select the field(s) for which the selection criteria need to be defined.
  18. For each field, the selections can be as below:
Pick from the Record Values will be checked from the set of records
Custom Values will be checked only for custom records.
In case of custom, enter the custom word.
  1. Select Next to Define Master Settings. These settings help define a master record from a set of duplicate records.
  2. Select rules according to preference by clicking on the ‘+’ symbol in front of the rule.
  3. After adding rule, select the preferred field accordingly within each rule.
  4. Select Finish.
  5. Once the rule populates on the Duplicacy check main tab, Select ‘+’ symbol on the levels and select ‘Add’.
  6. Define the percentage range and select the action for each range with a message.
  7. Select Save.
Create a User Defined Rule

To configure a dependency within an object:

  1. Login to MDO with valid credentials.
  2. Click on Settings icon in the top right corner à Click on the desired object à Click on Business Rules tab at the top.
  3. Select User Defined Rule tab
  4. Select Add New Rule
  5. Select either Active Governance or Passive Governance according to the rule.
  6. Select the field for which the rule needs to be set (in case of active governance) or write the description of the rule (in case of passive governance) and select save.

In case of Active Governance,

  1. Select Create on the right-hand side of the page.
  2. Select the JavaScript event.
  3. Choose either JavaScript or Template
  4. Enter company code
  5. Select Finish
  6. To modify the code, select code on the right-hand side of the page
  7. Write the code and select compile. If the code is compiled, select save.
  8. To activate the rule, check Active.

In case of Passive Governance,

  1. Select the field, operator and the comparison metric for each block
  2. Add another block by selecting ‘+’ on the left-hand side of the block and select the field, operator and comparison metric for this block.
  3. Repeat step c for all the blocks.
  4. When the condition blocks are complete, select generate.
  5. The code would be generated and the rule would be added.

Watch Video: