MDO Enterprise Data Platform / Data Quality / Setting Up Data Quality Rules

Setting Up Data Quality Rules

  1. Missing rule 

Usage 

Is used to apply a check on data entered in a field(s) that has a missing value(s). 

For e.g., Previously, Name was not a mandatory field but now it is required as mandatory. In this scenario, the Missing Rule will help identify the data which has Name missing. 

Steps to apply the missing rule 

Once Schema is created, named, and threshold scaled: 

  • Click on the “Add Business Rule” button and select “Missing Rule” as a rule type. 
  • Select weightage % 
  • Select category 
  • Entre rule name / description 
  • Type the error message to be displayed 
  • Select the field where the rule will be applied 

2. Metadata rule 

Is used to apply checks to a record based on the inherent properties of the fields involved, such as data type, length, predefined formats, etc.  

For e.g. If the Name field has been set to 25 characters, the Metadata Rule would ensure that if a name entered exceeds 25 characters, an error message would be shown.   

Steps to add Metadata rule:  

  1. Enter the name of the rule in Description. 
  2. Select the preferred category for the rule. 
  3. In Field, select the field(s) that need to be assigned metadata rule. 
  4. Select Finish.

3. Regex rule 

Is used to check the accuracy of a field’s value through a regular expression. 

For e.g. If the Name field should have the first character as the upper case then the field Name would be mentioned followed by the Regex:   

^(?:[A-Z!@#$%^&*-.,;:()/1234567890][^\s]*\s?)+$   

Steps to apply Regex Rule 

  1.  Enter the name of the rule in Description.  
  2. Select the preferred category for the rule. 
  3. In Field, select the field(s) that need to be assigned the regex rule. 
  4. In the Regular Expressions, add the desired Regular expression to define the rule. 
  5. In the Message field, add the message that should appear if the Regex rule fails. 
  6. Select Finish.

4. User-defined rule 

Is used to configure custom rules on a field or set of fields. It can be a single or a combination of set of rules. 

For e.g. When ordering parts, you can default the processing time for all “ABC123” parts to 8 days, except if the delivery location is “NSW” the Processing time would be “10” days. 

Steps to add UDR rule 

  1. Enter the name of the rule in Description. 
  2. Select the preferred category for the rule. 
  3. In Field, select the field(s) that need to be assigned the user-defined rule. 
  4. In Select Pseudo Code, select a user-defined rule (if it already exists) 
  5. If the pseudo code already exists, then the custom script would get auto-populated. If not, then enter the custom script. 
  6. In the Message field, add the message that should appear if the User-defined rule fails. 
  7. Select Finish.

5. Lookup Rule 

This rule can help apply validation scenarios across multiple datasets or within the same dataset based on the concept of the primary key. The lookup rule can be configured within the dataset, SAP table lookup, or SAP check code. The user can then select any of these to look up and join two datasets to fetch required records or apply validation scenarios using the following conditions. 

“When” condition 

Allow the user to join tables by providing a “When” condition between the primary key of both datasets. 

Henceforth, the user can define the condition for success for a record. 

The user can define a sub-rule here to scope the data. 

“Then” condition 

Allow the user to define the condition once the condition of success has been defined in the when condition. 

in the “Then” condition user can define a result count greater than 0 to fetch records. 

The user can define a sub-rule here to scope the data more.