Necessity of Data Quality Analysis
Ø Without guarantee of the quality of source data, one can not estimate the needed workload and time in data cleaning, nor the resulting cost and profit of data cleaning, hence impossible to prioritize data cleaning work.
Ø Without guarantee of the quality of data, one can not determine the effect of dirty data on the reliability of decision-making, hence impossible to identify the scope of data users and data owners.
Ø If the data in a data warehouse are dirty and inaccurate, query results and statements provided to users will be wrong. And if users use the query results and statements, they may likely make wrong decisions. Before long they would lose confidence in the data warehouse and refuse to use it anymore.
The Effect of Data Quality on BI System
Data Quality Analysis Methodology
|
|
First Step |
Second Step |
Third Step |
Fourth Step |
Fifth Step |
|
Determine Scope and Strategy |
Analysis on Needs |
Define Data Quality Benchmarks |
Data Quality Testing |
Feedback. |
|
Major Task |
Determine the Scope of Source Data in Analysis, Analyze Rules, and Determine Analysis Steps and Tools to be Used |
Deep Understanding of Business Needs and Business Analysis Models. Delineate Data Elements subject to Data Quality Analysis. |
Work with Business Experts and Project Directors to Determine the Data Quality Benchmarks to Meet for each Data Element |
Test Data Quality. List out Conversion Rules for Data Fail to Meet Data Quality Benchmarks |
Communication with Project Director on the Status and Progress of Current Data Quality Analysis |
|
Output |
Table of Data Quality Analysis Steps, Table of Analysis Rules |
Table of Data Elements subject to Data Quality Analysis |
List out the Required Data Quality Benchmarks in Detail in the Table of Data Elements |
Table of Test Output |
Make Adjustment and Amendment on the Portion Affecting Project Progress |
Basic process of Data Quality Analysis Strategy
Solution to Business Problems
Enhancing data Quality: After data quality analysis, data analysts shall submit a data quality status report, and shall work with ETL designers, data management personnel, business experts, and users to determine the degree of cleanness for data elements and to come up with rules on cleaning dirty data.