Truth bomb #1: A Gartner survey found that 60% of companies weren’t aware that bad data had impacted their bottom line as they weren’t measuring it in the first place. Poor data quality led to 15 million dollars in losses per year.
Don’t you think it’s time we took a hard look into data quality?
In simple terms, data quality refers to the status of your data measured against several criteria. The criteria may vary depending on your business requirements. We’re highlighting the main ones here:
- Validity: The degree to which your data conforms to the defined business rules and validations
- Accuracy: Closeness of data to the true values
- Uniqueness: No duplicity of records
- Completeness: The extent to which all required data is available
- Consistency: How consistent the data appears within data sets or across different data sets
- Timeliness: Whether the data is up-to-date or not
Do you have data quality issues?
Now that we’ve got the definition out of the way, how does one determine whether their company has data quality issues?
Truth bomb #2: YOU are best suited to answer this question. You can tell right away by identifying some tell-tale signs in your day-to-day work situations. We’re just giving some common examples below.
More time (and resources) than necessary to collect data
When you asked for reports involving consolidation of data and numbers from various departments, people tend to ask for additional time and the help of other personnel to prepare them. Don’t mistake it for incompetence or lack of motivation.
When data quality isn’t already embedded, the work doesn’t just involve collecting data. They’ll have to manually check for data duplicates and errors, correct them, and do final review before presenting it to you.
General lack of trust in data
When your top management have to review reports or dashboards, they’d either partially or fully disregard them, citing low confidence level in the underlying data. Instead, they resort to asking the opinions of others, seeking past precedents, or even using ‘gut feel’ in driving decision-making which should be based on data.
Conflicting numbers
Ever been in a situation where you’re shown data from different departments that claim to answer your one single question, but contain different figures altogether? While there may be some level of data validation, it seems each unit has a different way of doing it. The same goes for data sourcing and collection methods. This makes your life harder, as you have no way to tell which ones are correct.
At this stage, you may feel like you’ve opened a can of worms—and there’s no turning back from it!
What’s next?
Now it’s clear that you can’t escape data quality, let’s look into ways to address it.
Truth bomb #3: You can’t achieve data quality overnight.
There needs to be a process transformation, with your people getting onboarded all throughout the change phases. And let’s not forget that you need to have a robust tool or software to automate the data quality tasks.
Process
First, you’ll need to define what data quality means to best serve your business. This makes up a part of the larger picture of your data governance process. The aim is to have a uniform and standardized data quality approach across the organization.
It doesn’t have to meet all the criteria of validity, accuracy, completeness, etc. If all you need is data timeliness and validity, then the focus should be on setting up and implementing business rules with a system that supports real-time data.
Of course, some business units may have unique data quality requirements. Finance, for instance, would need to have completeness too as they’d need to combine data from multiple units for their consolidated financial reporting.
So, the process and policies should be made flexible enough to accommodate additional data quality requirements and document the steps or rules adequately to avoid confusion or misinterpretation.
Well, easier said than done. Things will get complicated as data quality requirements become more elegant and differentiated.
This is where your people have to step up.
People
For your people to appreciate the importance of data quality, you need to instill ownership and accountability amongst them. Start by appointing data owners from different business units. They should play an active role in the conception of data quality criteria, what measures to use, and if there are unique treatments to accommodate specific requirements.
Data stewards should be responsible for the implementation and execution of the data quality rules and policies. They ensure business users follow specified data standards and governance. This also includes escalating recurring issues to data owners, especially those warranting different data quality approaches.
And what would be the driving force behind all this? Your leadership team, of course! They need to champion the usage of data and by extension, the importance of data quality. People need to see first-hand that the bosses use data and data analytics in everything they do to drive the point home that data quality matters.
Most importantly, leaders should be able to remove barriers that hinder the implementation and enforcement of data quality.
Software
As you dive into the nitty-gritty of details, you’ll realize that data quality tasks are a huge endeavor. Cleansing existing data, checking new data, and monitoring data status are among the next steps once the data quality policies are put in place. If these have to be done manually, your people might view this as an added burden on top of their existing tasks. You could risk losing their support!
This is where technology comes in to automate and expedite the more mundane and repetitive tasks. Data quality solutions can be used to process and validate large amounts of data against the agreed-upon business rules, flag and remediate errors, and monitor data health. Some software tools may have the added functionality to execute large-scale data cleansing and scrubbing.
Your data personnel can then set these tasks on ‘auto-pilot’ so they can dedicate more time and effort towards embedding data quality awareness, usage, and ownership amongst their peers.
How MDO Can Help
Choosing the right technology solution is mission-critical in ensuring data quality initiatives get lifted off the ground and don’t get abandoned halfway.
With MDO, you’ll get a complete solution suite encompassing data quality, cleansing, and governance. Key features include:
- Pre-defined business rules, templates, and workflows
- Governance and data quality rules based on industry and domain
- Self-service capabilities for data stewards/business users to deploy data quality metrics
- Data preparation and cleansing capabilities for migration projects
- Data integration combining multiple source systems
- Data enrichment based on content and machine learning models
By operationalizing data quality, your people will start to trust data and use it more. This paves the way for frictionless collaboration across departments to produce beneficial outcomes for your organization.
Written by: Shigim Yusof
