Making data quality work for you

Inaccurate or inconsistent data can hinder companies’ ability to understand current and future business problems, often leading to poor decisions that can cause a host of negative results – including lost profits, operational delays, customer dissatisfaction and much more.

However, an effective data quality strategy can help management to better understand the business environment and allow companies to maximise profitability and reduce costly operational inefficiencies. So what do you need to know to make data quality work for you?
Pascal Clement, EMEA Vice President of Enterprise Information Management solutions at Business Objects, and Andrew Greenyer, VP of International Marketing at leading data optimisation solutions provider Group 1 Software, explain.

CXO. Data quality is fundamental to the promise of business intelligence (BI). So what challenges do companies face with regard to ensuring their data is up to scratch?

AG. Business intelligence applications usually have to draw data down from disparate legacy systems. These are often old, line-of-business applications in a variety of different languages and formats. Data has to be extracted and conditioned into consistent forms so that it can be combined and analysed effectively. Doing this without interfering with the smooth running of the line-of-business systems is a challenge in its own right. Obtaining actionable insights from the data analysis provides the next hurdle. Where data is drawn from other areas of the business – in-store, customer service, marketing, etc. – then the challenge is to institute strict data capture routines that ensure both accuracy and comprehensiveness.

PC. The ever-growing amount of data from various sources is proving to put quite a strain on businesses of all sizes. In the time it takes to resolve issues with one information source, many additional sources might have been added. And the global nature of today’s economy adds to the complexity, as data might come from different geographical regions with culturally diverse meaning. The merger and acquisition dynamics of today’s market pose yet another challenge as organisations struggle to merge systems and data to keep up with ongoing corporate change.

CXO. Traditional data quality tools are limited in their ability to address increasing data volumes and disparities. What solutions are emerging to help process this ever-growing mountain of information?

PC. Of course there’s the basic premise of sound software development practices, such as those found in our own data quality solution which was written to take full advantage of multi-processor, multi-threaded environments. Combine that with today’s evolution in chip design and the new multi-core CPUs hitting the market, and you’ve already come a long way towards answering these issues. We feel data virtualisation, such as that delivered by our BusinessObjects Data Federator XI solution, will dramatically increase the customer’s ability to handle very disparate and large datasets, but remove the need to physically consolidate all these sources. And of course, using our data quality solutions to do proper householding will allow clients to reduce the size of their data sources by eliminating the number of duplicate records existing today as a result of even the slightest differences in how data was entered into the systems.

AG. As we have already said, getting the information into a single consistent format is the main hurdle. This used to require hard-coded (expensive) interfaces and specialist programmers. Over the last few years, a number of solutions have hit the market that allow this extract-transform-load process (ETL) to be performed by IT-savvy business managers, and for routines to be saved and tweaked as our understanding of the usefulness of the data and its analysis becomes more refined. This then becomes far closer to the quick-reaction business strategy formulation that is demanded by the business and the market.

CXO. And what impact is this having on the ability of companies to transform data into actionable information? What results are being generated through the use of better data quality tools?

AG. Some of the benefits involve quicker reaction to market threat/change; just-in-time stock management and supply-chain management; the ability to spot and deal with customer service bottlenecks; and a greater ability to track customer revenue and profitability on an individual and segment basis.

PC. Increased data quality impacts all aspects of the business. Not only does it allow organisations to become more effective in the use of their corporate systems, it also has an important human impact on the user community. After all, when multimillion dollar investments in a customer relationship management (CRM) implementation or an enterprise resource planning (ERP) system are not being used because users do not trust the data they see in those systems, those investments will bring significantly less positive results than originally anticipated.

For example, call centre agents using their local excel versions of customer lists, because the central database does not contain correct information, will spend time setting up and maintaining those spreadsheets, making the overall value of those corporate systems go down even further. Marketing departments running campaigns to customers who have moved, disappeared, or are simply out-of-date in the system lose valuable funds and time, and destroy the anticipated return on investment of the campaign tool. In many cases, using incorrect data can be more destructive to a company than not using it at all. Just think of the negative feelings you create by continuing to address a client a Ms Wijn when it should be Mr Wijn. The first time is annoying, but the second or third time around, the client might start talking about this to his friends and colleagues, damaging your reputation as a ‘customer-centric’ company. Where will this client look for services or products next time? Chances are, not with you!

CXO. How is the changing regulatory environment impacting how corporations manage their data? What effect is this having on the (BI) market?

PC. Compliance to regulatory change is clearly driving data quality awareness. Companies are increasingly focused on data governance, and we see data managers, data stewards, data quality ‘centres of excellence’ appearing more rapidly since the Sarbanes-Oxley and Basel-II regulations were established. Especially important is the ability to create audit trails documenting how reported data came to live –What was the source? How was it moved through the processes? What happened along the way? Was the data changed? Augmented? – and creating these audit trails in a very user-friendly way. With system-wide metadata management, customers will be able to do this transparently.

This is also the driver behind the BI market consolidation, since the task of managing several disparate and unconnected systems has become increasingly burdensome to customers. The Business Objects enterprise information management (EIM) platform is uniquely positioned to respond to this issue, as it bundles data management, data quality, and data reporting tools into a single architecture and can put common metadata management across these various components.

AG. Data management is becoming more complex (and more expensive) as data protection and marketing permission restrictions have to be legally introduced into the system. Ironically, though, companies regard compliance as a must-do budgetary matter – yet the data management routines and disciplines introduced under the compliance banner can also be very beneficial for e.g. marketing, improving targeting and eliminating waste

CXO. What are your top five tips for taking the data quality risk out of business intelligence? What steps should an organisation take in order to ensure best practices?

AG. My top five tips would involve the following. First, improve data capture routines. Second, spend more on understanding what data leads to actionable insight, then concentrate on keeping that data up to scratch on quality. Next, introduce early test measures for insights drawn from BI analysis, so that they are validated before you are too far down the road. Fourth, look to achieve simple things, done well and scrupulously tested, rather than trying to be too clever. And finally, invest in using third-party data for enhancement and validation.

PC. First, think big. Data quality really doesn’t deliver its full potential unless it becomes part of your entire company. Second, start small. No company-wide initiative will succeed if you try to implement it all at once. Third, focus on the data. All organisations have their share of bad data, and improving that situation is far more important than trying to figure out ‘who was responsible’. Fourth, focus on the people. The best tools and intentions will not matter unless you have employee buy-in. They will need to understand and believe that better data quality will improve their productivity and performance.

Last but not least, focus on the process. Data quality is not a one-time exercise! By the time you’ve cleaned up your customer data, for example, many of those customers will have gotten married, divorced, moved, etc. and suddenly your list is out-of-date again. Implementing data quality is an iterative process in which you analyse, take action, and report on data; then you analyse, take action, and report on that data again; and so on. This is exactly why you need to make it a part of everyday business. Data quality is so much more than a tool you purchase to get rid of an issue.

We feel the need for best practices is best answered by centralisation, collaboration, and consistency. Companies will want to have a centralised data quality service, which allows for corporate control with local flexibility. This requires a service-oriented architecture where data quality becomes a ubiquitous corporate service, resulting in reduced total cost of ownership (TCO) as support of the environment becomes equally standardised.

Organisations will need to collaborate so people can work on data quality measures together, leveraging efforts done at a corporate level or another business unit in their own business. By identifying data quality champions – who act as coordinators in bringing business and IT users together in data quality project teams and can establish data quality standards and processes – a company can maximise on the effect of such collaborative efforts.

Lastly, the implementation of data quality needs to be consistent. By selecting a platform that can be connected to the various internal back-end systems and platforms, and that complies with industry standards such as SOA, XML, SOAP, etc., a company is choosing a consistent, enterprise-wide integration where users can select the most appropriate integration approach, yet link into a common data quality system.

CXO. Finally, what do you think will be the major drivers of the data quality management market over the next 18 months? What trends/developments are you getting excited and/or concerned about?

AG. I think there are a number of key drivers. We’ll see increasing M&A activity, with investors demand that databases are rapidly and effectively merged to extract value from the joint customer base. Another key driver will be compliance – there will be no quarter given for the inability to find the right records at the right time.

Finally, financial and marketing directors are increasingly using similar measures to judge the success of their activities – particularly return on customer (customer profitability). Data quality management is essential to these measurements, so there will be drive and cooperation between two CxOs who traditionally have been at loggerheads.

PC. Historically managing the quality of business information has been done as an afterthought. Companies were either in denial of their data quality problems, or didn’t believe that their problems were bad enough to warrant dedicated investment in data quality solutions. Early adopters of data quality largely came from organisations facing data problems in new enterprise applications like CRM or ERP systems or from companies that were struggling with data issues in their BI and data warehouse initiatives.

More recently we see a growing trend of companies recognising that high quality, actionable information can be leveraged as a competitive advantage. Increasingly, these forward-thinking companies are putting formal teams, strategies and best practices around managing their data as a corporate asset and ensuring that the quality of data is built in at source systems, not just in the back office. Trusted information is the holy grail of companies implementing enterprise information management strategy. Data quality ensures this data is accurate, complete, and timely.

Organisations are also increasingly working to gain greater insight into their customers, partners, and suppliers through customer data integration and master data management. Data quality software provides the ability to identify relationships and matching records and consolidate that information into a master data set. Likewise, building a comprehensive view of the business is especially important for companies engaging in mergers and acquisitions.

Compliance is one of the greatest and most recent drivers of data quality. There is no room for error as compliance violations can result in multi-million dollar fines and jail time for corporate officers. For companies required to meet compliance regulations such as Sarbanes-Oxley or Basel-II, data quality provides management confidence in the accuracy of the data driving compliance reporting.