Data Management Problems Widespread: Organizations Should Regard Data as Their Greatest Asset-And Invest in Data Management Accordingly

Article excerpt

A recently published Virginia Commonwealth University (VCU) study--the first comprehensive look at how data management is practiced worldwide--reveals that most organizations do not manage information well.

And there's more: despite organizations recognizing that data is the lifeblood of their businesses, "Measuring Data Management Practice Maturity: A Community's Self-Assessment" also concludes that businesses face significant data management challenges.

The study, published in the April issue of Computer, the publication of the Institute of Electrical and Electronic Engineers Computer Society, assessed the data management practices of 175 public- and private-sector organizations between 2000 and 2006. A vast majority scored far too low for their own good.

According to lead study author Peter Aiken, associate professor in VCU's Department of Information Systems and the founding director and owner of data management consulting firm Data Blueprint, fewer than 10 percent of the organizations studied are using documented processes to manage data, which means that more than 90 percent are ineffective in this area.

Data Management Often Neglected

All organizations depend on data, and good data management practices are critical to many technology-based organizational initiatives, including business intelligence, customer relationship management, and data warehousing. Bad, incomplete, or inaccurate information has been the downfall of projects, departments, and even entire organizations.

To wit, the 2006 InformationWeek article "Hamstrung by Defective Data" cited a 2005 Gartner estimate that more than 25 percent of critical data within large businesses is somehow inaccurate or incomplete. In late 2005, the Data Warehousing Institute surveyed 750 IT professionals and business executives, and 53 percent of them said their companies had experienced problems and suffered losses or increased costs because of poor-quality data, up from 44 percent in a similar 2001 survey. (Alarmingly, 36 percent admitted they had not even studied the issue.)

Although data is a critical asset for organizations, the VCU study is an indication that organizations don't fully appreciate its value.

Aiken and other VCU researchers found that many businesses do not invest adequately in data management, treating data as a maintenance cost rather than as an asset. According to the study, that is a mistake that "is costly in terms of market share, profit, strategic opportunity, stock price, and so on."

The study's results are based on self-reporting by the organizations involved; approximately 15 percent of the organizations also participated in an in-person investigation by researchers to validate the self-assessments. Researchers tried to measure not only whether a data process was performed in an organization, but also the maturity with which the process was performed.

This is important because, as increasing amounts of data flow within and between organizations, the problems that can result from poor data management practices are becoming more common--and more serious. Studies cited by VCU research have shown that such poor practices are widespread. For example:

* PricewaterhouseCoopers reported that in 2004, only one in three organizations was highly confident in its own data, and only 18 percent were confident in data received from other organizations. Further, just two in five companies have a documented board-approved data strategy.

* According to Aiken, approximately two-thirds of organizational data managers have had formal data management training; slightly more than two-thirds of organizations use or plan to use formal metadata management techniques; and slightly less than one-half manage their metadata with computer-aided software engineering tools and repository technologies.

This is not good news, especially considering that information is only getting more difficult to manage. …