In an age when enterprises (and individuals) are handling more and more information, managing this data efficiently and securely has become a challenge. And with every challenge, innovators can step up to the plate with technology that makes the process of storing, modifying, and extracting data almost painless, whether the database is a computerized library system, automated teller machine network, or a remote inventory system.
As database management system (DBMS) vendors continue to enhance their products, it's becoming more critical to provide the right information at the right time to the right people, according to Donald Feinberg, vice president and analyst for Stamford, Conn.-based Gartner, Inc.
Just take a look at the increased investments to see the growing importance of database management systems. The DBMS market grew 10.1 percent, from $8.7 billion in 2003 to $9.6 billion in 2004; and the relational database management system (RDBMS) market grew 10.2 percent from $7.1 billion to $7.8 billion in new license sales over the same time frame.
"This is greater growth than the overall IT budget increase worldwide for the same time frame," Feinberg said. "Companies are still buying new and additional DBMS engine licenses, and we believe this trend will continue during our visionary horizon of  years."
The expanded use of data warehouses across enterprises, according to Feinberg, is driving the growth. Atone time, only information managers and technicians used data warehouses. But now, marketing, accounting, and other departments across the enterprise are accessing data.
By 2007, Feinberg expects half of all data warehouses to use closed loop systems that are continuously updated: As business users access data from the warehouse and for their own needs, any steps they take (i.e., implementing a new marketing program) will be fed back immediately into the data warehouse source systems.
Although companies using data warehouses have already grown, smaller markets are expanding, said Feinberg. "By 2007, due to increased speed of hardware and software, small and midsize [business] implementations of OLTP [Online Transaction Processing] and data warehouses will begin to merge to a single database with shared schema. For large enterprises, the use of a single database for both OLTP and data warehouses will not be possible until after 2010."
As users become more comfortable with the systems, the complexity of the queries puts more of a strain on the system's resources than the actual number of queries, according to Feinberg. This means that companies often underestimate the processing power they will need for their data warehouse management systems.
As query complexity increases, the data warehouse structure must simplify the process and may require some data warehouse appliances for peak performance, according to Feinberg.
Using XML will be one way companies can simplify the data warehouse structure, according to Feinberg. He predicts that as major DBMS vendors add native XML and XQuery leading to XML data in the DBMS, XML will become the majority portion of the data in the warehouse by 2008. Feinberg recommends that companies start to move their XML data to the DBMS engine for reliability, availability, and security.
More Data in More Formats
Feinberg cautions that data warehouses are being asked to store increasing amounts of disparate data in an increasing variety of formats. This includes data such as photos, music, and scanned records and faxes, as well as other data with meaning to specific types of applications (i.e., medical records, security and biometric data, and geodetic data).
"As we move to store all company data in a single data store, standard search engines must be able to use this data in searching," said Feinberg. The database management system models of today will continue to be the standards for growth in the next few years. …