The Digital Tsunami: A Perspective on Data Storage: To Meet Demands, Organizations Will Need to Increase Today's Storage Offerings 10 Times. but How Will Such Dramatic Increases Be Addressed by Technology and Systems in the Next Few Years?

Article excerpt

At the Core

This article

* examines current issues and future possibilities in data storage

* compares alternative technologies for data storage

* identifies existing and emerging technologies for data storage

The rapid emergence of digital video is just one indicator of the growing markets for storage systems with much lower costs and much larger capacities. Other major demands come from the standard business modal for corporate data storage" and the growth of storage' service providers (SSPs) that are accumulating data from small business and consumer markets.

According to "How Much Information," a study by the University of California Berkeley School of Information Management and Systems, to meet escalating requirements, each of these markets will need to improve today's storage offerings by 10 times. By 2010, a 100-fold increase will likely be necessary, the study shows. As records and information managers must deal with increasingly larger volumes of records, keeping up with storage media trends is becoming more critical.

According to International Data Corp., in fiscal year 2003, Fortune 500 companies spent $7 billion on approximately 1,200 terabytes (1 terabyte = 1,000 gigabytes) of data stored on magnetic tape. Additionally, Jim Porter of Disk/Trend Inc. reports that companies have spent $40 billion to store 1,400 million terabytes of data on magnetic disk. Even though storage requirements have now started to increase at more than 100 percent annually, technology is improving in cost/performance at only 35 percent each year.

This creates a significant gap in terms of the cost/performance of storage systems required to meet the growing need. To allow that data to be stored without causing large cost increases, data storage must move from relatively expensive disks to lower-cost media. In addition, the cost/performance of these lower-cost media must continue to improve at an annual rate of nearly 100 percent to meet the continuing data growth.

This growth in data can be felt in personal terms also. In the eddy 1990s analysts were foretelling that each Fortune 500 company would have a terabyte under management in a few years. In 2000, experts such as Patti Tobin predicted that in 2003, thanks to the advent of digital movies on DVD, commercial music collections on CD-ROMs, and personal music or digital photographs stored on CD-Rs or CD-RWs, each computer-savvy home would have the equivalent of a terabyte of data under its roof. By 2006, as wireless and video technologies become ubiquitous, we should reasonably expect that individuals will move from having hundreds of gigabytes to a terabyte of personal storage on their desktop or laptop. The U.S. government--often a "canary in the mine" for new technology--has several data storage systems under development that will provide about 80 petabytes (80,000 terabytes) each.

This explosive growth of data, called the "Digital Tsunami" by Gary Ashton in his 1996 National Media Laboratory Report "Future Trends in Storage, Interconnectivity, and Data transfer," is fundamentally driven by the conversion of analog systems to fully digital systems (e.g., film cameras to digital cameras) and a leveling of computer resources available (e.g., each home computer is now equivalent in performance to early supercomputers).

But how will these dramatic increases ill data storage be addressed by technology and systems over the next several years?

Comparing the Media

The means of storing computer data has changed little over the past few decades and still consists of only about half a dozen different types of media and formats. These include silicon based dynamic and static random access memory (RAM), magnetic hard disk drives, optical disks (including write once read many [WORM], magneto/optic, CD-ROM, and DVD disk, and magnetic tape.

In any specific storage application, the preferred medium is selected because of a particular characteristic that is superior to that offered by the other options. …