Long-term data storage: What's the best format?

Long-term data storage: What's the best format?

Serdar Yegulalp, Contributor
Nearly every company has to retain data for long-term storage of some kind, whether it's for their own records or for the sake of legal compliance. Usually, the size of the company and the available budget dictates which storage format they'll use. Larger companies go with tape or network-attached storage devices; smaller ones stick with burning data to CDs and DVDs.

But what's the best option if you have data that you want to still be accessible 15 to 20 years down the road?

There are many ways to store data, but for people who want to retain data for years at a time, there are four realistic options. With each format, the same precautions apply—such as making multiple copies of the same data, relying as little as possible on proprietary storage formats, and migrating incrementally whenever possible.

Hard storage

Many folks regard a well-engineered hard disk—i.e., an enterprise-class drive, not a consumer-level product—as a good way to record and preserve data for years at a time. With the cost per megabyte (or gigabyte, as the case may be now) of hard storage dropping continuously, it has become increasingly cost-effective to use hard drives in this manner. Once data is recorded to a given hard drive, it can be taken offline and then checked every so often in a relatively automatic way.

The main consideration with long-term storage is not always the longevity of the media itself, but whether the hardware or interface used for it

    Requires Free Membership to View

    Login

    By submitting your registration information to SearchWindowsServer.com you agree to receive email communications from TechTarget and TechTarget partners. We encourage you to read our Privacy Policy which contains important disclosures about how we collect and use your registration and other information. If you reside outside of the United States, by submitting this registration information you consent to having your personal data transferred to and processed in the United States. Your use of SearchWindowsServer.com is governed by our Terms of Use. You may contact us at webmaster@TechTarget.com.

will still be supported. Another consideration is how often you plan to perform "forward migration"—recopying the stored data to a newer version of the same media, or to an entirely new format. A good guideline for the scrupulous would be to check existing data once a year (or once every two years), and forward-migrate once every five or six years.

While it's unlikely that SCSI, IDE or SATA interfaces will soon become obsolete to such an extent that it becomes nearly impossible to find a controller to plug them into, it's worth recalling that in the early days of the PC era, no one thought that MFM/RLL controllers would become near-museum pieces within 15 years. Always keep at least one full set of hardware handy that can be used to access the devices in question.

Tape

Large companies routinely use low-tension linear tape formats like DLT for mass storage, since the relative low cost and longevity of the media make it ideal for such things. An open variant to DLT, called LTO, has emerged, and aside from being non-proprietary, the LTO spec itself contains forwards- and backwards-compatibility requirements. This way any given drive can read at least the previous two generations of LTO tapes made for it, which makes forward migration much easier and reduces the chances that a given tape won't be readable years down the road because the hardware isn't available.

CD/DVD

Burning data to CDs and DVDs has become the most common way to back up data by consumers and small businesses (aside from backing up one hard disk directly to another), and the broad and backwards-compatible support shown for CDs and DVDs doesn't show any sign of going away anytime soon.

Of course, the question about burning to disc is how long a given disc will last. That depends largely on the variety of media (i.e., what dye formulation used in the discs) and the storage conditions. The Digital Data Preservation Program provides detailed information about these issues, and makes specific recommendations for long-term care.

Magneto-optical discs

Magneto-optical storage (MO) is another format that's received attention as a way to store data reliably for a long time. In many European countries, it's a standard format for retaining medical-imaging data. Those who are not immediately familiar with it may balk at using it (if only because it's unfamiliar), but it has a solid reputation in the circles where it has found acceptance and consistent use.

About the author: Serdar Yegulalp is editor of the Windows Power Users Newsletter, which is devoted to hints, tips, tricks, news and goodies for Windows NT, Windows 2000 and Windows XP users and administrators. He has more than 10 years of Windows experience under his belt, and contributes regularly to SearchWinComputing.com and SearchSQLServer.com.

More information on this topic:


This was first published in September 2006

Join the conversationComment

Share
Comments

    Results

    Contribute to the conversation

    All fields are required. Comments will appear at the bottom of the article.

    Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.

    Back to top