Javascript Navigation Menu

Defining best practices in long term digital information retention

WHAT'S NEW?

The 100 Year Archive Task Force has several projects underway in which you can participate:

a. Help collect business requirements for long term retention by taking the online "Business and IT Requirements survey". We also need assistance in spreading the word. Forward this link to your 'customer' colleagues.

Register here to receive the final report.

b. We're currently reviewing a glossary of terms for inclusion in the "100 Year Archive Best Practices Guidelines" and SNIA dictionary. Please contribute. The working document is posted in the members area in the 100 Year Archive Task Force portal.

JOIN US!

We invite you to join us on our conference calls as we progress the work. Bi-monthly on Thursday at 3-4 PM ET. Dial 712-580-0600 - passcode 961226#.

Then, Join the DMF and participate further.

For more information, please contact Co-Chair Gary Zasman. We look forward to working with you!

The SNIA’s 100 Year Archive Task Force is a global, multi-agency group working to define best practices and storage standards for long term digital information retention. All interested parties and organizations are invited to join in this work effort.

Our Goals

  • Produce with a multi-disciplinary team a “best practices for long term digital information retention” paper similar to the Sedona project
  • Influence ILM as a core management and automation practice for long term archive
  • Guide the impact of new storage technologies such as XAM & Grid to improve long term retention methods
  • Define a standard for on-media formats & long term readability

Active Projects:

Documents and Presentations:

To Participate:

Are you a vendor, integrator or service provider involved in long-term archiving, or a governmental, RIM, IT, or regulatory compliance responsible professional? Join us and help shape industry standards and best practices as we define storage solutions to the challenges of long-term digital information retention.  Our charter includes educating customers so we also have an active speaker's bureau which you can access or participate in.

The Problem:

Although corporate and legal issues have recently brought data archiving to the light of day, the problems associated with preserving digital information are not new. Archiving for a few years is hard enough, but when requirements dictate that data be retained for longer, problems with media deterioration and technology obsolescence can seem insurmountable.
- Source: Galen Schreck - Forrester Aug. 2005

Corporate archiving systems tend to be very reliant on older servers and tape backup systems, both of which degrade with age. While the technology does exist to recovery drives and backup tape in clean room environments, hard drive recovery services are always considered an afterthought to most companies. This has to change, as companies discover the mortality of their backup media.
- Source: Maureen Davies - Hard Drive Recovery Group Aug. 2005

Long term preservation of digital content is a big challenge in the Information Society era, digital information in any form is at risk to be lost forever. Technology on which digital content relies becomes obsolete and application versions and files formats change, making data soon inaccessible. Even if content is coded in the simplest format, such as ASCII code, storage media degradation and obsolescence could make it disappear. Even on-line information such as web pages and databases, are vulnerable as much as their web structure become complex thanks to (aging) hyperlinks and cross references.
- Source: Alfredo M. Ronchi - Medici Framework

What is the state of best practices today?

Migrate: NARA says if information is on disk drives, migrate it every 3 years and if on tape, every 5 years. But, what about the ability to read and interpret the information?

Physically, long-term storage is not about media life (because migration is required) but, about periodic migration to newer media. But, how do you migrate a PB every year?  At some point migration becomes overwhelming.

Logically, long-term retention is about the ability to read the information and to be able to use it. Applications have relatively short life and rarely have the ability to read information older than a few revisions. While RAID arrays have helped provide a layer of security, these systems do fail, and prices for RAID recovery can be quite high. Even standard formats evolve, change, and become obsolete. At some point, information has to be migrated periodically to a new standard logical format.

Then what about issues such as compliance, integrity, and authenticity? How are you guaranteeing these over the retention period?

Standards and Resources (a few examples):

Copyright 2006, SNIA Data Management Forum. All rights reserved.