The Advisory Council
Personalized Advisory Response
To: Paul Scott
RE: Data Retention Guidelines
Ref Number: 281
Date: Nov 3 2005 4:24PM
Response by: Beth Cohen
Question restatement: Hot Issue: How can I properly archive and manage 20 years of backup tapes? What are some of the best practices from the technical, legal and business perspectives?
Summary:

SmartSummary

  • Purpose of archives – How will the archives be used and what types of data are stored?
  • Value of data vs. cost of maintaining archive – Is it more costly to maintain the archive than the value of the data?
  • Types of archives – Are the tapes organized by department, project, year, machines, or something else? 
  • Tape formats – Do you have equipment that can still read all the tape formats?
  • Is the equipment properly tuned and maintained?
  • Tape storage – Have the tapes been stored in a climate controlled off-site environment?
  • Retrieval – Are procedures defined for access and retrieval of the data?

What seems like a simple request for information actually pinpoints a number of issues that you will need to look at to create a data archive management policy appropriate for your company.  As a privately held company, your business for the most part is not required to follow standard policies and best practices for records maintenance. However, many private companies find it convenient to create records management policies in line with Sarbanes-Oxley and other government-mandated guidelines. To address your immediate archive problem, you will need to approach the project from two directions.  First, you will need to develop a data management policy that sets the length of time that you will maintain various types of data records, what form they will take, and most importantly, operational procedures for retrieving the data from the archives.  Work with company management, the financial department and legal to set a realistic archiving policy – forever is not an option.  Sarbanes-Oxley and the IRS require seven years for financial records, so that is not a bad place to start from.  Include in the policy processes for reviewing the backup technologies themselves.  As the cost of near-line and on-line storage comes down, many companies are opting to maintain larger data sets on-line rather than archiving them off to tape or DVD (for smaller data sets).  

At the 50,000 foot level, data archives can be thought of as information that becomes increasingly less relevant over time.  Current financial records are highly valuable and must be fully protected, while15 year old records are of mostly historical interest and, unless there is a pending lawsuit, of little actual value to the on-going operations of your company.  At the same time as the content of the tape archives are aging, the tapes themselves are slowly deteriorating and the equipment and software to read the tapes is becoming less able to read them.  Even if you could retrieve the records from 18 year old nine-track reel-to-reel tapes, unless the records were maintained in ASCII textformat, you might not even have the application to read and interpret the data.  Over time, as the cost of retrieval rises,the value of the data drops.  At some point in time (which will vary on the type of data and company policy), the cost of maintaining the archives will exceed the value.  When that point is reached the tapes can be safely destroyed and discarded.

The first task in the creation of your data records archiving policy will be to take an inventory of your current set of records.  Identify all the different backup media and equipment for creating and retrieving records from the media.  Various kinds of tapes, DVDs, CDs, floppies,and optical drives have all been used as archive technology in the past 20 years.  If all the records are on asingle type of backup medium, the task is somewhat simplified, but it is important to confirm that old tapes are still readable by the equipment.  In the past I have had trouble with 8mm and 4mm DAT drives getting out of alignment over time and not being able to read tapes created on other drives, or in one case on the same drive, after it had been repaired and realigned.  Once you have identified your hardware and media, you need to review your backup and records management software.  Depending on how the archives were originally created, data retrieval might be easy or a nightmare.  For example, if the tapes were all created and labeled by machine name, and the machines had a mix of engineering documents, financial records, and random e-mail archives, then the ability to retrieve the data could well be seriously compromised.  If the archive tapes are organized by project, year, or some other more humanly meaningful way, then retrieval time could be significantly reduced.  Modern tape management and backup software such as Veritas Backup Exec tend to be more oriented toward creating tapes that will be used forsystem recovery rather than for archive purposes.  If you do not currently have an archive management system they do exist, but the software is quite different from the more common backup software.  Many of these systems are document management databases rather than tape management systems. A discussion of these types of systems for your archiving needs is,beyond the scope of the this inquiry.

Once your policies have been implemented, you can then cut your tape archive set to a more manageable and well managed number.  With proper tape management it is possible to significantly reduce off-site storage costs. Remember to test your retrieval procedures regularly.  Too many companies have found to their dismay that those great backup archives they were so carefully maintaining were completely blank or they couldn’t be read because the equipment was no longer functional or available.

SmartActions

Create an archiving policy based on the record keeping requirements of your company. Determine if you have the properly maintained equipment to read any old archives.  If you do not, and the tapes are older than the records policy requirements, look for cost savings from reducing the number and types of tape archives that are kept in controlled off-site storage.  Remember to test your archive retrieval procedures on a regular basis.

TAC SMART GRID
Alignment

Since Your company is privately held, your legal requirements for maintaining financial and other types of company records are considerable reduced. It is still important to create a proper data backup and storage policy for records archiving and management purposes. Your legal department should be included in the development of retention policies.
Cost Reduction

Off-site tape storage can be quite expensive, by creating a policy of maintaining only seven years of records and destroying tapes that are no longer retrievable, there is an opportunity to realize substantial cost savings in storage services.
Organization

From the organizational perspective, it is important to determine which records are vital to the organization and which records are likely to be accessed at some future date. Financial records are rarely required to be kept more than seven years for tax purposes, while personnel records might need to be kept for 10 years or more.
Technology

Archive and backup technology has transformed over the past 20 years. While tape archiving technology has remained relatively static, the cost of on-line and near-line storage has plummeted, meaning that vendors have focused on creating sophisticated disk and data management systems, in contrast to further development of tape backup. For companies like yours with large sets of older and possibly unreadable tape archives, this can translate for a need to maintain older equipment long past its useful life span. If your company is maintaining tape reading equipment for the sole purpose of reading archive tapes, there are additional opportunities to reduce maintenance overhead and equipment support costs, by creating a refresh policy for moving the archives onto more modern and denser media. The new technologies can hold up to a terabyte of data per tape.
Sources and Referrals: Database archiving versus backup: complementary best practices – July 2005 Storage Networking World Online article

The need for a Content Management Systems strategy – June 2005 Tech World article

The new buzzwords: Information lifecycle management – March 2003 Computerworld article

Steps for preserving the integrity of logdata – November 2005 Computerworld article

Information Lifecycle Management Initiative – ILM Initiative part of the SNIA industry standards group



©2002-2005 The Advisory Council, Inc. All rights reserved. Patent Pending Privacy Policy & Guidelines | Terms & Conditions