Archive insight: The digital preservation world moves on

These files were generated as part of a project to digitise a large portion of the 1” magnetic video tape from our core Television South West collection that was begun in 2010, and ran for 24 months or so. The project was called Strength in Numbers and involved several other regional Archives. The archives swapped copies of their data so that each held a full set of data belonging to the others: Strength in Numbers. Two network attached storage (NAS) devices, have held our local data since then, and we’ve made use of the files on a daily basis for supply to customers for a wide variety of programmes and screenings.

Time moved on and the NAS devices were well past their best before date (they have been likened to ‘Trigger’s broom’ - all the parts have been replaced so many times, they’re no longer the same thing), and the files that were created used a lossless compression that is no longer recognised as a preservation format for video files.

We laid out a plan to transcode our files to FFV1, a losslessly compressed preservation standard format for moving image files. We currently use uncompressed formats for capturing our film and video collections. This is all well and good but the file sizes can be huge – digitising half an hour of film for example, requires over half a terabyte of storage. When you multiply that by three for our three preservation copies, it can quickly get unwieldy and expensive.

Lossless formats, like FFV1, allow us to compress the data in a way that is completely reversible, something essential for preservation purposes, but results in typical files sizes one third of the original uncompressed. These files are easier to move around, reduce our spend on digital storage and reduce the energy required to run that storage.

We ran some tests to ensure that our files from 2010 could be losslessly transcoded to FFV1 and, once happy, we cooked up some batch files using ffmpeg 'recipes' to perform the transcode, generate viewing copies of the files, and to generate reports to confirm the lossless transcode that is so essential in the preservation of video files. Then, some bespoke Python scripts to deal with the file handling to ensure everything was packaged up in to the correct folders and verified.

Two of our powerful computer workstations crunched through the data during the evenings, churned out our lovely new preservation files and logs and placed them gently on our Xendata digital archive storage system.

What is to become of those NAS devices? After a long a faithful service, they are due to be retired and have a well earned rest. There will be a small ceremony, a giving of thanks and perhaps a Viking burial - or more likely an environmentally responsible disposal.

So, whether someone is after a copy of the 1980s Saturday morning children’s television show Freeze Frame, or, as we predict, ‘That’s My Dog’ is re-broadcast and achieves cult status, the digital preservation world move(age)s on and the legacy of that Strength in Numbers project is preserved for the next generation.

With thanks to:
James Gibbs, Ben Bullman, Mike Brewis and Stacey Anderson
The Box’s Engagement Programmes Manager for securing the funding for this project
The British Film Institute for guidance and advice around the use of ffmpeg and FFV1

Further reading:
https://kk.org/thetechnium/movage/ https://en.wikipedia.org/wiki/FFV1 https://ffmpeg.org/ffmpeg.html