[Pasig-discuss] WORM (Write Once Read Many) AIPs

[Tim.Gollins at nrscotland.gov.uk]

Dear PASIG 

I have been thinking recently about the challenge of managing "physical"  AIPs on offline or near line storage and how to optimise or simplify the use of managed storage media in a tape based (robotic) Hierarchical Storage Management (HSM) system. By "physical" AIPs I mean that the actual structure of the AIP written to the storage system is sufficiently self-describing that even if the management or other elements of a DP system were to be lost to a disaster then the entire collection could be fully re-instated reliably from the stored AIPs alone. 

I have also been thinking about the huge benefits of adopting the concepts of "Minimal Ingest" (MI) and "Autonomous Preservation Tools" (APT) in a new Digital Archive solution. 

One of the potential effects of the MI and APT concepts is that over time it is clear that while (of course) the original bit streams will never need to be updated, the metadata packaged in the AIP will need to change relatively often (through the life of the AIP) . This is of course in addition to any new renderings of the bit streams produced for preservation purposes (manifestations as termed in some systems).

If to update the AIP the process involves the AIP being "loaded" and "Modified" and "Stored" again as a whole then this will result in significant "churn" of the offline or near line media (i.e. tapes) in a HSM - which I would like to avoid. I think it would be really great if the AIP representation could accommodate the concept of an "update IP" (perhaps UIP?) where the UIP contains a "delta" of the original AIP - the full AIP then being interpreted as the original as modified by a series of deltas. This would then effectively result in AIPs (and UIPs) becoming WORM objects with clear benefits that I perceive in managing their reliable and safe storage.

I am not sufficiently familiar with the detail of all the different AIP models or implementations, I was wondering if anyone in the team would be able to comment on whether the they know of any AIP models, specifications or implementations that  would support such a use case.

I have just posted a version of this question to the E-Ark Linked in Group so my apologies to those who see it twice. 

Many thanks 

Tim
Tim Gollins | Head of Digital Archiving and Director of the NRS Digital Preservation Programme
National Records of Scotland | West Register House | Edinburgh EH2 4DF
+ 44 (0)131 535 1431 / + 44 (0)7974 922614 | tim.gollins at nrscotland.gov.uk | www.nrscotland.gov.uk

Preserving the past | Recording the present | Informing the future
Follow us on Twitter: @NatRecordsScot | http://twitter.com/NatRecordsScot


**********************************************************************
This e-mail (and any files or other attachments transmitted with it) is intended solely for the attention of the addressee(s). Unauthorised use, disclosure, storage, copying or distribution of any part of this e-mail is not permitted. If you are not the intended recipient please destroy the email, remove any copies from your system and inform the sender immediately by return.

Communications with the Scottish Government may be monitored or recorded in order to secure the effective operation of the system and for other lawful purposes. The views or opinions contained within this e-mail may not necessarily reflect those of the Scottish Government.


Tha am post-d seo (agus faidhle neo ceanglan  còmhla ris) dhan neach neo luchd-ainmichte a-mhàin. Chan eil e ceadaichte a chleachdadh ann an dòigh sam bith, a’ toirt a-steach còraichean, foillseachadh neo sgaoileadh,  gun chead. Ma ’s e is gun d’fhuair sibh seo le gun fhiosd’, bu choir cur às dhan phost-d agus lethbhreac sam bith air an t-siostam agaibh, leig fios chun  neach a sgaoil am post-d  gun dàil.  

Dh’fhaodadh gum bi teachdaireachd sam bith bho Riaghaltas na h-Alba air a chlàradh neo air a sgrùdadh airson dearbhadh gu bheil an siostam ag obair gu h-èifeachdach neo airson adhbhar laghail eile. Dh’fhaodadh nach  eil beachdan anns a’ phost-d seo co-ionann ri beachdan Riaghaltas na h-Alba. 
**********************************************************************