[Pasig-discuss] Experience with Bagit?

[Matthew Addis]

Hi Robin,

We use bagit at scale within our archiving service (following the spec but using our own implementation), e.g. we recently bagged up and stored approx 100M files totalling 2PB into something like 20,000 bags for one of our customers.

We also have some experience of bagit through Archivematica and know that it can produce AIPs in bagit format that can cope with 100,000 files in a bag and that single bags can be as large as 1TB in either 7zip or uncompressed format without problems using the python implementation.

Happy to talk more about this off-list if you’re interested.

Cheers,

Matthew

Matthew Addis
Chief Technology Officer

tel:   +44 1249 405060
mob:     +44 7703 393374
email:     matthew.addis at arkivum.com<mailto:matthew.addis at arkivum.com>
web:       www.arkivum.com<http://www.arkivum.com/>
twitter: @arkivum

This message is confidential unless otherwise stated.
Arkivum Limited is registered in England and Wales, company number 7530353. Registered Office: 24 Cornhill, London, EC3V 3ND, United Kingdom

From: Pasig-discuss <pasig-discuss-bounces at asis.org<mailto:pasig-discuss-bounces at asis.org>> on behalf of RICE Robin <R.Rice at ed.ac.uk<mailto:R.Rice at ed.ac.uk>>
Date: Friday, 20 January 2017 14:42
To: "pasig-discuss at mail.asis.org<mailto:pasig-discuss at mail.asis.org>" <pasig-discuss at mail.asis.org<mailto:pasig-discuss at mail.asis.org>>
Subject: [Pasig-discuss] Experience with Bagit?

Hi,

If anyone has experience of using Bagit with large and voluminous files (e.g. research data) could you please get in touch? We have some questions about performance, etc.

Cheers,

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Robin Rice
Data Librarian
EDINA and Data Library
University of Edinburgh

R.Rice at ed.ac.uk<mailto:R.Rice at ed.ac.uk>
@sparrowbarley (twitter)
#RDMSmooc -
www.coursera.org/learn/data-management
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/pasig-discuss/attachments/20170120/e0c9b17a/attachment.html>