Archival services

The development of large scale archive services now underway.

Within the coming years it is planned to enhance the LSDF with:

• An Easy-to-use interface for deposit and update
• Access to files via persistent URLs. Collections can be openly accessible or available to administrators only.
• Support for specific file types and raw data
• Permanent storage with tools for long-term management

Archival Projects

Several projects have started to make this happen. Research for large scale data management at KIT is contributing to the development of efficient and secure long time storage of petabytes of data. In the programme Supercomputing and Big Data of the Helmholtz Association the close collaboration with a multiple scientific communities resulted in clear requirements for future data management including the need to archive big datasets along with tools for provenance and curation.

bwDataArchiv

Focus of the project bwDataArchiv funded by the state of Baden-Wuerttemberg, is the installation and deployment of the High Performance Storage System (HPSS). The result is a reliable yet economical mass storage system with interfaces for users and programmed tools. First users will be HLRS to store expedited projects, the LSDF counterpart in Heidelberg and the GridKa LHC Tier1 center.

RADAR

The DFG project ‘Research Data Repository’ RADAR aims at deploying and establishing an infrastructure for both scientific data archiving and scientific data publication to support various research areas in scientific data management. Within the project a first low level approach will be developed which addresses basic archive functionalities. Later on, the offer is to be extended in order to provide an archive system which is adaptable to specific research areas and includes scientific data publication services. The project will use the infrastructue developed in the bwDataArchiv project.

Research and development

What else? The long-term vision is a standardization of preservation services and their application programming interfaces (APIs). The implementation of these services is not ready but many such services, or component services which can be brought together to produce the required results, already exist at the LSDF.

In addition we seek to guarantee that trust in the quality of the services is quantified using reproducible preservation metrics.

Please turn back to this page for regular updates

(last edit 19.02.2014)