BwDataArchiv FAQs

Technology

Q: What storage technologies do you use?

A: s. https://www.rda.kit.edu/technologie.php

Q: What ist HPSS?

A: HPSS is a data management application that is being developed at several computer centres that require long term storage for large amounts of data. See here for more detailed information: HPSS web site

Q: How is the data secured?

A: Data is stored on magnetic tape. We use the following tape drives and technologies: LTO5 (max. 1.5 TB per cartridge), STK 10kC (max. 4 TB per cartridge) and STK 10kD (max. 8 TB per cartridge), IBM TS1140 (max. 4.5 TB per cartridge)

Features

Q: I have a suggestion for improvement. What is the award if it gets implemented?

A: You will be named on the bwDataArchiv Hall of Fame pages and are eligible for 10 years of 1 TB of free storage.

Q: How many copies of the data are made and where are they stored?

A: all data in bwDataArchiv has at least 2 copies. Data is moved to disk and from there duplicated to tape storage. There are tape libraries in two data centres in CN as well as in CS.

Q: How long will the data remain in the archive?

A: The regular retention time for files on bwDataArchiv is ten years. After this ten year period bwDataArchiv will delete your data. A warning message is send 6 months ahead to the registered mail addresses. (This is probably the biggest reason to keep at least one of the two possible mail addresses up to date). Contact us at least three months in advance to have the retention time prolonged. If you want to terminate the cooperation with bwDataArchiv or if your data is no longer needed you can delete your data yourself.

Q: How can I make sure my data did not change. Do you support checksums?

A: We store a MD5 checksum for every file. When the file is read the checksum will be build again and compared with the stored checksum. If there is no match the file will not be delivered to the user. For detailed information s. https://www.rda.kit.edu/img/FAQ-bwDataArchiv%20Data%20Protection%20%20-%20V2.pdf
Also at a more basic level on disk and on tape the data is protected with checksums.

Q: There is a directory with the name .Trashcan in my archive directory. Can you explain me its function?

A: Files deleted from the archive are put in the Trashcan. Contents of the Trashcan are not deleted by the system and therefore are accounted and to the amount of data stored. To permanently delete the data you have to open the .Trashcan and delete the files. To recover files from the Trashcan move (or rename) the files to a different location outside the Trashcan.

Help

Q: I have a question. Who do I ask?

A: Support and help https://www.rda.kit.edu/english/65.php

Q: I did everything right. Still my client cannot access the archive. What could be wrong?

A: Please contact bwDataArchiv per E-Mail or, if you are a User from BW alternative via Baden-Württemberg Support Portal https://bw-support.scc.kit.edu/. Describe your problems and what you have done and add for example some screenshots.

User Registration

Q: Where do I register for the service?

A: visit https://www.rda.kit.edu/bwDA

Q: I have registered but still cannot access the service. What is wrong?

A: Maybe the registration workflow did not finish completely. This can happen because of network errors or unexpected browser behaviour. Go to https://bwidm.scc.kit.edu/user/index.xhtml, login with your credentials and unregister from the service. Then register again. You will receive an email after you have registered successfully.

Q: I lost my password

A: login using shibboleth (https://www.rda.kit.edu/bwDA/shibbLogin.php), click on 'manage your account' and change your password.

Q: why do I need a different password for the archive. Cant I use the one I use at my home - institution

A: the data will stay at least 10 years in the archive. By that time you may have left the organisation and your data is still there

Preparations for the usage

Checksums

Q: I want to routinely create and validate checksums of large amounts of files

A: This tool may be of help Hash build and check

Transfer Data

Q: What protocols do you support for uploading and downloading data to the archive

A: We support sftp and GridFTP for uploading and downloading to the archive. See https://www.rda.kit.edu/english/transmit.php

Read Data

Q: Accessing my data takes a long time. Why?

A: Long response time maybe due to several reasons:

• Has your data been stored a long time ago? Then it is probably no longer on disk and has to be copied in from tape. This may take up to several hours, depending on the current archive data traffic.
• Retrieval of lots of small files takes longer than of a few large files.
• Something is broken (but we are fixing it).

Delete Data

Q: I deleted [a file, some files, a directory, my files]. Can I recover the lost data?

A: Straight answer: No. Technical answer: maybe. Operational answer: it depends. Here is the deal: deleted data is kept around for sometime in a trashcan. However the current trashcan implementation has some limitations. Therefore don't count on a full rescue. Contact us and we will give our best to help.