Category:LSDF Online Storage: Difference between revisions

From Lsdf
Jump to navigationJump to search
No edit summary
No edit summary
 
(15 intermediate revisions by 4 users not shown)
Line 4: Line 4:
[[File:Lsdf_online_storage_services_2.jpg|right|top|thumb|250px|LSDF Online Storage @ KIT (SCC)]]
[[File:Lsdf_online_storage_services_2.jpg|right|top|thumb|250px|LSDF Online Storage @ KIT (SCC)]]


The LSDF is a central storage system for large research data from data-intensive scientific disciplines. It is available to KIT researchers, their partners in the federal state and the Helmholtz Association as a multi-petabyte storage system. The service LSDF Online Storage enables KIT users and their partners in the state of Baden-Württemberg to access the storage resources of the LSDF. The service is used by institutes with research funded by the Helmholtz Association as well as insitutes at the university level of the KIT from different scientific disciplines. The project is sponsored and funded by the Ministry of Science, Research and the Arts of Baden-Württemberg (MWK), the German Research Foundation (DFG) and the Helmholtz Association of German Research Centres. It is operated by the Steinbuch Centre for Computing (SCC) at KIT.
The LSDF is a central storage system for large research data from data-intensive scientific disciplines. It is available to KIT researchers, their partners in the federal state and the Helmholtz Association as a multi-petabyte storage system. The service LSDF Online Storage enables KIT users and their partners in the state of Baden-Württemberg to access the storage resources of the LSDF. The service is used by institutes with research funded by the Helmholtz Association as well as insitutes at the university level of the KIT from different scientific disciplines. The project is sponsored and funded by the Ministry of Science, Research and the Arts of Baden-Württemberg (MWK), the German Research Foundation (DFG) and the Helmholtz Association of German Research Centres. It is operated by the Scientific Computing Center (SCC) at KIT.


The LSDF enables scientists to store and access their research data via different services. These services, tailored to the requirements of the researchers, are offered on top of the LSDF storage. Important features are in particular high storage capacity, fast access to and transfer of the data, easy data access and management capabilities, as well as the connection to modern processing infrastructures such as the high-performance computing clusters based at KIT like the bwUniCluster or the ForHLR II.
The LSDF enables scientists to store and access their research data via different services. These services, tailored to the requirements of the researchers, are offered on top of the LSDF storage. Important features are in particular high storage capacity, fast access to and transfer of the data, easy data access and management capabilities, as well as the connection to modern processing infrastructures such as the high-performance computing clusters based at KIT like the bwUniCluster or the ForHLR II.
Line 27: Line 27:
| style="color:#000; padding:2px 5px;" |
| style="color:#000; padding:2px 5px;" |
* [https://bwidm.scc.kit.edu/user/register-service.xhtml?ssn=lsdf Signing up to exising storage projects] (for users)
* [https://bwidm.scc.kit.edu/user/register-service.xhtml?ssn=lsdf Signing up to exising storage projects] (for users)
* [https://www.lsdf.kit.edu/os/storagerequest/ Requesting new storage projects] (for project admins)
* [https://www.lsdf.kit.edu/os/storageprojects/new/ Requesting new storage projects] (for project admins)
* [http://www.scc.kit.edu/downloads/sdm/Nutzungsbedingungen-LSDF-Online-Storage.pdf Terms of use] (German version)
* [https://www.scc.kit.edu/downloads/SDM/Nutzungsbedingungen-LSDF-Online-Storage.pdf Terms of use] (German)
* [https://www.scc.kit.edu/downloads/SDM/Terms-of-Use-LSDF-Online-Storage.pdf Terms of use] (English)
* [[LSDF_Online_Storage:_Acknowledgements|Acknowledgements]]
|-
|-
<!-- Documentation -->
<!-- Documentation -->
Line 34: Line 36:
|-
|-
| style="color:#000; padding:2px 5px;" |
| style="color:#000; padding:2px 5px;" |
* [http://www.scc.kit.edu/downloads/sdm/Nutzerhandbuch-LSDF-Online-Storage.pdf User manual] (German version)
* [http://www.scc.kit.edu/downloads/SDM/Nutzerhandbuch-LSDF-Online-Storage.pdf User manual] (German)
* [https://www.scc.kit.edu/downloads/SDM/User-Manual-LSDF-Online-Storage.pdf User manual] (English)
|-
|-
<!-- Support -->
<!-- Support -->
Line 42: Line 45:
* [https://bw-support.scc.kit.edu/?mode=ticket_submit&respounit=LSDF%20Online%20Storage Baden Württemberg Support Portal]
* [https://bw-support.scc.kit.edu/?mode=ticket_submit&respounit=LSDF%20Online%20Storage Baden Württemberg Support Portal]
|-
|-
<!-- Web sites
<!-- Web sites -->
| style="padding:2px;" | <div style="margin:3px; background:#cef2e0; font-size:120%; font-weight:bold; border:1px solid #BBBBBB; text-align:left; color:#000; padding:0.2em 0.4em;">Web sites</div>
| style="padding:2px;" | <div style="margin:3px; background:#cef2e0; font-size:120%; font-weight:bold; border:1px solid #BBBBBB; text-align:left; color:#000; padding:0.2em 0.4em;">Web sites</div>
|-
|-
| style="color:#000; padding:2px 5px;" |
| style="color:#000; padding:2px 5px;" |
* [http://www.scc.kit.edu/forschung/11843.php Large Scale Data Facility (LSDF)] (German version)
* [http://www.scc.kit.edu/dienste/11228.php LSDF Online Storage] (German)
* [http://www.scc.kit.edu/en/research/11843.php Large Scale Data Facility (LSDF)] (English version)
* [http://www.scc.kit.edu/en/services/11228.php LSDF Online Storage] (English)
* [http://www.scc.kit.edu/dienste/11228.php LSDF Online Storage] (German version)
* [https://www.scc.kit.edu/dienste/17035.php Data services - selection according to application scenarios] (German)
* [http://www.scc.kit.edu/en/services/11228.php LSDF Online Storage] (English version)
* [https://www.scc.kit.edu/en/services/17035.php Data services - selection according to application scenarios] (English)
* [https://www.scc.kit.edu/services/17037.php Data services - selection according to specifications] (German)
* [https://www.scc.kit.edu/en/services/17037.php Data services - selection according to specifications] (English)
|-
|-

-->
<!-- Monitoring
<!-- Monitoring
| style="padding:2px;" | <div style="margin:3px; background:#cef2e0; font-size:120%; font-weight:bold; border:1px solid #BBBBBB; text-align:left; color:#000; padding:0.2em 0.4em;">Monitoring</div>
| style="padding:2px;" | <div style="margin:3px; background:#cef2e0; font-size:120%; font-weight:bold; border:1px solid #BBBBBB; text-align:left; color:#000; padding:0.2em 0.4em;">Monitoring</div>
Line 69: Line 74:
| style="color:#000; padding:2px 5px;" |
| style="color:#000; padding:2px 5px;" |
* [[LSDF_Online_Storage:_Secure_Shell|Secure Shell (ssh, scp, sftp, sshfs)]]
* [[LSDF_Online_Storage:_Secure_Shell|Secure Shell (ssh, scp, sftp, sshfs)]]
* [[LSDF_Online_Storage:_WebDAV|WebDAV / HTTPS]]<br>
* [[LSDF_Online_Storage:_SMB_/_CIFS|SMB / CIFS]] (KIT intern only)
* [[LSDF_Online_Storage:_SMB_/_CIFS|SMB / CIFS]] (KIT intern only)
* [[LSDF_Online_Storage_NFS|NFS]] (KIT intern only)
* [[LSDF_Online_Storage_NFS|NFS]] (KIT intern only)
* [[LSDF_Online_Storage:_WebDAV|WebDAV / HTTPS]]<br>
* [[LSDF_Online_Storage:_HPC|on the KIT HPC Clusters]]
* [[LSDF_Online_Storage:_HDF_AAI|using the AAI of the Helmholtz Data Fedaration (HDF)]]
* [[LSDF_Online_Storage_HPC|on the HPC Clusters]] ([http://www.scc.kit.edu/dienste/bwUniCluster.php bwUniCluster], [http://www.scc.kit.edu/dienste/forhlr.php ForHLR])


|-
|-
Line 86: Line 92:
|-
|-
| style="color:#000; padding:2px 5px;" |
| style="color:#000; padding:2px 5px;" |
* [[LSDF_Online_Storage_Filesystem|Filesystem characteristics]]
* [[LSDF_Online_Storage_Login_Cluster|Login Cluster]]
* [[LSDF_Online_Storage_WebDav_Cluster|WebDav Cluster]]
* [[LSDF_Online_Storage_Filesystem|Filesystem]]
* [[LSDF_Online_Storage_Quota|Quota]]
* [[LSDF_Online_Storage_Quota|Quota]]
<!-- * [[LSDF_Online_Storage_Price|Price]] -->
|}
|}
|}
|}

Latest revision as of 10:48, 8 October 2024

Lsdf online storage services.png
LSDF Online Storage @ KIT (SCC)

The LSDF is a central storage system for large research data from data-intensive scientific disciplines. It is available to KIT researchers, their partners in the federal state and the Helmholtz Association as a multi-petabyte storage system. The service LSDF Online Storage enables KIT users and their partners in the state of Baden-Württemberg to access the storage resources of the LSDF. The service is used by institutes with research funded by the Helmholtz Association as well as insitutes at the university level of the KIT from different scientific disciplines. The project is sponsored and funded by the Ministry of Science, Research and the Arts of Baden-Württemberg (MWK), the German Research Foundation (DFG) and the Helmholtz Association of German Research Centres. It is operated by the Scientific Computing Center (SCC) at KIT.

The LSDF enables scientists to store and access their research data via different services. These services, tailored to the requirements of the researchers, are offered on top of the LSDF storage. Important features are in particular high storage capacity, fast access to and transfer of the data, easy data access and management capabilities, as well as the connection to modern processing infrastructures such as the high-performance computing clusters based at KIT like the bwUniCluster or the ForHLR II.

Access to the data is possible via standard protocols. Permissions to access the data are controlled by the data owners using group memberships and access control lists (ACLs). The LSDF Online Storage currently offers 10PB of usable storage capacity with a combined maximum read/write speed of the storage system of 25GByte/s. Today it is used by about 400 scientists.

In order to use the LSDF Online Storage Service, interested users must first register for the service. Registration takes place via bwIDM web interface. Registration is only possible if a special Entitlement has been assigned to the user previously. Users usually receive this Entitlement after consultation with their responsible ITB or group representative or, if no project has yet been set up, by a separate request to the service operator.

If you have any questions regarding registration, requesting a new storage project or further questions, please contact the service provider via the BW support portal.

The setup and size of the available storage space is determined by agreements between the users and the service provider and technically realized by quotas. For new projects a Data Management Plan (DMP) is developed and stored in coordination with the Institute of the Users and the representatives of the service provider. The DMPs are used for reporting and applying for funding to operate, maintain and expand the LSDF. The storage space of the LSDF Online Storage is currently usually available free of charge.

Further information on the range of services and the usage of the service can be found in this Wiki the user manual and the current terms of use.


LSDF Online Storage Projects
Documentation
Getting Help
Web sites
Accessing Data
Snapshots/Backup
Miscellaneous