BWDAHub: Difference between revisions
(Created page with "The archive provided by the bwDataArchiv (BWDA) project offers a GridFTP interface for high-performance uploads and downloads. For convenience and ease of use...") |
|||
| (5 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
[[Preparing access to the BWDAHub|<small>Preparing access to the BWDAHub</small>]]<br\> |
|||
[[Quickstart guide for gtransfer|<small>Quickstart guide for gtransfer</small>]]<br\> |
|||
[[Quickstart guide for gsatellite|<small>Quickstart guide for gsatellite</small>]] |
|||
---- |
|||
The archive provided by the [[BwDataArchiv|bwDataArchiv (BWDA)]] project offers a GridFTP interface for high-performance uploads and downloads. For convenience and ease of use we created a dedicated host - the '''BWDAHub''' - on our side from where you can manage your GridFTP data transfers into and out of the archive. This way you don't need to install the somewhat complex software products that are needed for high-performance GridFTP transfers, but just need to install the GSISSH client and forward your X.509 certificate's DN to [mailto:support-bwarchiv@lists.kit.edu us]. Please follow the detailed guide "[[Preparing access to the BWDAHub]]" to get access to the BWDAHub. |
The archive provided by the [[BwDataArchiv|bwDataArchiv (BWDA)]] project offers a GridFTP interface for high-performance uploads and downloads. For convenience and ease of use we created a dedicated host - the '''BWDAHub''' - on our side from where you can manage your GridFTP data transfers into and out of the archive. This way you don't need to install the somewhat complex software products that are needed for high-performance GridFTP transfers, but just need to install the GSISSH client and forward your X.509 certificate's DN to [mailto:support-bwarchiv@lists.kit.edu us]. Please follow the detailed guide "[[Preparing access to the BWDAHub]]" to get access to the BWDAHub. |
||
| Line 5: | Line 12: | ||
=== Installed software === |
=== Installed software === |
||
The BWDAHub comes preinstalled with |
The BWDAHub comes preinstalled with ''gtransfer'' (gt) and ''gsatellite'' (gsat). Gtransfer's motto is "''GridFTP transfers made easy''". You can find a description of all features on its [https://github.com/fr4nk5ch31n3r/gtransfer/ GitHub repository]. To start transferring data with gtransfer now, please have a look at the [[quickstart guide for gtransfer]]. |
||
| Line 16: | Line 23: | ||
For long running transfers, e.g. more than a few hours, you should make use of the gsatellite job scheduler which allows to schedule and run gtransfer jobs in the background. This comes in handy if you cannot remain logged in to the BWDAHub during the full transfer or if there are network errors or outages that could end your connection to the BWDAHub, which would also end all your running commands (incl. gtransfer). Instead of issuing the gtransfer command on the command line, you create a script containing the gtransfer command and submit it to the gsatellite scheduler. Gsatellite will then notify you when your gtransfer job was started and when it was finished. Gsatellite includes special support for gtransfer, which allows for automatic job restarts in case of temporary errors during a transfer. A [[quickstart guide for gsatellite]] is also avaliable. |
For long running transfers, e.g. more than a few hours, you should make use of the gsatellite job scheduler which allows to schedule and run gtransfer jobs in the background. This comes in handy if you cannot remain logged in to the BWDAHub during the full transfer or if there are network errors or outages that could end your connection to the BWDAHub, which would also end all your running commands (incl. gtransfer). Instead of issuing the gtransfer command on the command line, you create a script containing the gtransfer command and submit it to the gsatellite scheduler. Gsatellite will then notify you when your gtransfer job was started and when it was finished. Gsatellite includes special support for gtransfer, which allows for automatic job restarts in case of temporary errors during a transfer. A [[quickstart guide for gsatellite]] is also avaliable. |
||
---- |
|||
[[BwDataArchiv|<small>back to bwDataArchiv</small>]] [[Using bwDataArchiv|<small>back to Using bwDataArchiv</small>]] |
|||
Latest revision as of 09:30, 6 December 2016
Preparing access to the BWDAHub<br\> Quickstart guide for gtransfer<br\> Quickstart guide for gsatellite
The archive provided by the bwDataArchiv (BWDA) project offers a GridFTP interface for high-performance uploads and downloads. For convenience and ease of use we created a dedicated host - the BWDAHub - on our side from where you can manage your GridFTP data transfers into and out of the archive. This way you don't need to install the somewhat complex software products that are needed for high-performance GridFTP transfers, but just need to install the GSISSH client and forward your X.509 certificate's DN to us. Please follow the detailed guide "Preparing access to the BWDAHub" to get access to the BWDAHub.
Installed software
The BWDAHub comes preinstalled with gtransfer (gt) and gsatellite (gsat). Gtransfer's motto is "GridFTP transfers made easy". You can find a description of all features on its GitHub repository. To start transferring data with gtransfer now, please have a look at the quickstart guide for gtransfer.
Other GridFTP clients available for usage on the BWDAHub and also used by gtransfer internally:
- uberftp for interactive usage, similar to SFTP
- globus-url-copy a low-level tool for experts
For long running transfers, e.g. more than a few hours, you should make use of the gsatellite job scheduler which allows to schedule and run gtransfer jobs in the background. This comes in handy if you cannot remain logged in to the BWDAHub during the full transfer or if there are network errors or outages that could end your connection to the BWDAHub, which would also end all your running commands (incl. gtransfer). Instead of issuing the gtransfer command on the command line, you create a script containing the gtransfer command and submit it to the gsatellite scheduler. Gsatellite will then notify you when your gtransfer job was started and when it was finished. Gsatellite includes special support for gtransfer, which allows for automatic job restarts in case of temporary errors during a transfer. A quickstart guide for gsatellite is also avaliable.
