BwDAHub: Difference between revisions

Line 1:

The archive provided by the [[BwDataArchiv|bwDataArchiv (BWDA)]] project offers a GridFTP interface for high-performance uploads and downloads. For convenience and ease of use we created a dedicated host - the '''BWDAHub''' - on our side from where you can manage your GridFTP data transfers into and out of the archive. This way you don't need to install the somewhat complex software products that are needed for high-performance GridFTP transfers, but just need to install the GSISSH client and forward your X.509 certificate's DN to [mailto:support-bwarchiv@lists.kit.edu us]. Please follow the detailed guide "[[Preparing access to the BWDAHub]]" to get access to the BWDAHub.

[[file:Bwdahub.png]]

=== Installed software ===

The BWDAHub comes preinstalled with _gtransfer_ (gt) and _gsatellite_ (gsat). Gtransfer's motto is "_GridFTP transfers made easy_". You can find a description of all features on its [https://github.com/fr4nk5ch31n3r/gtransfer/ GitHub repository]. To start transferring data with gtransfer now, please have a look at the [[quickstart guide for gtransfer]].

Other GridFTP clients available for usage on the BWDAHub and also used by gtransfer internally:

* [https://github.com/JasonAlt/UberFTP uberftp] for interactive usage, similar to [http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man1/sftp.1?query=sftp&sec=1 SFTP]

* [http://toolkit.globus.org/toolkit/docs/latest-stable/gridftp/user/#globus-url-copy globus-url-copy] a low-level tool for experts

For long running transfers, e.g. more than a few hours, you should make use of the gsatellite job scheduler which allows to schedule and run gtransfer jobs in the background. This comes in handy if you cannot remain logged in to the BWDAHub during the full transfer or if there are network errors or outages that could end your connection to the BWDAHub, which would also end all your running commands (incl. gtransfer). Instead of issuing the gtransfer command on the command line, you create a script containing the gtransfer command and submit it to the gsatellite scheduler. Gsatellite will then notify you when your gtransfer job was started and when it was finished. Gsatellite includes special support for gtransfer, which allows for automatic job restarts in case of temporary errors during a transfer. A [[quickstart guide for gsatellite]] is also avaliable.

BwDAHub: Difference between revisions

Revision as of 14:02, 12 July 2016

Navigation menu

Page actions

Page actions

Personal tools

Navigation

Search

Tools

@@ Line 1: / Line 1: @@
-The archive provided by the [[BwDataArchiv|bwDataArchiv (BWDA)]] project offers a GridFTP interface for high-performance uploads and downloads. For convenience and ease of use we created a dedicated host - the '''BWDAHub''' - on our side from where you can manage your GridFTP data transfers into and out of the archive. This way you don't need to install the somewhat complex software products that are needed for high-performance GridFTP transfers, but just need to install the GSISSH client and forward your X.509 certificate's DN to [mailto:support-bwarchiv@lists.kit.edu us]. Please follow the detailed guide "[[Preparing access to the BWDAHub]]" to get access to the BWDAHub.
-[[file:Bwdahub.png]]
-=== Installed software ===
-The BWDAHub comes preinstalled with _gtransfer_ (gt) and _gsatellite_ (gsat). Gtransfer's motto is "_GridFTP transfers made easy_". You can find a description of all features on its [https://github.com/fr4nk5ch31n3r/gtransfer/ GitHub repository]. To start transferring data with gtransfer now, please have a look at the [[quickstart guide for gtransfer]].
-Other GridFTP clients available for usage on the BWDAHub and also used by gtransfer internally:
-* [https://github.com/JasonAlt/UberFTP uberftp] for interactive usage, similar to [http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man1/sftp.1?query=sftp&sec=1 SFTP]
-* [http://toolkit.globus.org/toolkit/docs/latest-stable/gridftp/user/#globus-url-copy globus-url-copy] a low-level tool for experts
-For long running transfers, e.g. more than a few hours, you should make use of the gsatellite job scheduler which allows to schedule and run gtransfer jobs in the background. This comes in handy if you cannot remain logged in to the BWDAHub during the full transfer or if there are network errors or outages that could end your connection to the BWDAHub, which would also end all your running commands (incl. gtransfer). Instead of issuing the gtransfer command on the command line, you create a script containing the gtransfer command and submit it to the gsatellite scheduler. Gsatellite will then notify you when your gtransfer job was started and when it was finished. Gsatellite includes special support for gtransfer, which allows for automatic job restarts in case of temporary errors during a transfer. A [[quickstart guide for gsatellite]] is also avaliable.