View source for Quickstart guide for gtransfer

&nbsp;&nbsp;[[BWDAHub|<small>BWDAHub</small>]]<br\>
&nbsp;&nbsp;[[Preparing access to the BWDAHub|<small>Preparing access to the BWDAHub</small>]]<br\>
&nbsp;&nbsp;<small>Quickstart guide for gtransfer</small><br\>
&nbsp;&nbsp;[[Quickstart guide for gsatellite|<small>Quickstart guide for gsatellite</small>]]
----


= Basics =
 
1. Type <span style="color:red"><code>gt</code></span>  and hit the ENTER/RETURN key to get a brief usage message. Use <span style="color:red"><code>gt --help</code></span>  and <span style="color:red"><code>man gt</code></span>  to get a description of all options.

2. To start a transfer, enter <span style="color:red"><code>gt</code></span>  hit the SPACE key and then hit the TAB key three times. You'll get a listing of all available options. Start with <span style="color:red"><code>-s</code></span>  to enter the source address. The <span style="color:red"><code>-</code></span>  character was already provided by the gt bash completion. After entering <span style="color:red"><code>s</code></span>  hit the SPACE key and enter your source address, e.g. <span style="color:red"><code>gsiftp://gridftp.domain.tld:2811</code></span> . You can also hit the TAB key two times to get the preconfigured GridFTP source server addresses or [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/host-aliases.md host aliases].  
Add <span style="color:red"><code>/~/</code></span>  and then hit the TAB key two times to get a listing of the files and directories in your home directory on the remote server. Depending on the latency, it can take a few seconds until you see results and this will only work if your GSI proxy certificate is considered valid by the remote GridFTP server and you are trying to list a directory where you have <span style="color:red"><code>rx</code></span>  (read and execute) permissions.  
Type in the beginning of your desired file or directory and hit the TAB key to complete the name. If you want to copy all files in a directory, add <span style="color:red"><code>/*</code></span>  or just <span style="color:red"><code>/</code></span>  to the end of the path.  
Now continue with the destination address. Add <span style="color:red"><code>-d</code></span>  to the command line, hit the SPACE key and continue with the destination address just like you entered the source address. Enter a <span style="color:red"><code>/</code></span>  at the end of the destination path.

3. To recursively copy all files and directories below a given directory, add the <span style="color:red"><code>-r</code></span>  option to the gt command line.

<div style="border:1px solid #808080; margin:5px 3px 0px 3px; padding:3px 5px 2px 5px; background:#CEEEEE">'''NOTE:'''  Please note that since 2016-04-27 we've changed the default behaviour of gtransfer to also encrypt the data channel by default. This change was done by creating shell aliases for the <span style="color:red"><code>gt</code></span>  and <span style="color:red"><code>gtransfer</code></span>  commands that include the <span style="color:red"><code>-e</code></span>  option. You can list all active aliases with the <span style="color:red"><code>alias</code></span>  command.  
Though encrypting the data channel increases security for delicate data during transfer time, it also decreases performance considerably. Therefore if you don't need encryption for your data during transfer time, you can deactivate this new behaviour by using the following command (enter it without the shell prompt!):

</div> 

 $ unalias gtransfer gt

This will reset to the original behaviour of gtransfer (no data channel encryption) for your current session. To make this change permanent, include the command after the shell prompt in your shell configuration file (<span style="color:red"><code>$HOME/.bash_profile</code></span>  for the bash) at the end of the file.

To revert back to the new behaviour, simply exit your current session and relogin, or - if you already included the <span style="color:red"><code>unalias</code></span>  command in your shell configuration file - comment or remove the respective line in your shell configuration file, exit your current session and relogin.

<div style="border:1px solid #808080; margin:5px 3px 0px 3px; padding:3px 5px 2px 5px; background:#CEEEEE">
'''NOTE:''' Shell aliases are not expanded inside of shell scripts. And gtransfer jobs ran by [[Quickstart guide for gsatellite|gsatellite]] are just shell scripts that call the <span style="color:red"><code>gt(ransfer)</code></span> command. This means that in order to use data channel encryption for gtransfer jobs, you have to add the option <span style="color:red"><code>-e</code></span> to your gt command line inside of the job script.
</div>


= Hints =


== I have multiple user accounts at a remote GridFTP server. How can I choose a specific account? ==

This can be done by inserting a <span style="color:red"><code><USER>@</code></span> portion into your GridFTP URLs or prefixing host aliases with <span style="color:red"><code><USER>@</code></span>. Replace <span style="color:red"><code><USER></code></span> with your desired username on the remote site.

Examples:

* GridFTP URL:
<span style="color:red"><code>gsiftp://gridftp.domain.tld:2811/~/files/</code></span> => <span style="color:red"><code>gsiftp://user1@gridftp.domain.tld:2811/~/files/*</code></span>
* Host alias:  
<span style="color:red"><code>my-gridftp:/~/files/</code></span> => <span style="color:red"><code>user1@my-gridftp:/~/files/</code></span>


== Can gtransfer automatically create non-existing directories on the destination side? ==

Yes, this is possible and activated by default. Just enter the desired name or path in your destination URL and gtransfer will automatically create non-existing directories on the destination side (with the help of [http://toolkit.globus.org/toolkit/docs/latest-stable/gridftp/user/#globus-url-copy globus-url-copy]).


== Use host aliases for your GridFTP servers (including the BWDA) ==

There are already two host aliases defined which point to the archive provided by the BWDataArchiv project:

* <span style="color:red"><code>bwda:</code></span> 
* <span style="color:red"><code>bwa:</code></span>  (the old name!)

You can use them instead of the longer host part of a GridFTP URL in the source and destination URLs, e.g. you can use <span style="color:red"><code>bwda:/~/</code></span>  instead of <span style="color:red"><code>gsiftp://archive-sftp.lsdf.kit.edu:2811/~/</code></span> . To create your own host aliases, please refer to the host aliases documentation linked below.


== What if the gtransfer command fails during a data transfer? ==
 
[http://toolkit.globus.org/toolkit/docs/latest-stable/gridftp/user/#globus-url-copy Globus-url-copy] - the tool gtransfer actually uses through [https://github.com/fr4nk5ch31n3r/tgftp tgftp] to transfer data - is configured by gtransfer to retry the transfer of files that failed to transfer successfully to the destination GridFTP server. And if that fails, gtransfer will retry the whole process ten times until giving up on the transfer. And even if that happens, you can later continue a failed or interrupted transfer by simply issuing the very same gtransfer command. Gtransfer stores state information about a transfer in your home directory below <span style="color:red"><code>.gtransfer</code></span> . So this mechanism will work on the same machine and with the same user account and as long as the state files are not touched in between.

== What if I need to interrupt a data transfer? ==
 
You can always interrupt a gtransfer data transfer by hitting CTRL+C during a data transfer, which effectively sends a <span style="color:red"><code>SIGINT</code></span>  to the gtransfer process group and interrupts the data transfer. You can continue the transfer from where it was interrupted by issuing the very same gtransfer command - as with failed transfers described above. The same restrictions - same host, same user account, no fiddling with the state files in between - apply here.


= Documentation =


== General ==

* [https://github.com/fr4nk5ch31n3r/gtransfer/ gtransfer GitHub repository and README]


== Man pages ==

Man(ual) pages are also available locally on the BWDAHub. Simply enter <span style="color:red"><code>man</code></span>  and the name of the manpage (e.g. <span style="color:red"><code>gtransfer</code></span>  or <span style="color:red"><code>dpath</code></span> ) to read a specific page. If man pages with the same name exist in different sections you also have to specify the section number after the <span style="color:red"><code>man</code></span>  command but before the name of the man page to read a man page from a specific section. E.g. to read the <span style="color:red"><code>dparam(5)</code></span>  man page - which contains the file format description for dparams - you would enter <span style="color:red"><code>man 5 dparam</code></span> .


=== Section 1 ===

* [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/gtransfer.1.md gtransfer(1)]
* [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/dparam.1.md dparam(1)]
* [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/dpath.1.md dpath(1)]
* [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/halias.1.md halias(1)]


=== Section 5 ===

* [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/dparam.5.md dparam(5)]
* [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/dpath.5.md dpath(5)]


== Special functionality ==

* [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/host-aliases.md Host aliases]


----
[[BWDAHub|<small>back to BWDAHub</small>]]&nbsp;&nbsp;&nbsp;[[BwDataArchiv|<small>back to bwDataArchiv</small>]]&nbsp;&nbsp;&nbsp;[[Using bwDataArchiv|<small>back to Using bwDataArchiv</small>]]