View source for Preparing access to the BWDAHub

&nbsp;&nbsp;[[BWDAHub|<small>BWDAHub</small>]]
&nbsp;&nbsp;<small>Preparing access to the BWDAHub</small>
&nbsp;&nbsp;[[Quickstart guide for gtransfer|<small>Quickstart guide for gtransfer</small>]]
&nbsp;&nbsp;[[Quickstart guide for gsatellite|<small>Quickstart guide for gsatellite</small>]]
----


== Introduction ==

For using gridFTP with the BWDataArchiv (BWDA) service it is mandatory to use certificates. X.509 certificates allow the mapping of user credentials to real persons and are issued or signed by so-called registration authorities (RAs). Due to this mapping certificates allow for very trustworthy communication. Please see [https://info.pca.dfn.de/grid-ras.html 1] for a listing of Grid RAs available in Germany.

<div style="border:1px solid #808080; margin:5px 3px 0px 3px; padding:3px 5px 2px 5px; background:#CEEEEE">'''NOTE:''' Commands or options in brackets are optional and may depend on your currently used UID or your choice. The <span style="color:red">$</span> character marks the shell prompt for a non-root user in Linux.
</div>

== GSI proxy credential ==

For accessing the BWDAHub via gsissh and performing your data transfers with GridFTP you need a GSI proxy credential (GPC) signed by your personal X.509 certificate. Please see [http://toolkit.globus.org/toolkit/docs/4.0/security/key-index.html 2] for more information about GSI proxy certificates.

I.e. this means that you first need a personal X.509 certificate signed by your organization or institute. In addition the source and destination GridFTP services must be able to verify your GPC to enable the data transfer. By default a GPC derived from a personal X.509 certificate issued by the two German grid certificate authorities:

* "DFN-Verein PCA Grid - G01"
* "GridKa-CA"

...or their affiliated RAs is required for data transfers. Other CAs can be accepted locally at the BWDataArchiv after manual configuration by BWDA personnel, but you also need to make sure that your derived GPC is also accepted by the source/destination GridFTP service (depending on the direction of the GridFTP data transfer).

Please contact your IT department on how to acquire such a personal X.509 certificate. After receiving your personal X.509 certificate you need to forward the certificate's distinguished name (DN) to the BWDA personnel in order to activate access to the BWDAHub. You can do this either using the webinterface https://www.rda.kit.edu/bwDA or by sending an email to bwarchiv-support[at]lists.kit.edu. Note that it may take some days until your DN will be usable for data transfers. To determine the DN you can use the following openssl command on your personal X.509 certificate:

<pre>
$ openssl x509 -noout -subject -in <YOUR_PERSONAL_X509_CERTIFICATE_FILE>
</pre>


=== Procedure (Linux) ===


'''Install the <span style="color:red"><code>globus-proxy-utils </code></span> package'''  

RHEL and compatible:

* Activate EPEL repository (see [https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F 3] for details) or the Globus Alliance repository (see [http://toolkit.globus.org/toolkit/downloads/latest-stable/ 4] for details)
* install package with <span style="color:red"><code>$ [sudo] yum install globus-proxy-utils</code></span>

Debian and compatible:

* Activate the Globus Alliance repository (see [http://toolkit.globus.org/toolkit/downloads/latest-stable/ 4] for details)
* install package with <span style="color:red"><code>$ [sudo] apt-get install globus-proxy-utils</code></span>

'''Create a GSI proxy credential''' 

Place your personal X.509 certficate and corresponding private key as PKCS#12 keystore in the directory <span style="color:red"><code>$HOME/.globus</code></span> (you usually can export such a keystore from your web browser!). File system access permissions are important, so please follow the next commands exactly:

* <span style="color:red"><code>$ mkdir $HOME/.globus; chmod 0700 $HOME/.globus</code></span>
* <span style="color:red"><code>$ umask 0177; touch $HOME/.globus/usercred.p12</code></span>
* Now export your keystore to the file <span style="color:red"><code>$HOME/.globus/usercred.p12</code></span>

Actually create your GPC:

* <span style="color:red"><code>$ grid-proxy-init [-valid <NUMBER_OF_HOURS:NUMBER_OF_MINUTES>]</code></span>
* Your GSI proxy certficate will be valid for the given number of hours and minutes or 12 hours by default

== Install and configure gsissh ==

Gsissh is a modified version of ssh which allows authentication with a GPC.


=== Procedure (Linux) ===


'''Install the <span style="color:red"><code>gsi-openssh-clients</code></span> package''' 

RHEL and compatible:

* Activate EPEL repository (see [https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F 3] for details) or the Globus Alliance repository (see [http://toolkit.globus.org/toolkit/downloads/latest-stable/ 4] for details)
* Install package with <span style="color:red"><code>$ [sudo] yum install gsi-openssh-clients</code></span>

Debian and compatible:

* Activate the Globus Alliance repository (see [http://toolkit.globus.org/toolkit/downloads/latest-stable/ 4] for details)
* Install package with <span style="color:red"><code>$ [sudo] apt-get install gsi-openssh-clients </code></span>
* On Debian additionally install the <span style="color:red"><code>libglobus-usage0 </code></span> package with <span style="color:red"><code>$ [sudo] apt-get install libglobus-usage0 </code></span>


'''Configure the trusted CA certficates directory'''

When accessing a gsissh service on a remote site the gsissh client checks the authenticity of the host certificate offered before continuing with authentication of the user. To be able to verify the offered host certificate, the client needs to trust the certificate of the CA that signed the host certificate. The BWDAHub is hosted by [http://www.kit.edu KIT] in Karlsruhe and hence its host certificate was signed by the "GridKa-CA". Hence your gsissh client does only need to trust the CA certificate of the "GridKa-CA" to successfully verify the host certificate of the BWDAHub.

First create the needed directory for the CA certificate:

<pre>
[$ mkdir -p $HOME/.globus; chmod 0700 $HOME/.globus]
$ mkdir $HOME/.globus/certificates
</pre>

Then download the [https://www.rda.kit.edu/downloads/certificates.tar.gz tarball] containing the necessary certificate and support files via your web browser and place it in <span style="color:red"><code>$HOME/.globus/certificates</code></span>. The SHA256 hash of the tarball is:

<pre>
154bf3698be4502dffa68b65e94d9e5e1c3be5b0e73e045294d24c523041b445
</pre>

If your download has a different hash value, don't use it and contact the [mailto:support-bwarchiv@lists.kit.edu BWDA personnel] for further assistance.

Now untar it with <span style="color:red"><code>$ tar -xzf certificates.tar.gz</code></span>

== Login to the BWDAHub ==


After following the descriptions made above you will be able to login to the BWDAHub with the following command:

<pre>
$ gsissh bwdahub.lsdf.kit.edu -p 22222
Last login: Thu May 19 14:22:00 2016 from userhost.domain.tld
+-[Welcome]-------------------------------------------------------------------+
|                                                                             |
|                       BWDAHub (bwdahub.lsdf.kit.edu)                        |
|                                                                             |
+-[Contact]-------------------------------------------------------------------+
|                                                                             |
| General support:                                                            |
|                                                                             |
| * <support-bwarchiv@lists.kit.edu>                                          |
|                                                                             |
+-[Docs]----------------------------------------------------------------------+
|                                                                             |
| Before you start, please have a look at the documentation available in:     |
|                                                                             |
| /usr/share/doc/bwdahub-0.5.0                                                |
|                                                                             |
| * gtransfer-quickstart.md                                                   |
| * gsatellite-quickstart.md                                                  |
|                                                                             |
+-[News]----------------------------------------------------------------------+
|                                                                             |
| 2016-09-06:                                                                 |
|  New gtransfer version (v0.7.0) installed on the BWDAHub. With this version |
|  it's now also possible to specify the user account on GridFTP servers when |
|  using host aliases. I.e.:                                                  |
|                                                                             |
|  `$ gt -s user1@my-gridftp:/~/files/* -d user2@bwda:/~/files`               |
|                                                                             |
|  ...will now also work.                                                     |
|                                                                             |
|  For more details about the new release visit:                              |
|                                                                             |
|  `https://github.com/fr4nk5ch31n3r/gtransfer/releases/tag/v0.7.0`           |
|                                                                             |
+-----------------------------------------------------------------------------+
INFO: Disk quotas for user user (uid 123): 
    Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
     /dev/sda4    123M   1024M   1536M            1627       0       0         OK
INFO: Creating limited delegated GSI proxy credential "/home/user/.gsatellite/tmp/defaultGsiProxyCredential" for gtransfer data transfer jobs... OK
[user@archive-gftp-fuse ~]$
</pre>

== I wasn't able to install the required Globus tools ==

If the Globus tools cannot be installed or if there are no binaries for your platform, please follow the next steps to get access to the BWDAHub and the tools installed there.

=== Alternative GSI proxy credential creation ===

Download the tool [https://www.rda.kit.edu/downloads/genproxy.bash genproxy] from our website (right klick on the link and use "Save Link as..." for downloading the file because otherwise it is displayed by the browser as it is a plain text bash script file). Before continuing, check that the computed SHA256 hash for the content of the tool matches the value given below (e.g. on Linux use <span style="color:red"><code>$ sha256sum genproxy</code></span>):

<pre>
98b4232f709331527a5edcddc3d3561180e001bb634e775cab085d7a24d29344
</pre>

If your download has a different hash value, don't use it and contact the [mailto:support-bwarchiv@lists.kit.edu BWDA personnel] for further assistance.

Make the tool executable (<span style="color:red"><code>$ chmod +x genproxy</code></span>) and include it in your <span style="color:red"><code>PATH</code></span> environment variable so that you can use it from everywhere. Some Linux distributions also automatically include executables placed in <span style="color:red"><code>$HOME/bin</code></span> in your <span style="color:red"><code>PATH</code></span> environment variable. Alternatively copy <span style="color:red"><code>genproxy</code></span> to your <span style="color:red"><code>$HOME/.globus</code></span> directory and use it from this directory exclusively. Don't forget to prepend the path to the tool then or <span style="color:red"><code>./</code></span> when calling it from there directly.

When finished make sure your personal certificate (<span style="color:red"><code>usercert.pem</code></span>) and private key (<span style="color:red"><code>userkey.pem</code></span>) are available as PEM files in <span style="color:red"><code>$HOME/.globus</code></span>. If this is the case, you can now create a GSI proxy credential by just calling <span style="color:red"><code>genproxy</code></span> and entering the pass phrase to decrypt your private key:

Example:
<pre>
$ genproxy 
Your identity: /C=DE/O=Grid/OU=University #1/CN=User
Enter pass phrase for /home/user/.globus/userkey.pem:
Your proxy `/tmp/x509up_p6729.file8x0ds0.1' is valid until: Wed Aug 17 12:31:23 CEST 2016
</pre>

<div style="border:1px solid #808080; margin:5px 3px 0px 3px; padding:3px 5px 2px 5px; background:#CEEEEE">'''NOTE:''' By default the <span style="color:red"><code>genproxy</code></span> tool uses a "non-guessable" name for the generated GPC. This is important on multi-user hosts because otherwise symlink attacks are possible in <span style="color:red"><code>/tmp</code></span> which can expose the GPC to other users. If you're running <span style="color:red"><code>genproxy</code></span> from a single user host, you can also predefine the path and name of the GPC in the environment variable <span style="color:red"><code>X509_USER_PROXY</code></span> (for example with <span style="color:red"><code>$ export X509_USER_PROXY="$HOME/.globus/mygpc"</code></span>).</div>

=== Copy GSI proxy credential to the BWDAHub ===

<div style="border:1px solid #808080; margin:5px 3px 0px 3px; padding:3px 5px 2px 5px; background:#CEEEEE">'''NOTE:''' For the following step you need to know your password on the BWDAHub or have already placed your SSH public key there, so you can login to the BWDAHub via <span style="color:red"><code>ssh</code></span>.</div>

When finished with the GSI proxy credential creation, copy the created GSI proxy credential to the BWDAHub using <span style="color:red"><code>scp</code></span> and the exact name for the destination file (<span style="color:red"><code>X509_USER_PROXY</code></span>) as given in the following command:

<pre>
$ scp /tmp/x509up_p6729.file8x0ds0.1 user@bwdahub.lsdf.kit.edu:.globus/X509_USER_PROXY
</pre>

...or when you used the environment variable <span style="color:red"><code>X509_USER_PROXY</code></span> also with the following command:

<pre>
$ scp "$X509_USER_PROXY" user@bwdahub.lsdf.kit.edu:.globus/X509_USER_PROXY
</pre>

=== Logging in to the BWDAHub using ssh ===

Now you are ready to login to the BWDAHub and using the tools installed there with <span style="color:red"><code>ssh</code></span> alone.

<div style="border:1px solid #808080; margin:5px 3px 0px 3px; padding:3px 5px 2px 5px; background:#CEEEEE">'''NOTE:''' With <span style="color:red"><code>ssh</code></span> you need to use your username on the BWDAHub, as otherwise <span style="color:red"><code>ssh</code></span> tries to login with your local username.</div>

<pre>
$ ssh user@bwdahub.lsdf.kit.edu
Last login: Thu May 19 14:22:00 2016 from userhost.domain.tld

+-[Welcome]-------------------------------------------------------------------+
|                                                                             |
|                       BWDAHub (bwdahub.lsdf.kit.edu)                        |
|                                                                             |
+-[Contact]-------------------------------------------------------------------+
|                                                                             |
| General support:                                                            |
|                                                                             |
| * <support-bwarchiv@lists.kit.edu>                                          |
|                                                                             |
+-[Docs]----------------------------------------------------------------------+
|                                                                             |
| Before you start, please have a look at the documentation available in:     |
|                                                                             |
| /usr/share/doc/bwdahub-0.5.0                                                |
|                                                                             |
| * gtransfer-quickstart.md                                                   |
| * gsatellite-quickstart.md                                                  |
|                                                                             |
+-[News]----------------------------------------------------------------------+
|                                                                             |
| 2016-09-06:                                                                 |
|  New gtransfer version (v0.7.0) installed on the BWDAHub. With this version |
|  it's now also possible to specify the user account on GridFTP servers when |
|  using host aliases. I.e.:                                                  |
|                                                                             |
|  `$ gt -s user1@my-gridftp:/~/files/* -d user2@bwda:/~/files`               |
|                                                                             |
|  ...will now also work.                                                     |
|                                                                             |
|  For more details about the new release visit:                              |
|                                                                             |
|  `https://github.com/fr4nk5ch31n3r/gtransfer/releases/tag/v0.7.0`           |
|                                                                             |
+-----------------------------------------------------------------------------+

INFO: Disk quotas for user user (uid 123): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
      /dev/sda4    123M   1024M   1536M            1627       0       0         OK
INFO: Creating limited delegated GSI proxy credential "/home/user/.gsatellite/tmp/defaultGsiProxyCredential" for gtransfer data transfer jobs... OK
[user@archive-gftp-fuse ~]$
</pre>

----
[[BWDAHub|<small>back to BWDAHub</small>]]&nbsp;&nbsp;&nbsp;[[BwDataArchiv|<small>back to bwDataArchiv</small>]]&nbsp;&nbsp;&nbsp;[[Using bwDataArchiv|<small>back to Using bwDataArchiv</small>]]