Preparing access to the BWDAHub
BWDAHub Preparing access to the BWDAHub Quickstart guide for gtransfer Quickstart guide for gsatellite
Introduction
For using the BWDataArchiv (BWDA) service it is mandatory to use certificates. X.509 certificates allow the mapping of user credentials to real persons and are issued or signed by so-called registration authorities (RAs). Due to this mapping certificates allow for very trustworthy communication. Please see 1 for a listing of Grid RAs available in Germany.
GSI proxy credential
For accessing the BWDAHub via gsissh and performing your data transfers with GridFTP you need a GSI proxy credential (GPC) signed by your personal X.509 certificate. Please see 2 for more information about GSI proxy certificates.
I.e. this means that you first need a personal X.509 certificate signed by your organization or institute. In addition the source and destination GridFTP services must be able to verify your GPC to enable the data transfer. By default a GPC derived from a personal X.509 certificate issued by the two German grid certificate authorities:
- "DFN-Verein PCA Grid - G01"
- "GridKa-CA"
...or their affiliated RAs is required for data transfers. Other CAs can be accepted locally at the BWDataArchiv after manual configuration by BWDA personnel, but you also need to make sure that your derived GPC is also accepted by the source/destination GridFTP service (depending on the direction of the GridFTP data transfer).
Please contact your IT department on how to acquire such a personal X.509 certificate. After receiving your personal X.509 certificate you need to forward the certificate's distinguished name (DN) to the BWDA personnel in order to activate access to the BWDAHub. To determine the DN you can use the following openssl command on your personal X.509 certificate:
$ openssl x509 -noout -subject -in <YOUR_PERSONAL_X509_CERTIFICATE_FILE>
Procedure (Linux)
Install the globus-proxy-utils package
RHEL and compatible:
- Activate EPEL repository (see 3 for details) or the Globus Alliance repository (see 4 for details)
- install package with
$ [sudo] yum install globus-proxy-utils
Debian and compatible:
- Activate the Globus Alliance repository (see 4 for details)
- install package with
$ [sudo] apt-get install globus-proxy-utils
Create a GSI proxy credential
Place your personal X.509 certficate and corresponding private key as PKCS#12 keystore in the directory $HOME/.globus (you usually can export such a keystore from your web browser!). File system access permissions are important, so please follow the next commands exactly:
$ mkdir $HOME/.globus; chmod 0700 $HOME/.globus$ umask 0177; touch $HOME/.globus/usercred.p12- Now export your keystore to the file
$HOME/.globus/usercred.p12
Actually create your GPC:
$ grid-proxy-init [-valid <NUMBER_OF_HOURS:NUMBER_OF_MINUTES>]- Your GSI proxy certficate will be valid for the given number of hours and minutes or 12 hours by default
Install and configure gsissh
Gsissh is a modified version of ssh which allows authentication with a GPC.
Procedure (Linux)
Install the gsi-openssh-clients package
RHEL and compatible:
- Activate EPEL repository (see 3 for details) or the Globus Alliance repository (see 4 for details)
- Install package with
$ [sudo] yum install gsi-openssh-clients
Debian and compatible:
- Activate the Globus Alliance repository (see 4 for details)
- Install package with
$ [sudo] apt-get install gsi-openssh-clients - On Debian additionally install the
libglobus-usage0package with$ [sudo] apt-get install libglobus-usage0
Configure the trusted CA certficates directory
When accessing a gsissh service on a remote site the gsissh client checks the authenticity of the host certificate offered before continuing with authentication of the user. To be able to verify the offered host certificate, the client needs to trust the certificate of the CA that signed the host certificate. The BWDAHub is hosted by KIT in Karlsruhe and hence its host certificate was signed by the "GridKa-CA". Hence your gsissh client does only need to trust the CA certificate of the "GridKa-CA" to successfully verify the host certificate of the BWDAHub.
First create the needed directory for the CA certificate:
[$ mkdir -p $HOME/.globus; chmod 0700 $HOME/.globus] $ mkdir $HOME/.globus/certificates
Then download the tarball containing the necessary certificate and support files via your web browser and place it in $HOME/.globus/certificates. The SHA256 hash of the tarball is:
52136e8943f03b8accfc8573273786a84fe6ee50f4ad33a9a45e8d379d5199a8
If your download has a different hash value, don't use it and contact the BWDA personnel for further assistance.
Now untar it with $ tar -xzf certificates.tar.gz
Login to the BWDAHub
After following the descriptions made above you will be able to login to the BWDAHub with the following command:
$ gsissh bwdahub.lsdf.kit.edu -p 22222
Last login: Thu May 19 14:22:00 2016 from userhost.domain.tld
+-[Welcome]-------------------------------------------------------------------+
| |
| BWDAHub (bwdahub.lsdf.kit.edu) |
| |
+-[Contact]-------------------------------------------------------------------+
| |
| General support: |
| |
| * <support-bwarchiv@lists.kit.edu> |
| |
+-[Docs]----------------------------------------------------------------------+
| |
| Before you start, please have a look at the documentation available in: |
| |
| /usr/share/doc/bwdahub-0.5.0 |
| |
| * gtransfer-quickstart.md |
| * gsatellite-quickstart.md |
| |
+-[News]----------------------------------------------------------------------+
| |
| 2016-09-06: |
| New gtransfer version (v0.7.0) installed on the BWDAHub. With this version |
| it's now also possible to specify the user account on GridFTP servers when |
| using host aliases. I.e.: |
| |
| `$ gt -s user1@my-gridftp:/~/files/* -d user2@bwda:/~/files` |
| |
| ...will now also work. |
| |
| For more details about the new release visit: |
| |
| `https://github.com/fr4nk5ch31n3r/gtransfer/releases/tag/v0.7.0` |
| |
+-----------------------------------------------------------------------------+
INFO: Disk quotas for user user (uid 123):
Filesystem blocks quota limit grace files quota limit grace
/dev/sda4 123M 1024M 1536M 1627 0 0 OK
INFO: Creating limited delegated GSI proxy credential "/home/user/.gsatellite/tmp/defaultGsiProxyCredential" for gtransfer data transfer jobs... OK
[user@archive-gftp-fuse ~]$
I wasn't able to install the required Globus tools
If the Globus tools cannot be installed or if there are no binaries for your platform, please follow the next steps to get access to the BWDAHub and the tools installed there.
Alternative GSI proxy credential creation
Download the tool genproxy from our website (right klick on the link and use "Save Link as..." for downloading the file because otherwise it is displayed by the browser as it is a plain text bash script file). Before continuing, check that the computed SHA256 hash for the content of the tool matches the value given below (e.g. on Linux use $ sha256sum genproxy):
98b4232f709331527a5edcddc3d3561180e001bb634e775cab085d7a24d29344
If your download has a different hash value, don't use it and contact the BWDA personnel for further assistance.
Make the tool executable ($ chmod +x genproxy) and include it in your PATH environment variable so that you can use it from everywhere. Some Linux distributions also automatically include executables placed in $HOME/bin in your PATH environment variable. Alternatively copy genproxy to your $HOME/.globus directory and use it from this directory exclusively. Don't forget to prepend the path to the tool then or ./ when calling it from there directly.
When finished make sure your personal certificate (usercert.pem) and private key (userkey.pem) are available as PEM files in $HOME/.globus. If this is the case, you can now create a GSI proxy credential by just calling genproxy and entering the pass phrase to decrypt your private key:
Example:
$ genproxy Your identity: /C=DE/O=Grid/OU=University #1/CN=User Enter pass phrase for /home/user/.globus/userkey.pem: Your proxy `/tmp/x509up_p6729.file8x0ds0.1' is valid until: Wed Aug 17 12:31:23 CEST 2016
genproxy tool uses a "non-guessable" name for the generated GPC. This is important on multi-user hosts because otherwise symlink attacks are possible in /tmp which can expose the GPC to other users. If you're running genproxy from a single user host, you can also predefine the path and name of the GPC in the environment variable X509_USER_PROXY (for example with $ export X509_USER_PROXY="$HOME/.globus/mygpc").Copy GSI proxy credential to the BWDAHub
ssh.When finished with the GSI proxy credential creation, copy the created GSI proxy credential to the BWDAHub using scp and the exact name for the destination file (X509_USER_PROXY) as given in the following command:
$ scp /tmp/x509up_p6729.file8x0ds0.1 user@bwdahub.lsdf.kit.edu:.globus/X509_USER_PROXY
...or when you used the environment variable X509_USER_PROXY also with the following command:
$ scp "$X509_USER_PROXY" user@bwdahub.lsdf.kit.edu:.globus/X509_USER_PROXY
Logging in to the BWDAHub using ssh
Now you are ready to login to the BWDAHub and using the tools installed there with ssh alone.
ssh you need to use your username on the BWDAHub, as otherwise ssh tries to login with your local username.$ ssh user@bwdahub.lsdf.kit.edu
Last login: Thu May 19 14:22:00 2016 from userhost.domain.tld
+-[Welcome]-------------------------------------------------------------------+
| |
| BWDAHub (bwdahub.lsdf.kit.edu) |
| |
+-[Contact]-------------------------------------------------------------------+
| |
| General support: |
| |
| * <support-bwarchiv@lists.kit.edu> |
| |
+-[Docs]----------------------------------------------------------------------+
| |
| Before you start, please have a look at the documentation available in: |
| |
| /usr/share/doc/bwdahub-0.5.0 |
| |
| * gtransfer-quickstart.md |
| * gsatellite-quickstart.md |
| |
+-[News]----------------------------------------------------------------------+
| |
| 2016-09-06: |
| New gtransfer version (v0.7.0) installed on the BWDAHub. With this version |
| it's now also possible to specify the user account on GridFTP servers when |
| using host aliases. I.e.: |
| |
| `$ gt -s user1@my-gridftp:/~/files/* -d user2@bwda:/~/files` |
| |
| ...will now also work. |
| |
| For more details about the new release visit: |
| |
| `https://github.com/fr4nk5ch31n3r/gtransfer/releases/tag/v0.7.0` |
| |
+-----------------------------------------------------------------------------+
INFO: Disk quotas for user user (uid 123):
Filesystem blocks quota limit grace files quota limit grace
/dev/sda4 123M 1024M 1536M 1627 0 0 OK
INFO: Creating limited delegated GSI proxy credential "/home/user/.gsatellite/tmp/defaultGsiProxyCredential" for gtransfer data transfer jobs... OK
[user@archive-gftp-fuse ~]$
back to BWDAHub back to bwDataArchiv back to Using bwDataArchiv
