Globus Online for HPSS: Difference between revisions

From Lsdf
Jump to navigationJump to search
No edit summary
No edit summary
 
(21 intermediate revisions by one other user not shown)
Line 1: Line 1:
This is a guide on setup the Globus Online service to storge data on HPSS.
This is a guide on setup the Globus Online service to storge data on HPSS at KIT.

== Components ==
* GridFTP Server connected to HPSS-DSI
* myProxy Server for managing and creating user proxies
* OAuth Server for User logins to the globus-Online Endpoint at KIT.
* [https://www.globus.org/ Globus Online] Web Interface.


== Requirements ==
== Requirements ==


* A working [[HPSS Frontends setup| HPSS Frontend]] with HPSS Client software installed and configured.
* A working [[HPSS Frontends setup| HPSS Frontend]] with HPSS Client software installed and configured.
* A working GridFTP Server
* A working GridFTP Server with a valid Grid Host certificate. (GridKa Host certificate)
* A working [https://github.com/JasonAlt/GridFTP-DSI-for-HPSS HPSS-GridFTP-DSI] compiled package
* A working [https://github.com/JasonAlt/GridFTP-DSI-for-HPSS HPSS-GridFTP-DSI] compiled package
** Notice: a HPSS-Fuse instead of DSI would also work.
** Notice: a HPSS-Fuse instead of DSI would also work.
* A working Connection to an LDAP Server to authorized the user locally on the GridFTP server.
* A working Connection to an LDAP Server to authorize the user locally on the GridFTP server.
* [https://www.globus.org/SignUp A Globus Online account] if not already exists.
* [https://www.globus.org/SignUp A Globus Online account] if not already exists.
* A valid Grid User certificate.
* A valid Grid User certificate.


== Installation ==
== Installation ==

Hostname: archive-tgftp.lsdf.kit.edu
OS: SL 6.4

* Download and install Globus Connect server repository

# curl -LOs http://toolkit.globus.org/ftppub/globus-connect-server/globus-connect-server-repo-latest.noarch.rpm
# rpm --import http://www.globus.org/ftppub/globus-connect-server/RPM-GPG-KEY-Globus
# yum install globus-connect-server-repo-latest.noarch.rpm

* Install
# yum install globus-connect-server

== Configuration ==
* Adapt the config files. Both files contain detailed information on configuration possibilities. Please check!
/etc/globus-connect-server.conf
/var/lib/globus-connect-server/myproxy-server.conf

* globus-connect-server.conf

[Globus]
User = %(GLOBUS_USER)s
Password = %(GLOBUS_PASSWORD)s
[Endpoint]
Name = bwda-go-1
Public = True
DefaultDirectory = /~/
[Security]
FetchCredentialFromRelay = False
CertificateFile = /etc/grid-security/hostcert.pem
KeyFile = /etc/grid-security/hostkey.pem
TrustedCertificateDirectory = /etc/grid-security/certificates
IdentityMethod = OAuth
AuthorizationMethod = MyProxyGridmapCallout
[GridFTP]
Server = archive-tgftp.lsdf.kit.edu
IncomingPortRange = 50000,51000
OutgoingPortRange = 50000,51000
RestrictPaths = RW~,R/hpss/fs/GFTP/public (!! to check)
# still not working, needs registration
Sharing = True
SharingRestrictPaths = R/hpss/fs/GFTP/public
SharingStateDir = /var/globusonline/sharing/$USER
[MyProxy]
Server = %(HOSTNAME)s
ServerBehindNAT = False
CADirectory = /var/lib/globus-connect-server/myproxy-ca
ConfigFile = /var/lib/globus-connect-server/myproxy-server.conf
[OAuth]
Server = %(HOSTNAME)s

* myproxy-server.conf

authorized_retrievers "*"
default_retrievers "*"
authorized_renewers "*"
default_renewers "none"
default_key_retrievers "none"
trusted_retrievers "*"
default_trusted_retrievers "none"
accepted_credentials "*"
certificate_issuer_cert "/var/lib/globus-connect-server/myproxy-ca/cacert.pem"
certificate_issuer_key "/var/lib/globus-connect-server/myproxy-ca/private/cakey.pem"
certificate_issuer_key_passphrase "globus"
certificate_serialfile "/var/lib/globus-connect-server/myproxy-ca/serial"
certificate_out_dir "/var/lib/globus-connect-server/myproxy-ca/newcerts"
certificate_issuer_subca_certfile "/var/lib/globus-connect-server/myproxy-ca/cacert.pem"
max_cert_lifetime 168
cert_dir /etc/grid-security/certificates
pam "required"
pam_id "login"
certificate_mapapp /var/lib/globus-connect-server/myproxy-ca/mapapp
accepted_credentials_mapapp /usr/local/bin/myproxy-accepted-credentials-mapapp

== Endpoint Creation ==

Now setup your Frontend GridFTP Server as an Endpoint for globus Online

# globus-connect-server-setup

* The globus-connect-server-setup asks you for your Globus Online account name and password.
* If you run globus-connect-server-setup many times you get this error message:
"You are not an admin of the MyProxy Delegation Service"
*Solution: run as root:

# rm /var/lib/myproxy-oauth/myproxy-oauth.db

* The name of the Endpoint is: hpss#bwda-go-1


* Login via ssh to a Host where you have your valid Grid user Certificate and the myproxy packae installed.

* Create and store a credential on Myproxy Server which is the same as the Endpoint Server at KIT.
myproxy-init -s archive-tgftp.lsdf.kit.edu -l <LDAP username> <nowiki>[--cred_lifetime 0 | --no_passphrase]</nowiki>
* You will be prompted to enter your Grid user key password. You also will be prompted to set a so called MyProxy passphrase twice to protect your created credential on Myproxy Server. This password must be the same as your PAM password or LDAP-Account password.
* [http://grid.ncsa.illinois.edu/myproxy/pam.html Update to Myproxy passphrase issue] When using PAM for authentication to retrieve credentials stored in the MyProxy repository, it is simplest to store unencrypted credentials, via the '''--no_passphrase''' argument (equivalent to '''-n''') to myproxy-init. That way, the PAM password can change independently of MyProxy and users' stored credentials.
* Your created (proxy) credentials will last for 7 days but if you use the option '''--cred_lifetime 0''' (aquivalent to '''-c 0''') it will last for 366.9 days.
* Your credential will be saved on the MyProxy Server under
/var/lib/globus-connect-server/myproxy-ca/store
* myproxy-init executes a script to create the user's Home directory structure with private/ public/.
/usr/local/bin/myproxy-accepted-credentials-mapapp <User Cert DN><LDAP name>
* In case myproxy-init has failed complaining about missing CA-cerificates download the CA certificates directory via:
myproxy-get-trustroots -s archive-tgftp.lsdf.kit.edu
* Login to [https://www.globus.org Globus Online] with your Globus Online account credentials and activate the Endpoint.
* for the activation and File Transfer You will be redirected to the OAuth Server at KIT to login with your LDAP credentials.

== Usage for user without a Grid User certificate ==
* If a User does not have a valid Grid User Certificate the user can directly login to Globus Online chose the Endpoint (hpss#bwda-go-1) and a dummy proxy certificate will be created for him.
* You only must have an account on the archive-tgftp.lsdf.kit.edu.

Latest revision as of 09:34, 26 July 2017

This is a guide on setup the Globus Online service to storge data on HPSS at KIT.

Components

  • GridFTP Server connected to HPSS-DSI
  • myProxy Server for managing and creating user proxies
  • OAuth Server for User logins to the globus-Online Endpoint at KIT.
  • Globus Online Web Interface.

Requirements

  • A working HPSS Frontend with HPSS Client software installed and configured.
  • A working GridFTP Server with a valid Grid Host certificate. (GridKa Host certificate)
  • A working HPSS-GridFTP-DSI compiled package
    • Notice: a HPSS-Fuse instead of DSI would also work.
  • A working Connection to an LDAP Server to authorize the user locally on the GridFTP server.
  • A Globus Online account if not already exists.
  • A valid Grid User certificate.

Installation

Hostname: archive-tgftp.lsdf.kit.edu OS: SL 6.4

  • Download and install Globus Connect server repository
# curl -LOs http://toolkit.globus.org/ftppub/globus-connect-server/globus-connect-server-repo-latest.noarch.rpm
# rpm --import http://www.globus.org/ftppub/globus-connect-server/RPM-GPG-KEY-Globus
# yum install globus-connect-server-repo-latest.noarch.rpm
  • Install
# yum install globus-connect-server

Configuration

  • Adapt the config files. Both files contain detailed information on configuration possibilities. Please check!
/etc/globus-connect-server.conf
/var/lib/globus-connect-server/myproxy-server.conf
  • globus-connect-server.conf
[Globus]
User = %(GLOBUS_USER)s
Password = %(GLOBUS_PASSWORD)s
[Endpoint]
Name = bwda-go-1
Public = True
DefaultDirectory = /~/
[Security]
FetchCredentialFromRelay =  False
CertificateFile = /etc/grid-security/hostcert.pem
KeyFile = /etc/grid-security/hostkey.pem
TrustedCertificateDirectory = /etc/grid-security/certificates
IdentityMethod = OAuth
AuthorizationMethod = MyProxyGridmapCallout
[GridFTP]
Server = archive-tgftp.lsdf.kit.edu
IncomingPortRange = 50000,51000
OutgoingPortRange = 50000,51000
RestrictPaths = RW~,R/hpss/fs/GFTP/public (!! to check)
# still not working, needs registration
Sharing = True
SharingRestrictPaths = R/hpss/fs/GFTP/public
SharingStateDir = /var/globusonline/sharing/$USER
[MyProxy]
Server = %(HOSTNAME)s
ServerBehindNAT = False
CADirectory = /var/lib/globus-connect-server/myproxy-ca
ConfigFile = /var/lib/globus-connect-server/myproxy-server.conf
[OAuth]
Server = %(HOSTNAME)s
  • myproxy-server.conf
authorized_retrievers      "*"
default_retrievers         "*"
authorized_renewers        "*"
default_renewers           "none"
default_key_retrievers     "none"
trusted_retrievers         "*"
default_trusted_retrievers "none"
accepted_credentials       "*"            
certificate_issuer_cert "/var/lib/globus-connect-server/myproxy-ca/cacert.pem"
certificate_issuer_key "/var/lib/globus-connect-server/myproxy-ca/private/cakey.pem"
certificate_issuer_key_passphrase "globus"
certificate_serialfile "/var/lib/globus-connect-server/myproxy-ca/serial"
certificate_out_dir "/var/lib/globus-connect-server/myproxy-ca/newcerts"
certificate_issuer_subca_certfile "/var/lib/globus-connect-server/myproxy-ca/cacert.pem"
max_cert_lifetime 168
cert_dir /etc/grid-security/certificates
pam  "required"
pam_id "login"
certificate_mapapp /var/lib/globus-connect-server/myproxy-ca/mapapp
accepted_credentials_mapapp /usr/local/bin/myproxy-accepted-credentials-mapapp

Endpoint Creation

Now setup your Frontend GridFTP Server as an Endpoint for globus Online

# globus-connect-server-setup
  • The globus-connect-server-setup asks you for your Globus Online account name and password.
  • If you run globus-connect-server-setup many times you get this error message:
"You are not an admin of the MyProxy Delegation Service"
  • Solution: run as root:
# rm /var/lib/myproxy-oauth/myproxy-oauth.db
  • The name of the Endpoint is: hpss#bwda-go-1


  • Login via ssh to a Host where you have your valid Grid user Certificate and the myproxy packae installed.
  • Create and store a credential on Myproxy Server which is the same as the Endpoint Server at KIT.
myproxy-init -s archive-tgftp.lsdf.kit.edu -l <LDAP username> [--cred_lifetime  0 | --no_passphrase]
  • You will be prompted to enter your Grid user key password. You also will be prompted to set a so called MyProxy passphrase twice to protect your created credential on Myproxy Server. This password must be the same as your PAM password or LDAP-Account password.
  • Update to Myproxy passphrase issue When using PAM for authentication to retrieve credentials stored in the MyProxy repository, it is simplest to store unencrypted credentials, via the --no_passphrase argument (equivalent to -n) to myproxy-init. That way, the PAM password can change independently of MyProxy and users' stored credentials.
  • Your created (proxy) credentials will last for 7 days but if you use the option --cred_lifetime 0 (aquivalent to -c 0) it will last for 366.9 days.
  • Your credential will be saved on the MyProxy Server under
/var/lib/globus-connect-server/myproxy-ca/store
  • myproxy-init executes a script to create the user's Home directory structure with private/ public/.
/usr/local/bin/myproxy-accepted-credentials-mapapp <User Cert DN><LDAP name> 
  • In case myproxy-init has failed complaining about missing CA-cerificates download the CA certificates directory via:
myproxy-get-trustroots -s archive-tgftp.lsdf.kit.edu
  • Login to Globus Online with your Globus Online account credentials and activate the Endpoint.
  • for the activation and File Transfer You will be redirected to the OAuth Server at KIT to login with your LDAP credentials.

Usage for user without a Grid User certificate

  • If a User does not have a valid Grid User Certificate the user can directly login to Globus Online chose the Endpoint (hpss#bwda-go-1) and a dummy proxy certificate will be created for him.
  • You only must have an account on the archive-tgftp.lsdf.kit.edu.