hidden:S3 on WOS: Difference between revisions

From Lsdf
Jump to navigationJump to search
mNo edit summary
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==S3 on WOS==
==Intro==


The S3 protocol is delivered via gateways to the web object scaler (WOS) nodes. Currently there are 2 WOS nodes and 5 gateway nodes in production.

==WOS==

Each WOS nodes has 2 10 GE interfaces. Currently only a single 10 GE connection is active. IP adresses and cabling have been reserved for connecting the second interfaces (link aggregration). The 2 WOS nodes are logically in 2 different zones though physically they are in the same rack.

<pre>
wos-2-a 141.52.220.80
wos-2-b 141.52.220.81 (not active)
wos-3-a 141.52.220.82
wos-3-b 141.52.220.83 (not active)
</pre>

==S3 Gateways==
The S3 gateway is installed on 5 servers:
The S3 gateway is installed on 5 servers:
141.52.220.75
141.52.220.75
Line 13: Line 27:
* configctl
* configctl


Administrators can manage and monitor the activities on WOS Access S3 platform through the Web UI at http://141.52.220.79:8990/ddn_login.php.
Administrators can manage and monitor the activities on WOS Access S3 platform through the Web UI.


[[File:DDNwebUI.png|thumb|none|500px]]
[[File:DDNwebUI.png|thumb|none|500px]]

Latest revision as of 17:49, 24 October 2014

Intro

The S3 protocol is delivered via gateways to the web object scaler (WOS) nodes. Currently there are 2 WOS nodes and 5 gateway nodes in production.

WOS

Each WOS nodes has 2 10 GE interfaces. Currently only a single 10 GE connection is active. IP adresses and cabling have been reserved for connecting the second interfaces (link aggregration). The 2 WOS nodes are logically in 2 different zones though physically they are in the same rack. 

wos-2-a 141.52.220.80
wos-2-b 141.52.220.81 (not active)
wos-3-a 141.52.220.82
wos-3-b 141.52.220.83 (not active)

S3 Gateways

The S3 gateway is installed on 5 servers: 

141.52.220.75
141.52.220.76
141.52.220.77
141.52.220.78
141.52.220.79

Node 141.52.220.79 is the management node, where several administration tools are installed:

  • accesskeyctl
  • adminctl
  • configctl

Administrators can manage and monitor the activities on WOS Access S3 platform through the Web UI.

DDNwebUI.png

Managing credentials

The tool for managing credentials is DDN's accesskeyctl. The following actions are possible:

List all existing keys: 

[root@node31 ~]# accesskeyctl -l

      Access Key                         Secret Key                                     UUID                         Tag          Enabled
--------------------------+------------------------------------------+------------------------------------------+----------------+---------
AKIA1DG6UP1704Y60RV7      | 4yuZGOuH8g+89t8+0mE6ZNj4/6mgSFlCJee2aOgR | 89aa4c56229e5c821f58f4f7a70ec4d3f424f85f |         user01 |      yes
AnonymousUser             |                                          | 82acc9bba799fe5766c16a5f2229d9cbfe0e1b8c |     (reserved) |      yes
S3AllUsersGroup           |                                          | 3ee2545bf7032ff442e4610e0861a9387123416b |     (reserved) |      yes
S3AuthenticatedUsersGroup |                                          | 83038be711ef79522283f9b1b240dbc5316ea8e6 |     (reserved) |      yes

Generate a new pair of keys: 

[root@node31 ~]# accesskeyctl -g tag

Credentials generated with tag 'tag':
  accessKey='AKIA2UBKM1AODG1L178D'
  secretKey='+4urJWrORF1EEFdQz9hmlLO6ptV5bxHAALiSuq8b'

Enable/disable credentials: 

[root@node31 ~]# accesskeyctl --disable AKIA2UBKM1AODG1L178D
Credentials for AccessKey 'AKIA2UBKM1AODG1L178D' have been disabled.

      Access Key                         Secret Key                                     UUID                         Tag          Enabled
--------------------------+------------------------------------------+------------------------------------------+----------------+---------
AKIA1DG6UP1704Y60RV7      | 4yuZGOuH8g+89t8+0mE6ZNj4/6mgSFlCJee2aOgR | 89aa4c56229e5c821f58f4f7a70ec4d3f424f85f |         user01 |      yes
AKIA2UBKM1AODG1L178D      | +4urJWrORF1EEFdQz9hmlLO6ptV5bxHAALiSuq8b | 42a06e41566a18a530e78c63123482939f70b960 |            tag |       no
AnonymousUser             |                                          | 82acc9bba799fe5766c16a5f2229d9cbfe0e1b8c |     (reserved) |      yes
S3AllUsersGroup           |                                          | 3ee2545bf7032ff442e4610e0861a9387123416b |     (reserved) |      yes
S3AuthenticatedUsersGroup |                                          | 83038be711ef79522283f9b1b240dbc5316ea8e6 |     (reserved) |      yes

Change tag associated with an access key: 

[root@node31 ~]# accesskeyctl --tag AKIA2UBKM1AODG1L178D tag2
Tag for Credentials with AccessKey 'AKIA2UBKM1AODG1L178D' has been successfully updated to 'tag2'.

      Access Key                         Secret Key                                     UUID                         Tag          Enabled
--------------------------+------------------------------------------+------------------------------------------+----------------+---------
AKIA1DG6UP1704Y60RV7      | 4yuZGOuH8g+89t8+0mE6ZNj4/6mgSFlCJee2aOgR | 89aa4c56229e5c821f58f4f7a70ec4d3f424f85f |         user01 |      yes
AKIA2UBKM1AODG1L178D      | +4urJWrORF1EEFdQz9hmlLO6ptV5bxHAALiSuq8b | 42a06e41566a18a530e78c63123482939f70b960 |           tag2 |       no
AnonymousUser             |                                          | 82acc9bba799fe5766c16a5f2229d9cbfe0e1b8c |     (reserved) |      yes
S3AllUsersGroup           |                                          | 3ee2545bf7032ff442e4610e0861a9387123416b |     (reserved) |      yes
S3AuthenticatedUsersGroup |                                          | 83038be711ef79522283f9b1b240dbc5316ea8e6 |     (reserved) |      yes

Regenerate secret key associated with an access key: 

[root@node31 ~]# accesskeyctl --regenerate AKIA2UBKM1AODG1L178D
SecretKey for Credentials with AccessKey 'AKIA2UBKM1AODG1L178D' has been regenerated, new SecretKey is 'fgPanwW9BiVY1uFWdh8ag6DtuwJXW6mvem8cNFwl'.

Managing cluster services

The adminctl tool helps user manage the cluster services. 

[root@node31 ~]# adminctl --status
Executing status operation for all nodes... (1 sec)

Summary

Node   Status Roles
------ ------ --------------------------------------
node27 UP     journal, metanode, monitor
node28 UP     journal, metanode, monitor
node29 UP     journal, metanode, monitor
node30 UP     metanode, monitor, saltmaster, standby
node31 UP     management, monitor, saltmaster

Detail By Role

Journal hadoop-hdfs-journalnode salt-minion
------- ----------------------- -----------
node27  UP                      UP
node28  UP                      UP
node29  UP                      UP 

Management gmetad hadoop-hdfs-namenode hadoop-hdfs-zkfc hbase-master httpd salt-master salt-minion zookeeper-server
---------- ------ -------------------- ---------------- ------------ ----- ----------- ----------- ----------------
node31     UP     UP                   UP               UP           UP    UP          UP          UP

Metanode ddncloud hadoop-hdfs-datanode hbase-master hbase-regionserver httpd salt-minion zookeeper-server
-------- -------- -------------------- ------------ ------------------ ----- ----------- ----------------
node27   UP       UP                   UP           UP                 UP    UP          UP
node28   UP       UP                   UP           UP                 UP    UP          UP
node29   UP       UP                   UP           UP                 UP    UP          UP
node30   UP       UP                   UP           UP                 UP    UP          UP

Monitor gmond salt-minion
------- ----- -----------
node27  UP    UP
node28  UP    UP
node29  UP    UP
node30  UP    UP
node31  UP    UP

Saltmaster salt-master salt-minion
---------- ----------- -----------
node30     UP          UP
node31     UP          UP

Standby hadoop-hdfs-namenode hadoop-hdfs-zkfc salt-minion
------- -------------------- ---------------- -----------
node30  UP                   UP               UP
Retrieved from "https://wiki.scc.kit.edu/lsdf/index.php?title=hidden:S3_on_WOS&oldid=2530"