Difference between revisions of "Data services"

From Lsdf
Line 5: Line 5:
 
* Hadoop data intensive computing framework
 
* Hadoop data intensive computing framework
   
The "Hadoop cluster" consists of
+
The "Hadoop cluster" consists of '''58 nodes''' with '''464 physical cores''' in total, each node having:
  +
* 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active (16 cores total)
* 58 nodes with 464 physical cores
 
** 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active
 
 
* 36 GB of RAM
 
* 36 GB of RAM
* 2 TB of disk each
+
* 2 TB of disk
 
* 1 GE network connection
 
* 1 GE network connection
 
* OS Scientific Linux 5.5
 
* OS Scientific Linux 5.5
 
** Linux kernel 2.6.18
 
** Linux kernel 2.6.18
  +
plus '''2 headnodes''', each having:
plus
 
  +
* 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active (16 cores total)
* 2 headnodes
 
** 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active
 
 
* 96 GB of RAM
 
* 96 GB of RAM
 
* 10 GE network connection
 
* 10 GE network connection

Revision as of 18:03, 25 May 2012

The services currently provided by LSDF are:

  • Storage, comprising two systems with 1,4 PB and 500 TB
  • Cloud computing: OpenNebula cloud environment
  • Hadoop data intensive computing framework

The "Hadoop cluster" consists of 58 nodes with 464 physical cores in total, each node having:

  • 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active (16 cores total)
  • 36 GB of RAM
  • 2 TB of disk
  • 1 GE network connection
  • OS Scientific Linux 5.5
    • Linux kernel 2.6.18

plus 2 headnodes, each having:

  • 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active (16 cores total)
  • 96 GB of RAM
  • 10 GE network connection
  • OS Scientific Linux 5.5
    • Linux kernel 2.6.18

All nodes are however SHARED between the different Hadoop tasks and the OpenNebula virtual machines.