Hadoop data intensive computing framework: Difference between revisions

From Lsdf
Jump to navigationJump to search
No edit summary
mNo edit summary
 
(One intermediate revision by the same user not shown)
Line 14: Line 14:
All nodes are however SHARED between the different Hadoop tasks and the OpenNebula virtual machines.
All nodes are however SHARED between the different Hadoop tasks and the OpenNebula virtual machines.



[[Setup Hadoop Nodes]]<p>
hadoop installation and maintenance info: http://wiki.scc.kit.edu/lsdf/index.php/Hadoop
[[Hadoop Disk Repair Guide]]

Latest revision as of 11:28, 27 November 2014

The "Hadoop cluster" consists of 58 nodes with 464 physical cores in total, each node having:

  • 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active (16 cores total)
  • 36 GB of RAM
  • 2 TB of disk
  • 1 GE network connection
  • OS Scientific Linux 5.5
    • Linux kernel 2.6.18

plus 2 headnodes, each having:

  • 2 sockets Intel Xeon CPU E5520 @ 2.27GHz, 4 cores each, hyperthreading active (16 cores total)
  • 96 GB of RAM
  • 10 GE network connection
  • OS Scientific Linux 5.5
    • Linux kernel 2.6.18

All nodes are however SHARED between the different Hadoop tasks and the OpenNebula virtual machines.


hadoop installation and maintenance info: http://wiki.scc.kit.edu/lsdf/index.php/Hadoop