Setup Hadoop Nodes

From Lsdf
Revision as of 10:02, 25 August 2014 by C Pfeiler (talk | contribs)
Jump to navigationJump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
  • setup node to pxe boot
  • SSH to ics.lsdf.kit.edu (ask Ursula Epting)
  • Go to directory /srv/tftp/pxelinux.cfg/lsdf/
  • Rename link mv hph-c-x.disabled hph-c-x
  • Go to directory /srv/tftp/pxelinux.cfg/
  • Create link ln -s lsdf/hph-c-0XX YYYYYYYY where XX is the node number, YYYYYYYY is the IP address (sedecimal)
  • Reboot client, monitor on console
  • After installation finished, invoke /usr/local/sbin/cfe-enable-client.sh hph-c-0XX.lsdf.kit.edu
  • Check /etc/cfengine/cfrun.hosts whether the node is included
  • Invoke cfrun hph-c-0XX.lsdf.kit.edu -- -q
  • Go to node, check if .ssh directory exists.
  • Invoke cfagent --verbose --debug locally on client, if not
  • Invoke on client yum --disableexcludes=all update
  • /usr/local/sbin/postinst-packages-cleanup.sh
  • yum install -y --disableexcludes=all hadoop-0.20 hadoop-0.20-datanode hadoop-0.20-tasktracker hadoop-0.20-fuse.x86_64 hadoop-0.20-libhdfs.x86_64 hadoop-0.20-native.x86_64 hadoop-0.20-pipes.x86_64 hadoop-0.20-sbin.x86_64 hadoop-hbase hadoop-hbase-regionserver hadoop-zookeeper
  • /usr/local/sbin/postinst-network-bridge-setup.sh
  • The cfengine configuration misses two steps:

chkconfig --list jexec and ln -s /usr/java/jdk1.6.0_41 /usr/java/latest

Retrieved from "https://wiki.scc.kit.edu/lsdf/index.php?title=Setup_Hadoop_Nodes&oldid=2357"