LSDF Migration to new hardware ############################## Basic installation of OS, recommendations TSM 6.3.X Red Hat Enterprise Linux 5, Update 3 or later Red Hat Enterprise Linux 6, all updates are supported SUSE Linux Enterprise Server 10, Service Pack 2 or later SUSE Linux Enterprise Server 11, all updates are supported GNU C libraries, Version 2.3.3-98.38 and later. TSM 7.1.x Red Hat Enterprise Linux 6, all updates are supported Red Hat Enterprise Linux 7 (starting with TSM v7.1.1.100) SUSE Linux Enterprise Server 11, Service Pack 2 or later service packs SUSE Linux Enterprise Server 12 (starting with TSM v7.1.4.000) ==> Redhat Enterprise Linux 6 (Red Hat Enterprise Linux 7 does not support current TSM version 6.3) IP-Adress for installation: lsdf-tsm01-new.lsdf.kit.edu, 141.52.212.42 , Mask:255.255.255.0, DGW 141.52.212.1 Bonding !!, cabled with Port5 and Port6 on se-LSDF-07-01/1 LACP mode active New alias in DNS: scc-backup-n02.lsdf.kit.edu Management-Netz: scc-backup-n02-mgmt.lsdf.kit.edu, 172.18.95.70 (se-mgr-LSDF-07-01) # cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.8 (Santiago) ==> OK :) 1. Hardware ----------- Rack: LSDF-07-111 Internal Storage 2 x 480GB SSD Raid1 for operating system 5 x Raid1 each 2 x 800GB SSD for filesystems 1. Raid 150 GB /tsm/actlog 400 GB /tsm/archlog 250 GB /tsm/faillog 2. Raid 800 GB /tsm/dbvol0 3. Raid 800 GB /tsm/dbvol1 4. Raid 800 GB /tsm/dbvol2 5. Raid 800 GB /tsm/dbvol3 External Storage: Rack LSDF-07-100 to LSDF-07-108 NetApp E5600, about 300 TB for TSM pools Direct attached via fibre channel 2. Software ----------- 2.1 sg_utils # yum install sg3_utils.x86_64 2.2 QLogic Tools QLogic-SuperInstaller-2016u3F-20161118-6.tgz # rpm -ivh QConvergeConsoleCLI-2.1.00-11.x86_64.rpm # which qaucli /usr/local/bin/qaucli HowTo: General Information: qaucli -pr fc -g HBA Information: qaucli -pr fc -i z.B. 0 Host Information: qaucli -pr fc -z Lun List: qaucli -pr fc -l Target List: qaucli -pr fc -t Adapter statistics: qaucli -pr fc -gs Link Status: qaucli -pr fc -ls Display HBA Parameters: qaucli -pr fc -c 2.3 lin_tape # yum install rpm-build Installed: rpm-build.x86_64 0:4.8.0-55.el6 Dependency Installed: gdb.x86_64 0:7.2-92.el6 patch.x86_64 0:2.6-6.el6 redhat-rpm-config.noarch 0:9.0.3-51.el6 # rpmbuild --rebuild /tmp/lin_tape-3.0.20-src.rpm error: Failed build dependencies: kernel-devel is needed by lin_tape-3.0.20-1.x86_64 glibc-devel is needed by lin_tape-3.0.20-1.x86_64 gcc is needed by lin_tape-3.0.20-1.x86_64 # yum install kernel-devel glibc-devel gcc ==> reboot Does not work, kernel update neccessary ( 2.6.32-696.10.1.el6.x86_64 ) # yum update ==> reboot # rpmbuild --rebuild /tmp/lin_tape-3.0.20-src.rpm # rpm -ivh /root/rpmbuild/RPMS/x86_64/lin_tape-3.0.20-1.x86_64.rpm # rpm -ivh lin_taped-3.0.20-rhel6.x86_64.rpm # rmmod st Add entry in /etc/modprobe.d/blacklist.conf: # Tape Issues blacklist st 2.4 XFS Filesystem # yum install xfsprogs xfsdump ==> xfsprogs-3.1.1-20.el6.x86_64 xfsdump-3.0.4-4.el6_6.1.x86_64 2.5 Portmapper for SSI, access to tape library # yum install portmap rpcbind.x86_64 0:0.2.0-13.el6_9.1 libgssglue.x86_64 0:0.1-11.el6 libtirpc.x86_64 0:0.2.1-13.el6_9 2.6 Multipath drivers for external attached storage, NetApp # yum install device-mapper # yum install device-mapper-multipath # rpm -qa | grep device-mapper device-mapper-1.02.117-12.el6.x86_64 device-mapper-persistent-data-0.6.2-0.1.rc7.el6.x86_64 device-mapper-multipath-0.4.9-100.el6.x86_64 device-mapper-libs-1.02.117-12.el6.x86_64 device-mapper-event-1.02.117-12.el6.x86_64 device-mapper-event-libs-1.02.117-12.el6.x86_64 device-mapper-multipath-libs-0.4.9-100.el6.x86_64 3. SAN 3.1 Overview # qaucli -pr fc -g -------------------------------------------------------------------------------- Host Name : lsdf-tsm01-new OS Type : Linux - Red Hat Enterprise Linux Server release 6.8 x86_64 OS Version : 2.6.32-642.13.1.el6.x86_64 FO API Version : 3.0.1 build9 SDM API Version : v6.04 build13 -------------------------------------------------------------------------------- HBA Model QLE2562 (SN RFD1623L46043): Port 1 WWPN 21-00-00-24-FF-77-7E-E8 (HBA instance 0) Online ==> NetApp Backup-n02_HBA0 ==> Pooldisks Port 2 WWPN 21-00-00-24-FF-77-7E-E9 (HBA instance 1) Online ==> NetApp Backup-n02_HBA1 ==> Pooldisks HBA Model QLE2562 (SN RFD1623L46119): Port 1 WWPN 21-00-00-24-FF-77-7E-E6 (HBA instance 2) Online ==> NetApp Backup-n02_HBA2 ==> Pooldisks Port 2 WWPN 21-00-00-24-FF-77-7E-E7 (HBA instance 3) Online ==> NetApp Backup-n02_HBA3 ==> Pooldisks HBA Model QLE2562 (SN RFD1623L46168): Port 1 WWPN 21-00-00-24-FF-77-7E-88 (HBA instance 4) Link Down K-017-046 Port 2 WWPN 21-00-00-24-FF-77-7E-89 (HBA instance 5) Link Down K-026-564 HBA Model QLE2562 (SN RFD1623L46173): Port 1 WWPN 21-00-00-24-FF-77-7E-CE (HBA instance 8) Online K-009-421 Backup-n02_HBA8 ==> Tape Port 2 WWPN 21-00-00-24-FF-77-7E-CF (HBA instance 9) Online K-017-749 Backup-n02_HBA9 ==> Tape HBA Model QLE2562 (SN RFD1623L46239): Port 1 WWPN 21-00-00-24-FF-77-7F-14 (HBA instance 6) Online K-009-419 Backup-n02_HBA6 ==> Tape Port 2 WWPN 21-00-00-24-FF-77-7F-15 (HBA instance 7) Online K-009-405 Backup-n02_HBA7 ==> Tape -------------------------------------------------------------------------------- Total QLogic HBA(s) : 5 3.2 Tapedrives Zoning like LSDF_BA01_01 and LSDF_BA01_02, tape drives are distributed on 4 ports. WWPN 21:00:00:24:FF:77:7F:14 (HBA instance 6) Alias Backup-n02_HBA6 with tape drives: IU1_Frame1_Row5 IU1_Frame1_Row6 IU1_Frame1_Row7 NLIBSTK02_1_0_1_7 WWPN 21:00:00:24:FF:77:7F:15 (HBA instance 7) Alias Backup-n02_HBA7 with tape drives: IU1_Frame4_Row1 IU1_Frame4_Row2 IU1_Frame4_Row3 IU1_Frame4_Row4 NLIBSTK02_1_0_1_2 WWPN 21:00:00:24:FF:77:7E:CE (HBA instance 8) Alias Backup-n02_HBA8 with tape drives: IU1_Frame4_Row5 IU1_Frame4_Row6 IU1_Frame4_Row7 IU1_Frame4_Row8 NLIBSTK02_1_0_1_3 WWPN 21:00:00:24:FF:77:7E:CF (HBA instance 9) Alias Backup-n02_HBA9 with tape drives: IU1_Frame4_Row9 IU1_Frame4_Row10 IU1_Frame4_Row11 IU1_Frame4_Row12 3.2.1 HBA Settings for tape Display HBA Parameters with # qaucli -pr fc -c Recommended: Enable FC Tape Support :default is Enabled => OK Enable Target Reset :default is Enabled => Disabled ==> qaucli -n EnableTargetReset 0 Done for all HBA Instances (6-9) with tape drives 3.2.2 Persistent Binding # cat /etc/udev/rules.d/98-lin_tape.rules KERNEL=="IBMchanger*[!n]", SYSFS{serial_num}=="0000078143560403",SYMLINK="lin_tape/changer_0000078143560403" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A18AE",SYMLINK="lin_tape/tapedrive_00078A18AE" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A1E49",SYMLINK="lin_tape/tapedrive_00078A1E49" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078ACDC2",SYMLINK="lin_tape/tapedrive_00078ACDC2" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A1E4C",SYMLINK="lin_tape/tapedrive_00078A1E4C" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A1890",SYMLINK="lin_tape/tapedrive_00078A1890" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078ACE91",SYMLINK="lin_tape/tapedrive_00078ACE91" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078ACE98",SYMLINK="lin_tape/tapedrive_00078ACE98" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078ACE8E",SYMLINK="lin_tape/tapedrive_00078ACE8E" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A18AC",SYMLINK="lin_tape/tapedrive_00078A18AC" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A186D",SYMLINK="lin_tape/tapedrive_00078A186D" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A1BCA",SYMLINK="lin_tape/tapedrive_00078A1BCA" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A1C0C",SYMLINK="lin_tape/tapedrive_00078A1C0C" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078F0934",SYMLINK="lin_tape/tapedrive_00078F0934" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A1891",SYMLINK="lin_tape/tapedrive_00078A1891" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A18B2",SYMLINK="lin_tape/tapedrive_00078A18B2" MODE="0666" KERNEL=="IBMtape*[!n]", SYSFS{serial_num}=="00078A1E48",SYMLINK="lin_tape/tapedrive_00078A1E48" MODE="0666" # udevadm trigger # ll /dev/lin_tape/ lrwxrwxrwx 1 root root 14 Oct 11 08:31 changer_0000078143560403 -> ../IBMchanger1 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A186D -> ../IBMtape9 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A1891 -> ../IBMtape8 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A18AC -> ../IBMtape7 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A18AE -> ../IBMtape2 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A1BCA -> ../IBMtape4 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A1C0C -> ../IBMtape6 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A1E49 -> ../IBMtape3 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078A1E4C -> ../IBMtape0 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078ACDC2 -> ../IBMtape1 lrwxrwxrwx 1 root root 12 Oct 11 08:31 tapedrive_00078ACE91 -> ../IBMtape10 lrwxrwxrwx 1 root root 12 Oct 11 08:31 tapedrive_00078ACE98 -> ../IBMtape11 lrwxrwxrwx 1 root root 11 Oct 11 08:31 tapedrive_00078F0934 -> ../IBMtape5 # cat /etc/udev/rules.d/65-tsm-t10kd-tape.rules #tape devices only for t10kd drives KERNEL=="sg*", SUBSYSTEM=="scsi_generic", PROGRAM=="/lib/udev/scsi_id --w /dev/sg%n", RESULT=="3500104f000bc03c8", OWNER="tsminst1", GROUP="tsmsrvrs", SYMLINK="tsmscsi/mt0" KERNEL=="sg*", SUBSYSTEM=="scsi_generic", PROGRAM=="/lib/udev/scsi_id --w /dev/sg%n", RESULT=="3500104f000bc03bc", OWNER="tsminst1", GROUP="tsmsrvrs", SYMLINK="tsmscsi/mt1" KERNEL=="sg*", SUBSYSTEM=="scsi_generic", PROGRAM=="/lib/udev/scsi_id --w /dev/sg%n", RESULT=="3500104f000bc03b9", OWNER="tsminst1", GROUP="tsmsrvrs", SYMLINK="tsmscsi/mt2" # ls -la /dev/tsmscsi lrwxrwxrwx 1 root root 7 Oct 18 07:50 mt0 -> ../sg13 ==> 579004003143 3500104f000bc03c8 1,0,1,2 579004003143 mt0 lrwxrwxrwx 1 root root 7 Oct 18 07:50 mt1 -> ../sg12 ==> 579004003156 3500104f000bc03bc 1,0,1,3 579004003156 mt1 lrwxrwxrwx 1 root root 7 Oct 18 07:50 mt2 -> ../sg29 ==> 579004007136 3500104f000bc03b9 1,0,1,7 579004007136 mt2 ## Formerly, now outdated and obsolent ## ##Generate devices in /dev/tsmscsi/ ## ## # cd /opt/tivoli/tsm/devices/bin;./autoconf ## ##Add entry in /etc/rc.local ## ## # Genrate special devices in /dev/tsmscsi, STK drives ## /opt/tivoli/tsm/devices/bin/autoconf 3.2.3 SSI for ACSLS # cd /opt/tivoli/tsm/devices/bin; cp rc.acs_ssi rc.acs_ssi.org Add these lines in /opt/tivoli/tsm/devices/bin/rc.acs_ssi # KIT ACSLS Server ACSLS_SERVER_HOSTNAME=scc-stk-lib01.tmn.scc.kit.edu; export ACSLS_SERVER_HOSTNAME ###OLD###SSI_HOSTNAME=lsdf-tsm01.tmn.scc.kit.edu ; export SSI_HOSTNAME SSI_HOSTNAME=lsdf-tsm01.lsdf.kit.edu ; export SSI_HOSTNAME # service rpcbind start # rpcinfo program version netid address service owner 100000 4 tcp6 ::.0.111 portmapper superuser 100000 3 tcp6 ::.0.111 portmapper superuser ............ Add entry in /etc/rc.local # Daemon for STK Library /opt/tivoli/tsm/devices/bin/rc.acs_ssi quiet Test if connection to ACSLS works # cd /opt/tivoli/tsm/devices/bin # ./lbtest 3.3 Disc NetApp Storagesystem direct wired with controller A/B Port 3/4 Hostname Controller A: scc-bapo-n03a.scc.kit.edu Host Port 1 K-009-440 20:12:00:80:e5:43:76:5c zoned with LSDF-TSM01 2 K-009-432 20:22:00:80:e5:43:76:5c zoned with LSDF-TSM02 3 K-009-405 20:32:00:80:e5:43:76:5c direct wired LSDF-07-111 4 K-017-749 20:42:00:80:e5:43:76:5c direct wired LSDF-07-111 Hostname Controller B: scc-bapo-n03b.scc.kit.edu Host Port 1 K-009-436 20:13:00:80:e5:43:76:5c zoned with LSDF-TSM01 2 K-009-445 20:23:00:80:e5:43:76:5c zoned with LSDF-TSM02 3 K-009-419 20:33:00:80:e5:43:76:5c direct wired LSDF-07-111 4 K-009-421 20:43:00:80:e5:43:76:5c direct wired LSDF-07-111 Configuration: # touch /etc/multipath.conf # /etc/init.d/multipathd start # chkconfig multipathd on # multipath oder /sbin/mpathconf # reboot # multipath -l ... 360080e50004372680000036e59dd6d22 dm-6 NETAPP,INF-01-00 size=305T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 rdac' wp=rw |-+- policy='round-robin 0' prio=0 status=active | |- 2:0:0:0 sde 8:64 active undef unknown | `- 0:0:0:0 sda 8:0 active undef unknown `-+- policy='round-robin 0' prio=0 status=enabled |- 3:0:0:0 sdg 8:96 active undef unknown `- 1:0:0:0 sdc 8:32 active undef unknown .... Edit /etc/multipath.conf ... .... # Internal IBM ServeRaid blacklist { wwid 3600605b000b20d10201ccf821951b0c8 wwid 3600605b000b20d10201ccef911292fa6 wwid 3600605b000b20d10201ccf401565ac32 wwid 3600605b000b20d10201ccea60c32a496 wwid 3600605b000b20d10201ccfbd1cd9fe24 } multipaths { multipath { wwid 360080e50004372680000036e59dd6d22 alias NetApp_Pool } multipath { wwid 360080e500043765c000002205743c570 alias NetApp_DBB } } 4. Filesystems Recommendations on Linux: Use the ext3 !! file system for the database and recovery log. You can use the ext4 file system for storage pool data. But we are using xfs for the pools. 4.1 OS 2 x 480GB SSD, Raid1 and for /home/tsminst1/tsm_actlogmirror 4.2 TSM Database 5 x Raid1 each 2 x 800GB SSD a. Raid 150 GB /tsm/actlog 200 GB /tsm/archlog 350 GB /tsm/failarch b. Raid 800 GB /tsm/dbvol0 c. Raid 800 GB /tsm/dbvol1 d. Raid 800 GB /tsm/dbvol2 e. Raid 800 GB /tsm/dbvol3 a.) # parted --align optimal /dev/sdf (parted) mklabel gpt (parted) mkpart primary 0 150GB (parted) mkpart primary 150GB 400GB (parted) mkpart primary 400GB 799GB (parted) print (parted) quit # mkfs.ext3 /dev/sdf1 # mkfs.ext3 /dev/sdf2 # mkfs.ext3 /dev/sdf3 # e2label /dev/sdf1 actlog # e2label /dev/sdf2 archlog # e2label /dev/sdf3 failarch b.) - d.) # parted --align optimal /dev/sdg (parted) mklabel gpt (parted) mkpart primary 0 799GB (parted) print # mkfs.ext3 /dev/sdg1 # e2label /dev/sdg1 dbvol0 # parted --align optimal /dev/sdh (parted) mklabel gpt ... .... # e2label /dev/sdj1 dbvol3 # mkdir -p /tsm/actlog # mkdir /tsm/archlog # mkdir /tsm/failarch # OLD! /tsm/ddn_tsm_db_backup/failarch # mkdir /tsm/dbvol0 # mkdir /tsm/dbvol1 # mkdir /tsm/dbvol2 # mkdir /tsm/dbvol3 Edit /etc/fstab: LABEL=actlog /tsm/actlog ext3 defaults 0 0 LABEL=archlog /tsm/archlog ext3 defaults 0 0 LABEL=failarch /tsm/failarch ext3 defaults 0 0 LABEL=dbvol0 /tsm/dbvol0 ext3 defaults 0 0 LABEL=dbvol1 /tsm/dbvol1 ext3 defaults 0 0 LABEL=dbvol2 /tsm/dbvol2 ext3 defaults 0 0 LABEL=dbvol3 /tsm/dbvol3 ext3 defaults 0 0 # mount -a # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup-lv_root 50G 3.5G 44G 8% / tmpfs 63G 0 63G 0% /dev/shm /dev/sde1 477M 115M 337M 26% /boot /dev/mapper/VolGroup-lv_home 386G 71M 366G 1% /home /dev/sdf1 138G 188M 131G 1% /tsm/actlog /dev/sdf2 230G 188M 218G 1% /tsm/archlog /dev/sdf3 366G 195M 347G 1% /tsm/failarch /dev/sdg1 733G 197M 696G 1% /tsm/dbvol0 /dev/sdh1 733G 197M 696G 1% /tsm/dbvol1 /dev/sdi1 733G 197M 696G 1% /tsm/dbvol2 /dev/sdj1 733G 197M 696G 1% /tsm/dbvol3 Performance test because "parted --align optimal" or "--align cylinder" notifies strange messages: dd if=/dev/zero bs=4k count=1310720 of=/tsm/actlog/testfile ==> Schreiben (5.4 GB) copied, 5.28658 s, 1.0 GB/s dd if=/dev/zero bs=4k count=1310720 of=/tsm/dbvol0/testfile ==> Schreiben (5.4 GB) copied, 5.28658 s, 1.0 GB/s dd if=/tsm/actlog/testfile bs=4k count=1310720 of=/dev/null ==> Lesen=5,5 GB/s ==> OK ! 4.2.1 User ID for the upgraded server instance # groupadd tsmsrvrs # useradd -d /home/tsminst1 -m -g tsmsrvrs -s /bin/bash tsminst1 # passwd tsminst1 # chown tsminst1:root /tsm # cd /tsm # chown tsminst1:tsmsrvrs actlog # chown tsminst1:tsmsrvrs archlog # chown tsminst1:tsmsrvrs failarch # chown tsminst1:tsmsrvrs dbvol0 # chown tsminst1:tsmsrvrs dbvol1 # chown tsminst1:tsmsrvrs dbvol2 # chown tsminst1:tsmsrvrs dbvol3 # su - tsminst1; mkdir actlogmirror # ls -la /home/tsminst1/ .... drwxr-xr-x 2 tsminst1 tsmsrvrs 4096 Sep 4 16:16 actlogmirror 4.3 Filesystems for TSM pools and database backup to disc SanTricity: Maximum volume size only 64 TB ?! Upgrade neccessary of the controller firmware, afterwards created LUN with 300TB, add Mapping on Storage-Array 4.3.1 TSM Pool # parted --align optimal /dev/mapper/NetApp_Pool (parted) mklabel gpt (parted) mkpart primary 0% 100% (parted) print Model: Linux device-mapper (multipath) (dm) Disk /dev/mapper/NetApp_Pool: 335TB Sector size (logical/physical): 512B/4096B Partition Table: gpt Number Start End Size File system Name Flags 1 1049kB 335TB 335TB primary (parted) quit Information: You may need to update /etc/fstab. # mkfs.xfs -L NetApp_Pool /dev/mapper/NetApp_Pool # cd /tsm; mkdir pool ; chown tsminst1:tsmsrvrs pool Add entry in /etc/fstab ... LABEL=NetApp_Pool /tsm/pool xfs defaults 0 0 # mount /tsm/pool 4.3.2 Filesystem for database backup # parted --align optimal /dev/mapper/NetApp_DBB (parted) mklabel gpt (parted) mkpart primary 0% 100% (parted) quit Information: You may need to update /etc/fstab. # mkfs.xfs -Lf db_backup /dev/mapper/NetApp_DBBp1 # cd /tsm; mkdir db_backup ; chown tsminst1:tsmsrvrs pool Add entry in /etc/fstab ... LABEL=db_backup /tsm/pool xfs defaults 0 0 # mount /tsm/db_backup 5. Spectrum Protect / TSM 5.1 Installing TSM Version 6.3.5.100 Current hostname is lsdf-tsm01-new !!! First install 6.3.0 because of the license, then upgrade to 6.3.5.100 Store binaries in /tmp and execute ./prereqcheck.bin, preinstallation checks occur # cd /tmp/; ./prereqcheck.bin - The check of the /etc/hosts file failed, which can produce invalid system prerequisites results. Edit the /etc/hosts file to add IP address and host name of the target system. ** # cat /etc/hosts .. 141.52.212.41 lsdf-tsm01.lsdf.kit.edu lsdf-tsm01 141.52.212.42 lsdf-tsm01-new.lsdf.kit.edu lsdf-tsm01-new - DB2 9.7 installation minimum requirements: Linux compat-libstdc++ (libstdc++so.6 is required for DB2 database servers and clients). # yum install libstdc++.so.6 Installed: libstdc++.i686 0:4.4.7-18.el6 Dependency Installed: libgcc.i686 0:4.4.7-18.el6 - Deployment Engine installation minimum requirements: libstdc++so.5 (32-bit and 64-bit) # yum install libstdc++.so.5 compat-libstdc++-33.i686 0:3.2.3-69.el6 # yum install /usr/lib64/libstdc++.so.5 Installed: compat-libstdc++-33.x86_64 0:3.2.3-69.el6 # yum install libaio.so.1 Installed: libaio.i686 0:0.3.107-10.el6 - The Korn shell (ksh) is not installed on your system. For Tivoli Storage Manager server database upgrades and uninstallations, ensure that the ksh is installed here: /bin/ksh. # yum install ksh NOW all is fine !!! # cd /tmp; ./install.bin -i console 2- IBM Tivoli Storage Manager Extended Edition Will your server perform any of the following? - LAN-free operations - Share a library and act as either the library manager or the library client 1- Yes 1- Tivoli Storage Manager server 2- Tivoli Storage Manager server languages 3- Tivoli Storage Manager license 4- Tivoli Storage Manager devices ... But installation fails, try to install 6.3.1.500 cd /tmp/6.3.1.500;./install.bin -i console ... Success !! Installation Complete --------------------- The following components were successfully installed: Tivoli Storage Manager server DB2 9.7 Global Secure Toolkit 8 Tivoli Storage Manager client api 64 bit Tivoli Storage Manager devices BUT TIVsm-license is missing [root@lsdf-tsm01-new]# rpm -ivh /tmp/6.3.0/COI/PackageSteps/TSM_License/FILES/TIV-TSMLIC-Linux/TIVsm-license-6.3.0-0.x86_64.rpm [root@lsdf-tsm01-new tmp]# rpm -qa | grep TIV TIVsm-server-6.3.5-100.x86_64 TIVsm-tsmscsi-6.3.5-100.x86_64 TIVsm-license-6.3.0-0.x86_64 5.2 Configuration of Instance # cd /opt/tivoli/tsm/server/bin # ./dsmicfgx -i GUI Output: Issuing cmd: . ~/sqllib/db2profile; /opt/tivoli/tsm/server/bin/dsmserv FORMAT dbfile=\"/home/tsminst1/dbfile.1\" activelogdir=\"/tsm/actlog\" archlogdir=\"/tsm/archlog\" mirrorlogdir=\"/home/tsminst1/tsm_actlogmirror\" archfailoverlogdir=\"/tsm/failarch\" activelogsize=131072 >Format.Out 2>&1 on id tsminst1 Wed Oct 11 16:25:01 CEST 2017 com.tivoli.dsm.ServerConfig.ProcessMonitor.run(): enter, Monitoring file /home/tsminst1/Format.Out ==> ANR0162W Supplemental database diagnostic information: -970:SQLSTATE 55009: The system attempted to write to a read-only file or a write-protected storage medium. :-970 (SQL0970N The system attempted to write to a read-only file. SQLSTATE=55009 ). Failed (sqlca.sqlcode = -970 ) to create database TSMDB1 ANR2678E Server database format failed. ==> Look in /home/tsminst1/sqllib/db2dump/db2diag.log for "Target" ...A total of 2 analysis will be performed : - User info - Path access permission Target file = /tmp/SQLDIR.LK0 ==> /tmp has wrong permissions !!?? drwxr-xr-x. 7 root root 4096 Oct 12 12:44 tmp should be drwxrwxrwt 37 root root 4096 Oct 12 12:44 tmp # chmod 1777 tmp Execute command again: # ./dsmicfgx -i GUI ... Copy dsmserv.opt from the old server to /home/tsminst1/dsmserv.opt, alter settings for directories !! Copy Volhist.dat and devconf.dat from the old server to /home/tsminst1 and /tsm/archlog Increase the number of "open files" (default=1024): # ulimit -n 16384 Edit /etc/security/limits.d/90-nproc.conf * soft nproc 16348 5.3 Restore the database on the new server First make a backup on the old server Protect: LSDF-BA-01> backup db type=FULL devclass=ERMM_LTO After successful backup halt the server Cartridges UV4192L5, UV0780L5 Excert from activity-log: 10/17/2017 08:18:45 ANR0985I Process 6828 for Database Backup running in the BACKGROUND completed with completion state SUCCESS at 08:18:45 AM. (SESSION: 459506, PROCESS: 6828) 10/17/2017 12:55:03 ANR0985I Process 6846 for Database Backup running in the BACKGROUND completed with completion state SUCCESS at 12:55:03 PM. (SESSION: 460757, PROCESS: 6846) Protect: LSDF-BA-01> halt Now alter Network-Settings on both servers and reboot. On the old server alter the IP address to lsdf-tsm-test.lsdf.kit.edu, 141.52.212.42 and edit /etc/inittab to prevent following from starting at boottime ... #####ssi:23:once:/opt/tivoli/tsm/devices/bin/rc.acs_ssi quiet #####tsm0:235:once:/opt/tivoli/tsm/server/bin/rc.dsmserv -u tsminst1 -i /tsm/server -q >/dev/console 2>&1 & # Tivoli Storage Manager #####tsm:35:respawn: env LANG=en_US /usr/bin/dsmc schedule -servername=localbackup On the new server alter IP adresses to the the old server Edit /etc/sysconfig/network .. NETWORKING=yes NETWORKING_IPV6=no #HOSTNAME=lsdf-ba-01 HOSTNAME=lsdf-tsm01 Edit /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 #IPADDR=141.52.212.42 IPADDR=141.52.212.41 NETMASK=255.255.255.0 ONBOOT=yes BOOTPROTO=none USERCTL=no NM_CONTROLLED=no !! VERY IMPORTANT !! Edit /home/sqllib/db2nodes.cfg ==> New hostname # cat /home/tsminst1/sqllib/db2nodes.cfg 0 lsdf-tsm01 0 Edit /etc/services ... ..... DB2_SVCE 51500/tcp # This is the connection port for instance DB2 ==> # reboot ##!! DB2 had not startet yet clean Create a file /home/tsminst1/databaselist # cat /home/tsminst1/databaselist /tsm/dbvol0 /tsm/dbvol1 /tsm/dbvol2 /tsm/dbvol3 # su - tsminst1 # opt/tivoli/tsm/server/bin/dsmserv -i /home/tsminst1 restore db todate=today on=/home/tsminst1/databaselist -preview=yes Two problems occured: File /home/tsminst1/devconf.dat has a wrong entry, line "Define Server", the password was not in the same line, but with "Set Serverpassword" 2 lines below File /home /tsminst1/dsmserv.opt has a entry for failarchive, delete it and add the failarchive after the restore of the database. Corresponding error messages: ..... ANR0226S The database ID file could not be found for server startup. ANR0172I rdbicrst.c(1754): Error encountered performing action Update Various Log Path DB CFG Parameter. ANR0162W Supplemental database diagnostic information: -5099:SQLSTATE 08004: The application server rejected establishment of the connection. :-5099 (SQL5099N The value "/tsm/failarch/failarch" indicated by the database configuration parameter "FAILARCHPATH" is not valid, reason code "2". SQLSTATE=08- 004 ). ANR0172I rdbicrst.c(1766): Error encountered performing action set DB2 parameters. ANR0162W Supplemental database diagnostic information: -5099:SQLSTATE 08004: The application server rejected establishment of the connection. :-5099 (SQL5099N The value "/tsm/failarch/failarch" indicated by the database configuration parameter "FAILARCHPATH" is not valid, reason code "2". SQLSTATE=08- 004 ). ANR4635I Point-in-time database restore complete, restore date 10/17/2017 11:59:59 PM. ANR0369I Stopping the database manager because of a server shutdown. ==> Edit dsmserv.opt and delete the entry ARCHFAILOVERLOGDirectory /tsm/failarch Do it again without preview=yes ! # su - tsminst1 # opt/tivoli/tsm/server/bin/dsmserv -i /home/tsminst1 restore db todate=today on=/home/tsminst1/databaselist Startup TSM server at boot time: # cp /opt/tivoli/tsm/server/bin/dsmserv.rc /etc/rc.d/init.d/tsminst1 # chkconfig --list tsminst1 service tsminst1 supports chkconfig, but is not referenced in any runlevel (run ’chkconfig --add tsminst1’) # chkconfig --add tsminst1 # chkconfig --list tsminst1 tsminst1 0:off 1:off 2:off 3:off 4:off 5:off 6:off # chkconfig --level 35 tsminst1 on # chkconfig --list tsminst1 tsminst1 0:off 1:off 2:off 3:on 4:off 5:on 6:off Now command "service tsminst1 status|start|stop" is available. First test run, register licenses: # su - tsminst1 You have to be in the /home/tsminst1 directory # /opt/tivoli/tsm/server/bin/dsmserv ... ..... TSM:LSDF-BA-01> register license file=/opt/tivoli/tsm/server/bin/tsmee.lic TSM:LSDF-BA-01> halt # service tsminst1 start # ps -ef | grep dsmserv root 13404 1 0 17:04 pts/0 00:00:00 su - tsminst1 -c /opt/tivoli/tsm/server/bin/dsmserv -i /home/tsminst1/tsminst1 -q tsminst1 13498 13404 0 17:04 ? 00:00:00 /opt/tivoli/tsm/server/bin/dsmserv -i /home/tsminst1/tsminst1 -q ==> Does not work !!! :( Edit /etc/init.d/tsminst1 Alter line instance_dir="${instance_home}/tsminst1" to instance_dir="${instance_home}" # service tsminst1 start # ps -ef | grep dsmserv root 14765 1 0 17:17 pts/0 00:00:00 su - tsminst1 -c /opt/tivoli/tsm/server/bin/dsmserv -i /home/tsminst1 -q tsminst1 14859 14765 0 17:17 ? 00:00:00 /opt/tivoli/tsm/server/bin/dsmserv -i /home/tsminst1 -q 6. TSM Client First stop localbackup on the old server Edit /etc/inittab #####tsm:35:respawn: env LANG=en_US /usr/bin/dsmc schedule -servername=localbackup Install the client on the new server # rpm -Uvh TIVsm-BA.x86_64.rpm TIVsm-API64.x86_64.rpm gskcrypt64-8.0.50.66.linux.x86_64.rpm gskssl64-8.0.50.66.linux.x86_64.rpm # rpm -qa | grep TIV ... TIVsm-API64-8.1.0-2.x86_64 TIVsm-BA-8.1.0-2.x86_64 Add to /opt/tivoli/tsm/client/ba/bin/dsm.sys: managedservices schedule domain / exclude.dir /tmp exclude.dir /var/log domain /boot domain /home exclude.dir /home/tsminst1/actlogmirror domain /archlog include "/archlog/.../*" DATABASE_PROTECT 7. crontab # SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC SCC # Adjusting date and time 00 12 * * * /usr/sbin/ntpdate -u -s -t 2 dns-int-03.kit.edu 1>/dev/null 2>&1 # database protection , backup /archlog all 15 minutes to tape on NB03 15 * * * * dsmc i /archlog -se=backup 1>/dev/null 2>/dev/null # Backup System nach CN ==> NOW with Scheduler ##10 18 * * * /usr/bin/dsmc incr -se=localbackup 1>/dev/null 2>/dev/null # Speicherplatzbedarf HITS 00 09 1 * * /usr/tsm/bin/hits_auditocc.script 1>/dev/null 2>/dev/null # TSM-Kontakte 00 10 28 * * /usr/tsm/bin/contact.script 1>/dev/null 2>/dev/null # KS: Icinga Monitoring von /var/log/messages */10 * * * * /usr/local/icinga/bin/check_logfile_tape.sh 8. Mail edit /etc/aliases # Person who should get root's mail .. .... root: dataprotection@lists.kit.edu 9. yum autoupdate NOT ACTIVE NOW # yum install yum-cron Edit /etc/sysconfig/yum-cron cp etc/sysconfig/yum-cron etc/sysconfig/yum-cron.org YUM_PARAMETER="-x kernel* -x *firmware* -x redhat-release* -x glibc*" ... ..... MAILTO="root" CHECK_ONLY="true" By default, the yum-cron service is disabled and needs to be activated and started manually: # chkconfig yum-cron on # service yum-cron start Enabling nightly yum update: [ OK ] To verify the status of the service, run the following command: # service yum-cron status The script included in the yum-cron package can be configured to change the extent and frequency of the updates, as well as to send notifications to e-mail. To customize yum-cron, edit the /etc/sysconfig/yum-cron file. 10. Limits Edit /etc/security/limits.conf * soft nofile 16384 * hard nofile 16384 11. Logrotate OK 12. Installing SNMP for Icinga: ?????????? # yum install net-snmp installed ==> net-snmp.x86_64 1:5.5-54.el6 In /etc/cron.d stehen die Checks Copy /etc/icinga.conf from the old server to the new one. 13. Logwatch deinstallieren # rpm -e logwatch-7.3.6-52.el6.noarch 14. Nacharbeiten 1. Um Verwechselungen zu verhindern bitte im DNS folgende Änderungen vornehmen lsdf-tsm01-new.lsdf.kit.edu (141.52.212.42) bitte in lsdf-tsm-test.lsdf.kit.edu umbenennen lsdf-tsm01.lsdf.kit.edu (141.52.212.41) einen Alias scc-backup-n02.lsdf.kit.edu hinzufügen 2. Put this in dsmserv.opt and restart the TSM server MIRRORLOGDIRECTORY /home/tsminst1/actlogmirror ARCHFAILOVERLOGDIRECTORY /tsm/failarch 3. Nmon cp /tmp/nmon16e_x86_rhel65 /usr/bin/nmon