November 28, 2010

Few usefull 3494 library commands


1. Query Library Data:
mtlib -l /dev/lmcp0/ -qL | more

2.Query Volume Count:
mtlib -l /dev/lmcp0/ -qK

3.Query Statistical Data:
mtlib -l /dev/lmcp0/ -qS

4.Query Volume Inventory:
mtlib -l /dev/lmcp0/ -qI

5.Query Expanded Device Information:
mtlib -l /dev/lmcp0/ -qD -E

6.Query Mounted volumes:
mtlib -l /dev/lmpc0/ -qM

7.Query Reserverd volumes:
mtlib -l /dev/lmcp0/ -qR

Upgrading tsm 6.1 to the next maintenance level

Initial TSM server version:6.1.3.2
Target TSM server version:6.1.4.2
OS Plateform version:AIX 6.1.4.0 64 bit

  • Download tsm server code from IBM ftp site:
  • Halt tsm server. Halting TSM server would automatically stop DB2 server.
  • If after stopping TSM server db2 still keep on running ( process db2sync ) then it can be stopped  using db2stop and If db2stop not working db2stop force can be tried or simply kill db2sync process.
  • I experienced for some reason in some cases halting tsm server does not kill dsmserv process immediately so need to kill dsmserv and then stop tsm db2 server as stated above.
  • Save the downloaded bin file inside a dir and extract code by executing ./6.1.4.2-TIV-TSMALL-AIX.bin.This will extract code inside the same directory.
  • Using root execute following command:
./install.bin -i console

Applying fixes would look similar to fresh tivoli storage manager installation except it would not include license component.

Installed component would include following:

TSM Server
DB2 9.5
TSM Devices
TSM Storage Agent.

November 27, 2010

TSM ba client 6.1.3.2 experiences slow backup performance

TSM ba client version:6.1.3.2
OS plateform version:Windows 2008 R2

I noticed that clients which are at level 6.1.3.2 they backup slow than those which are at 6.2.0.1.
I upgraded clients to version 6.2.0.1 and found improvements in backup performance.
I noticed that clients which are running on windows 2008 R2 with exchange 2010 configured and running are specially backing up slow.

I am planning to upgrade them to the latest fix available at the time which is 6.2.1.3 as taking them upto 6.2.1.0 has not really helped in their filesystem backup performance.

Expire inventory hangs on TSM 6.1.3.2.

TSM Server Version:6.1.3.2
OS Plateform Version:AIX 6.1 64 bit

I recently faced that expire inventory runs very slow on tsm 6.1.3.2 and hangs on some particular node's data expiration. I noticed hanging on some nodes was random in nature. I was expecting windows 2008 system state the culprit and responsible for this mess but I found that it was hanging on some AIX nodes as well. I openned a PMR and was advised by IBM support to upgrade server to the maintenance level 6.1.4.2 as TSM 6.1 expire inventory hangs while processing some grouped objects like windows system state.
After applying this fix expiration inventory hang problem has been eliminated but expiration process was still running very slow which was creating locks on some threads restarting and slowing down some client backup sessions. Further from IBM support I have been given a cumulative fix for this problem which I am due to apply.
During trobleshooting and analysing the problem I ran instrumentation on tsm server and found that "DB2 MFtch Exec" is eating most of the time of a client backup session.
It has been identified as a db2 bug.
Once I apply this fix I'll update the results on this post.

 Applying thix efix has resolved expiration problem completely.

Related APARS:
APAR IC69729 - V6.1 AND V6.2 SERVER CAN EXPERIENCE SLOW EXPIRATION
PROCESSING OF GROUPED OBJECTS 6.1.5 fixhttp://www-01.ibm.com/support/docview.wss?uid=swg1IC69729

APAR IC69156 - V6.1 SERVER CAN EXPERIENCE SLOW EXPIRATION PROCESSING
ON GROUPED OBJECTS fix at 6.1.4
http://www-01.ibm.com/support/docview.wss?uid=swg1IC69156

APAR IC70269 - PERFORMANCE PROBLEMS ON LEVEL 6.1.4.0 OF THE TIVOLI
STORAGE MANAGER SERVER fix 6.1.4.1
http://www-01.ibm.com/support/docview.wss?uid=swg1IC70269

Applying efix and scheduling undocumented expiration command listed below has resolved slow expiration problem completely.Before applying efix I was advised to upgrade tsm server on to the level 6.1.4.3 since fix was built on this code.I also splited TSM DB on 4 separate disks to eliminate disk bottleneck as earlier TSMDB was configured on only one disk and whenever any high I/O intensive process was running it was making this disk 100% utilized.Spliting DB on 4 disks allowed higher I/O operations balacing them on all four disks.
Do not forget to configre tsm db disks into RAID 5 ( recommended raid level for tsm db) as we can't mirror tsb db any more starting with version 6.1.

Expiration command: expi inv dur=600 sk=y restart=no

Information on how to move TSM DB to alternate location can be found in article "Moving TSM V6 database to new directories".


November 23, 2010

Configuring TDP for Exchange 2010 Full VSS Backups

Exchange Server OS Plateform: Windows 2008 R2
TDP for Exchange version: 6.1.2.1
TSM ba client version: 6.2.1.0

TDP for exchange 6.1.2.1 needs ba client version 6.2.1.0 to be installed.
For detailed system requirements for TDP for exchange 6.1 click on below link.

Detailed Documentation available on following link:

Additionnal Software Requirement:
Basic VSS operations require one of the following applications:
–IBM Tivoli Storage Manager Microsoft Exchange VSS Integration Module Version 5.5.2 or later
– IBM Tivoli Storage FlashCopy Manager Version 2.1 or later
If backups are stored directly on Tivoli Storage Manager Server, these applications are not required.
Written procedure is tested on windows 2008 r2 exchange 2010 stand alone environment with
tdp 6.1.2.1 and ba client version 6.2.1.0 backing up directly to TSM server. It assumes default client installation location.

Procedure:
  • Download Windows 64 bit TSM ba client 6.2.1.0 from IBM ftp site and install it. 
  • Register ba client node on TSM server and configure it's backup.
  • Configure CAD and scheduler services for ba client running below commands:
dsmcutil install cad /name:"TSM Client Acceptor" /password:xxxxxxx /cadschedname:"TSM Scheduler Service" /httpport:1581 /autostart:yes /startnow:yes

dsmcutil install /name:"TSM Scheduler Service" /password:tsmadmin /autostart:no /startnow:no
  • Configure WEB Client services using below command:
dsmcutil install remoteagent /name:"TSM Remote Client Agent" /node:xxxxxxx /password:xxxxxx /optfile:dsm.opt" /partnername:"TSM Client Acceptor"
  • Download TDP for Exchange 6.1 from passport advantage and install it. Download TDP for exchange patch 6.1.2.1 from IBM ftp site and apply it on installed base software.
  • Resister Exchange TDP node in TSM server specifically useid=none not configured.
  • Grant proxy node authority to ba client running following command :
grant proxynode target=<exchange tdp node name> agent=<exchange ba client node name>
  • Configure dsm.opt for exchange with following parameters:
NODename          <Exchange node name>
CLUSTERnode       no
COMPRESSIon       Off
PASSWORDAccess    Generate
COMMMethod        TCPip
TCPPort                   1500
TCPServeraddress  xx.xx.xx.xx
TCPWindowsize     63
TCPBuffSize       32
SCHEDMODE             Polling
SCHEDLOGRetention     15,d
ERRORLOGRretention    15,d 
SCHEDLOGNAME        C:\Program Files\Tivoli\TSM\TDPExchange\tdpsched.log
ERRORLOGNAME        C:\Program Files\Tivoli\TSM\TDPExchange\tdperror.log 
  • Configure tdpexc.cfg for exchange with following parameters:
logprun 14
vsspolicy * * full tsm <mgmt_class_name>
localdsmagentnode <ba client  node name>
storemailboxinfo no
STOREMAILBOXINFO NO option would enable fast backup since exchange 2010 takes a lot of time in saving mailbox history file.If this option is selected then at the time of restoration of moved mail box user needs to provide it's location.
  • Edit excfull.smp sample file as per your environment and save it as excfull.cmd.
  • Define a client schedule on tsm server using below sample command:
def sched <domain name>  <sched name> action=command obj="c:\progra~1\tivoli\tsm\tdpexchange\tdpexc.cmd" startt=hh:mm:ss priority=2 duration=2 duru=h period=1 perunit=day dayofweek=any
  • Associate TDP node with this schedule
def assoc <domain name> <sched name> <node_name1>
  • Configure TSM TDP Scheduler Service running following command:
    dsmcutil install /name:"TSM TDP Scheduler Service" /node:xxxxxxx /password:xxxxxxx /optfile:"c:\program files\tivoli\tsm\tdpexchange\dsm.opt" /autostart:yes /startnow:yes

    User running TSM TDP Scheduler services should have following rights:

    1) Member of AD's Organization Management group.
    2) Local admin

    November 22, 2010

    TSM ba clients miss scheduled backups ( servers behind firewall)

    TSM ba client version=6.2.1.0
    Client OS version=Windows 2008 R2

    Recently I faced this problem of tsm clients behind firewall miss their scheduled backups.I deleted  CAD and configured tsm scheduler directly to poll tsm server.This resolved the problem.

    Probable reason could be that when TSM server contacts tsm client behind firewall it fails to contact tsm client and hence don't get any reply.
    I also noticed that client was not listening at default port 1501 while it was listening on server comm port 1500.
    If any one struggling with this kind of issue they can try this workaround.

    ANS0106E Message index not found for message 14085

    TSM Cleint Version=6.2.1.0
    OS version=Windows 2008 R2

    ANS0106E Message index not found for message 14085

    This error comes because dscenu.txt file gets corrupted.This could be because of the upgradation of client while scheduler service running.Uninstalling and reinstalling ba client can resolve the problem or the very simple fix of this problem is to copy the file dscenu.txt from other client running on same level.This file resides inside ba client dir ( C:\Program Files\Tivoli\TSM\Baclient).