Performing automated DLPAR operations

One of my closest friend asked recently , how he can perform DLPAR operations automatically without login into HMC itself.

Infact it is not so much difficult task to perform. On following sections , you will find a step by step guide , how you can increase or decrease memory on automated basis and offcouse same technique can be used for increasing/decreasing CPU resources as well.

A) Host Name resolution setup:

I , first of all identified one of Lpar on P570-2 ( aqbtest) as a management Lpar for this whole solution. I established name resolution setup on this Lpar so that all other Lpars on P570-1, P570-2 and P570-3 servers should be pingable by name and Ip address from this management Lpar. Hmc should also be resolvable by hostname from this management Lpar.
One important thing which you should consider while implementing this whole solution is that Lpar names (as they appear on HMC interface) should also be resolvable from management Lpar. In most of the cases, hostnames and Lpar names are same , but however if they are different then still you can use /etc/hosts file or DNS to resolve Lpar names also from management Lpar. This requirement arises from shell scripting done for this solution and will be explained further in Section D.

B) SSH access setup:

Second step would be establishing SSH relationship and access from this management Lpar to each and every other Lpar as well as to HMC.
SSH access to HMC is established in slightly different way as compared to SSH access to Lpars. I used Openssh on AIX from Bull web site ( freeware.openssh.rte 3.8.1.0), installed it alongwith openssl library ( openssl 0.9.6.7).I created a user on aqbtest Lpar named as hscadmin and also on Hmc HMC1. I assigned "Managed system profile " to hscadmin user on HMC .I also allowed remote comnmand execution ( so that HMC can allow SSH remote connections to be established with it )
On AIX lpar "aqbtest", i generated RSA key pairs by following commands
/home/root> su - hscadmin
/home/hscadmin> ssh -keygen -t rsa ( accept default values with blank passphrase )
/home/hscadmin> export hscadminkey=`cat id_rsa.pub`
/home/hscadmin> ssh hscadmin@HMC1 mkauthkeys -a \ "$hscadminke\/" ( replace it with back slash while final editing )
The above command will copy public key from AIX Lpar aqbtest to HMC1. Once copied , you can also directly login to HMC as hscadmin using ssh and varify that key has been copied successfully or not by executing " cat .ssh/authorized_keys2 " command.
You should now be able to login to HMC from AIX management Lpar without any password prompt. You can verify by executing
/home/hscadmin> ssh HMC1 lsusers

which will show all users presnt on hmc.

If you face any problem while login into hmc using ssh , you can always make the authorized_keys file empty and then try again with above procedure. To make this file empty, you can follw the following command sequence on AIX management lpar
/home/hasadmin> touch /tmp/mykeyfile ( an empty file )
/home/hscadmin> scp /tmp/mykeyfile hscadmin@HMC1:.ssh/authorized_keys2

Now the same management Lpar should also be able to execute commands remotely to all other Lpars without any password prompt.For this , again i decided to use SSH with DSA authentication so that management lpar should be login and excute commands on all lpars remotely without any password prompt.

I created hscadmin user ( which may be an ordinary users ) on another Lpar on P570-1 server ( named aqbdb) and then install openssh on this Lpar. I then generated DSA key pair on management Lpar "aqbtest"

/home/hscadmin> ssh-keygen -t dsa -b 2048

/home/hscadmin> scp id_dsa.pub hscadmin@aqbdb:/home/hscadmin/.ssh/dsa_aqbtest.pub

on aqbdb Lpar ,

/home/hscadmin> touch .ssh/authorized_keys
/home/hscadmin> cat .ssh/dsa_aqbtest.pub >> authorized_keys

Now you should be able to login from management Lpar aqbtest to Lpar aqbdb , as hscadmin user, using ssh , without any password prompt.


Shell Scripts Creation for Resources Movement

Now after you are able to login to HMC without any password prompt from your AIX server, next step would be creation of shell scripts to do the resources allocation/reallocation.

Assume that you want to increase memory on aqbdb Lpar from 30 GB to 40 GB before running any batch job or backup process and then revert it back to 30 GB after backup process finishes

Ideally this could be done from same backup shell script by calling memory increase script (memincr.sh) before backup process and then calling memory decrease script ( memdecr.sh) as follows:
-----------------------------------------------------------------------------
#!/bin/ksh

Su – hscadmin –c “/home/hscadmin/memincr.sh” # increasing memory

(Backup process commands)

Su – hscadmin –c “/home/hscadmin/memdescr.sh” #decreasing memory
--------------------------------------------------------------------------------


While memincr.sh script would be simply as follows (it will de allocate memory resources from test partition and allocate the same to production partition


#!/bin/ksh

ssh hscroot@hmc chhwres -m p550_itso1 -o r -p aqbtest -r mem -q 6144 -w 15 #Release 6 GB from aqbtest partition

ssh hscroot@hmctot184 chhwres -m p550_itso1 -o a -p aqbprod -r mem -q 6144 -w 15 #Add 6 GB to Aqbprod partition

------------------------------------------------------------------------

A similar kind of memory decreasing script can easily be written for reverting back memory resources distribution , as it was originally.