You Can't Triple Stamp a Double Stamp -
Using vMotion To Move An Application Cluster Inside an ESX Cluster
This may be one of the more complex things you can be asked to do in vSphere. Why would you want to cluster two VMs together when you have the protection of ESX clustered servers using vMotion, High Availability (HA), Distributed Resources Scheduler (DRS) and Fault Tolerance (FT)? A common answer is because at the application layer you can't easily vMotion a clustered active node or fail over without significant configuration and testing. Let's look at the parts involved to successfully vMotion application cluster nodes. We'll also consider firmware and compatibility. Finally, we'll talk a little about automating all of this to happen seamlessly.
Configure Anti-Affinity Rules
Remember a VM is a server. It looks and feels like a physical server. Keep that in mind when working with application clusters. If you have an active node in a cluster that you need to vMotion to another host in the ESX cluster then you'd likely fail over to the passive node first. Once the active node has been successfully failed over (automate this as much as possible) then it will become the passive node in the application cluster. Consider manipulating the heartbeat timeout period before vMotioning the new passive to a new ESX host in the cluster.What you want to do is utilize Anti-Affinity rules in ESX to keep cluster node A on one specific host and cluster node B on a different host at all times. Otherwise DRS will manage that for you!
Right Click on the ESX cluster, Edit Settings, Under VMware DRS
Click Rules > Add.
Click the DRS Groups Manager tab.
Click Add under Host DRS Groups to create a new Host DRS Group
containing half of the ESX/ESXi hosts in the cluster. Create a
second Host DRS Group containing the remaining hosts.
Click Add under Virtual Machine DRS Groups to create a Virtual
Machine DRS Group for each anti-affinity virtual machine.
Click the Rule tab, from the Type drop-down menu, click Virtual
Machines to Hosts.
Select the first virtual machine from Cluster Vm Group > Must
run on hosts in group.
Select a host from the Cluster Host Group.
Click Must run hosts in group.
Select the second virtual machine from Cluster VM Group.
Select the second host from the Cluster Host Group and click OK.
Upgrade Firmeware and BIOS
Considerations before attempting to vMotion an application cluster. BIOS, Firmware (HBA, NIC), ESX, Windows 2008 Server. I recently did this sort of cluster within an ESX cluster and here are the items you need to verify/upgrade before you start. This would apply to an ESX 4.1u2 system using the Service Console. The process to upgrade Emulex firmware on ESXi is very different.
BIOS - Upgrade BIOS especially if you are doing this on a blade server!
HBA - Upgrade your Emulex Firmware.
Emulex requires a RPM be
installed on the host first.
Download the following RPM from Emulex
site directly (no link):
- cat /proc/scsi/lpfc820/1 (for example) to find the model of Emulex card installed.
- rpm -ivh elxvmwarecorekit-esx40-4.0a46-1.x86_64.rpm
- /usr/sbin/hbanyware/hbacmd ListHBAs | grep Port (for WWPN info)
- /usr/sbin/hbanyware/hbacmd download 50:06:0b:00:00:c2:7a:01 zf282a4.all
You must do the “download” for each
WWPN in the output of ListHBAs.
Download Emulex firmware here:
Broadcomm drivers must be
installed in maintenance mode followed by a reboot. Again, mount the
ISO using Virtual Media.
- mount /mnt/cdrom
- vimsh -n -e /hostsvc/maintenance_mode_enter
- expupdate –bundle=/mnt/cdrom/offline-bundle/BCM-bnx2x-1.70.34.v41.1-offline_bundle-547965.ziip update
- reboot
- ethtool -i vmnic0 (to verify)
and
vimsh -n -e /hostsvc/maintenance_mode_exit
Here is the current (at the time of this post) Broadcomm fix from VMware
vmware-esx-drivers-bnx2-bnx2x-cnic-bnx2i.547965.iso
Automate, Automate, Automate
Use tools like PowerShell to manage the failover. Microsoft Cluster Manager is the GUI way to do the same thing. You want to to script fail over so it happens automatically based on real conditions and thresholds. In a Disaster Recovery situation you will WANT to automate the fail over. Consider this. Node A resides on an ESX host at your local data center. Node B resides in your disaster recovery hot site many miles away. If a condition is met on Node A, fail over automatically to Node B. Scripting this will take much time and most importantly - lots of testing! It's worth your effort. I prefer Linux Clusters in this scenario for the ease of scripting and the reliability of these systems.
Clusters are complex environments with lots of inter-dependencies. You may ask yourself why would you cluster within a cluster. The situation can be build just like physical clusters are but within VMware. All the configurations and tools exist to make it so. Pay close attention to detail during your configuration. Work off an existing standard, template or profile. Lastly, automate all the steps that are possible to automate. Your boss will thank you when disaster strikes.
Good luck!
No comments:
Post a Comment