Notes on Applying the vSphere 5.1a Update

Applied the vSphere 5.1 update to the ESXi hosts and vCenter server. In theory, this is relatively straight-forward process, however it did have some glitches and ended up taking several hours. An issue to consider with VMware is how to test disruptive updates. In this case the cluster was not yet in production, so the disruption was not an issue. In the future could deal with this in a few ways, one would be to basically keep the VMware setup static and only apply needed security updates. VMware does allow give free limited evaluation licenses for vSphere --- it would be possible to get eval licenses, do a test setup to test version migrations on. Anyways, this upgrade issue is baked into the VMware setup --- the setup is somewhat complex to support the advanced VMware features, VMware management is apart from regular Linux management (compare to KVM management which is just builds on the usual Linux skills), VMware is a pretty critical infrastructure for the cluster, and it is hard to setup a test environment.

Much of the troubles were related hitches in getting some of the ESXi host config migrated. So, having a more robust way of configuring the ESXi hosts would be helpful and could avoid downtime.

Overview of Upgrade

The starting point was the 2 r710s ESXi hosts (msu-esxi-0 and msu-esxi-1) running vSphere 5.0 and vCenter Server Appliance (msu-vcenter) 5.0 running within the cluster. The ESXi hosts were functional for vMotion, NFS storage, and the MD3220 storage.

ESXi would be upgraded using the esxcfg command, which downloads the updates from vmware.com and applies them.

VCSA would be upgraded by installing VCSA 5.1a and then using the provided process for migrating cluster configuration between VCSAs to transfer configuration from the 5.0 to 5.1a VCSA.

Some References:

• http://www.vmware.com/support/vsphere5/doc/vsphere-esx-vcenter-server-51-release-notes.html

Notes on Procedure

I updated ESXi on msu-esxi-1, did the vCenter migration (with vCenter running on msu-esxi-0), then updated ESXi on msu-esxi-0.

ESXi Update

Make a backup of ESXi host config (I only did this after the v5.1.0 install on msu-esxi-1 ...).

Login to vMA, point fastpass to the host to work with, pull backup.

ssh vi-admin@msu-vma.msulocal
vi-admin@msu-vma:~> mkdir config-backups
vi-admin@msu-vma:~> vifptarget -s msu-esxi-0.msulocal
vi-admin@msu-vma:~[msu-esxi-0.msulocal]> vicfg-cfgbackup -s config-backups/msu-esxi-0.msulocal.20121118

Used the "esxcli software sources profile" command to update ESXi. This works sort of like yum --- refers to external software repository and can perform install or updates of packages. The normal firewall setup blocks outgoing http access from ESXi host, instead of sorting how to change the rule, I simply disabled the firewall during the install. The ESXi host only is on the 10.10 network, so use proxy to get to vmware.com.

#See that no VMs running
~ # esxcli vm process list

~ # hostname
msu-esxi-1.msulocal

# Disable firewall temporarily
~ # esxcli network firewall set --enabled false

~ # esxcli software sources profile list --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml --proxy=http://proxy.msulocal:3128
Name                              Vendor        Acceptance Level
--------------------------------  ------------  ----------------
ESXi-5.0.0-20120904001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120404001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120701001s-standard  VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120301001s-standard  VMware, Inc.  PartnerSupported
ESXi-5.0.0-469512-no-tools        VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120604001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120901001s-standard  VMware, Inc.  PartnerSupported
ESXi-5.0.0-20111204001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20111104001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.1.0-799733-standard        VMware, Inc.  PartnerSupported
ESXi-5.0.0-20110904001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120704001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120701001s-no-tools  VMware, Inc.  PartnerSupported
ESXi-5.0.0-20111104001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120704001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120901001s-no-tools  VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120504001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120904001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120301001s-no-tools  VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120604001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120504001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-469512-standard        VMware, Inc.  PartnerSupported
ESXi-5.1.0-799733-no-tools        VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120302001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20111204001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.1.0-20121004001-standard   VMware, Inc.  PartnerSupported
ESXi-5.1.0-20121004001-no-tools   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20110904001-notools    VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120404001-standard   VMware, Inc.  PartnerSupported
ESXi-5.0.0-20120302001-no-tools   VMware, Inc.  PartnerSupported

Want the ESXi-5.1.0-20121004001-standard update.

~ # esxcli software profile update --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml --proxy=http://proxy.msulocal:3128 --profile ESXi-5.1.0-20121004001-standard

Update Result
   Message: The update completed successfully, but the system needs to be reboot

...
~ # esxcli network firewall set --enabled true
~ # reboot 
~ # Connection to msu-esxi-1 closed by remote host.
Connection to msu-esxi-1 closed.

Reboot took about 7 minutes.

After this update, vCenter (5.0.0) wouldn't connect to msu-esxi-1 (5.1.0).

-- TomRockwell - 21 Nov 2012