Today I was creating an upgrade baseline for some old ESX 4.x hosts to be patched up to a newer update level. I ran into an error whilst uploading an ESX ISO with the new update version and subsequently found myself troubleshooting the issue. I thought I would do a quick post on general things to check when troubleshooting VMware Update Manager.
First of all check you are of course uploading the correct file / ISO! Note that ESX upgrade baselines work with ISO files and ESXi deal with .zip files. Ensure you are using the correct file and build of ESX or ESXi depending on which you are planning on using.
Consult the log files! Logs are kept in different locations depending on your OS that VUM is running on.
Windows XP, 2000, and 2003 – C:\Documents and Settings\All Users\Application Data\VMware\VMware Update Manager\Logs
Windows Server 2008 and above – C:\ProgramData\VMware\VMware Update Manager\Logs\
ESX update manager logs are kept in – /var/log/vmware/esxupdate.log
use cat /var/log/vmware/esxupdate.log | more to view the log file in ESX from the shell / PuTTy SSH session.
The log file in Windows should be named something similar to “vmware-vum-server-log4cpp.log”
You should be able to locate an issue that has occurred by noting the time the issue happened in Update Manager. Open up the relevant log file and navigate down to the time it happened in your logs. Hopefully the description / entry will point you in the right direction.
In my case today, I was trying to upload an ESX 4.0 Update 4 ISO (Complete) to create a new Upgrade Baseline for some older ESX 4.0 hosts. I got an error after uploading the ISO using the new baseline upgrade wizard. See below:
Error message after uploading ISO for Host Upgrade Baseline
Although the message above in the GUI is not very descriptive, after looking at the log files on the Update Manager server I found an entry which explained what my problem was:
The problem was that my ISO file was corrupt – the download had seemed to complete just fine when downloading from vmware.com, but there must have been an issue. A quick check of the ISO using my md5sum command line utility in Windows confirmed that the MD5 hash for this ISO file did not match the MD5 hash listed for the ISO on vmware.com (as pointed out in the log files above).
I downloaded a fresh copy of the ISO, checked the MD5 again to ensure it matched this time, and re-uploaded to create a new baseline. Everything worked as expected this time around.
In Part 2 of this series, I covered the steps needed in order to manually upgrade your ESX(i) 4.x hosts to ESXi 5.0.0. This is useful for smaller deployments or lab set ups where you don’t have that many hosts to upgrade. However, with larger deployments, you’re going to want to look at a way of automating this process. VMware Update Manager (VUM) is one of these ways, and in this part, I’ll explain how to use it to upgrade your hosts to ESXi 5.0.0 by using baselines.
Using VMware Update Manager to upgrade ESX(i) 4.x hosts to 5.0
There is a little bit of ground work required to set up your ESXi 5.0.0 ISO image and create a VMware Update Manager baseline in this process, but once this is done, it is quite easy to attach your new baseline to your older hosts and remediate them against this (effectively upgrading them / bringing them up to date against this new baseline). To go through this process, you will of course need a VUM server. Update Manager comes with all editions of vSphere 4 and 5, so if you don’t already have it up and running, you should seriously think about deploying it, as it will save you a lot of time with otherwise routine, time consuming tasks when it comes to upgrading/updating Hosts, VMs, and Applications.
Start off by downloading the ESXi 5.0.0 ISO image to your vSphere Client machine if you don’t already have this. Once this is downloaded, you’ll then want to go to your vSphere client home screen and choose VMware Update Manager from the home screen.
This will load the VUM interface and provide you everything you need to work with Update Manager. Select the “ESXi Images” tab along the top, then click on the link called “Import ESXi Image…” In the wizard that appears, browse for your recently downloaded ESXi 5.0.0 ISO image and select it. Follow through the wizard to Upload the ISO to your VUM server. You may receive a security warning (SSL) which you will need to ignore/accept to continue. All going well, you should reach an “Upload Successful” point and see the details of your ISO, similar to the below screenshot.
Move to the next step, and we’ll now create a Baseline out of this Image. Tick the box for “Create a baseline using the ESXi image” and give it a meaningful name and an optional description. Finish the wizard when you have named the baseline, and you’ll now have a shiny new baseline with which you can use to remediate your hosts against.
Move along to the “Baselines and Groups” tab in the main VUM area, and verify that your new baseline is showing and that the details look correct. It should show up as a “Host Upgrade” under the Component column.
For the next step, we’ll be looking to attach this baseline to the older ESX(i) hosts that need to be upgraded. To attach to all your hosts in the same cluster all at once, go to your Hosts & Clusters Inventory view, select your Cluster name, then go to the “Update Manager” tab near the end. From here, click on the “Attach” link as seen below:
Now, you’ll want to select the ESXi5 Upgrade Baseline we created in the previous steps to the selected Cluster. Simply tick the box next to your “Host Upgrade” Baseline name, then click “Attach”.
Now that your baseline has been attached, it will apply to all the ESX(i) hosts in the cluster. From the same Update Manager screen, click the “Scan” link to initiate a compliance scan. In other words, we’ll be looking to find out which hosts are not in compliance with our new ESXi 5.0.0 image, from which point we can move on to remediating (upgrading). Select the tickbox for “Upgrades“, then click “Scan” to continue. Once this is done, you should notice that all the hosts which have your baseline attached will show as Non-Compliant (unless you have manually upgraded any of these).
If you are ready to begin the upgrade process for your hosts, click on the yellow “Remediate” button on this screen.
You’ll now be taken to the Remediation Wizard / Selection screen. Tick (Select) which hosts you would like to remediate (upgrade) and ensure the correct Host Upgrade Baseline is selected, then continue on to the next step.
There are a few options that you will need to configure in this wizard based on your environment. Preferences such as Host and Cluster Remediation options can be set up, which control how your cluster and hosts should handle the remediation tasks. For example, Power State to put your VMs into, or Cluster features to keep enabled or disable. You can also define a schedule for remediation in this wizard if you wish. Here is a quick rundown of the steps in this wizard that I went through, along with a couple of screenshots of the options I went with in my lab upgrade.
1. Selection screen
2. Read and Accept the EULA
3. Choose whether to remove third-party installed software that is incompatible with the upgrade or not.
4. Optionally, set up task name and description for the upgrade remediation task. Select Immediately, or specify a time to begin the task.
5. Choose how you would like to handle VMs for the remidiation. Note that these options also apply to hosts in clusters. I left mine at “Do not change VM Powerstate” as I have DRS set to fully-automated. Therefore VMs running on this host will be vMotioned off when the host enters maintenace mode, and I won’t need to worry about moving them myself.
6. Select how the cluster should be handled during remediation. For example you can disable DPM during remediation. It will be re-enabled after the task is complete.
7. Check summary and finish the wizard.
Host remediation options
Cluster remediation options
Checking through the summary and finishing off the wizard
Once you finish the wizard, your selected hosts will begin the upgrade (Remediation) process. Depending on the options you chose, your VMs should automatically be vMotioned off the current host being upgraded as it enters maintenance mode. If you’d like to follow along with one of the hosts, have a look at the server console to see how progress is made. Working with a cluster of hosts, you should be able to use this method to upgrade hosts systematically in an automated fashion. One of the benefits of this method is obviously along the lines of time saving, but also the fact that you can have zero downtime if it is done correctly. As each host is upgraded to ESXi 5.0.0, VMs can vMotion back to upgraded hosts, allowing you to shift the VM workload around hosts that need to go down for maintenance. Lastly, don’t forget to ensure your licensing is sorted out after the upgrades are complete.
In the next part, we’ll take a look at other tasks involved with an upgrade to vSphere 5.0.
