Live Migrating a VM on a Hyper-V Failover Cluster fails – Processor-specific features not supported

 

I have been working on setting up a small cluster of Hyper-V Hosts (running as VMs), nested under a bunch of physical VMware ESXi 5.0 hosts. Bear in mind I am quite new to Hyper-V, I have only ever really played with single host Hyper-V setups in the past. Having just finishing creating a Hyper-V failover cluster in this nested environment, and configuring CSV (Cluster Shared Volume) Storage for the Hyper-V hosts, I created a single VM to test the “live migrate” feature of Hyper-V. Upon telling the VM to live migrate from host “A” to host “B”, I got the following error message.

“There was an error checking for virtual machine compatibility on the target node”. The description reads “The virtual machine is using processor-specific features not supported on physical computer “DEVHYP02E”.

 

So my first thought was, perhaps there is a way to mask processor features, similar to the way VMware’s EVC for host physical CPU compatibility works? If you read the rest of the error message it does seem to indicate that there is a way of modifying the VM to limit processor features used.

 

So the solution in this case is to:

  • First of all power down your VM
  • Using Hyper-V Manager, right-click the VM and select “Settings”
  • Go to the “Processor” section and tick the option on for “Migrate to a physical computer with a different processor version” under “Processor compatibility”
  • Apply settings
  • Power up the VM again

 

Processor compatibility settings - greyed out here as I took the screenshot after powering the VM up again.

 

So now if you try and live migrate to another compatible Hyper-V host, the migration should work.

 

HP P2000 G3 FC MSA – troubleshooting a faulty Controller (blinking Fault/Service Required LED)

Setting up a new HP P2000 G3 FC MSA with dual controllers over the last couple of days for a small staging environment, I ran into issues from the word go. The device in question was loaded with 24 SFF disks and two Controllers (Controller A and B).

 

On the very first boot we noticed a fault (amber) LED on the front panel. Inspecting the back of the unit, I noticed that Controller A and B were both still flashing their green “FRU OK” LEDs, (which according to the manual means that the controllers were still booting up), even after waiting a number of hours. On Controller A, I could see a blinking amber “Fault/Service Required” LED. Following through the troubleshooting steps in the manuals lead nowhere as the end synopsis was to check the event logs. Even the Web interface was acting up – I could not see the controller’s listed, could not see any disks and the event logs were completely empty. Obviously there was a larger issue at hand preventing the MSA and even the Web interface from functioning properly. To further confuse matters, after shutting down and restarting the device, controller B starting blinking the amber LED instead of A this time, both still stuck in their “Booting up” state. Refer to the linked LED diagram below and you’ll see that the LED flashing green is labelled as 6, and the amber blinking LED is the one labelled as 7 on the top controller in the diagram.

LED Diagram

HP Official documentation

After powering the unit down completely, and then powering back up again, the MSA was still stuck in the same state. Powering down the unit once more, removing and reseating both controllers did not help either. Lastly, I powered it all off again, removed controller A completely, then powered up the device with just Controller B installed. Surprisingly the MSA booted up perfectly, and LED number 6 (FRU OK) went a nice solid green after a minute or so of booting up. No amber LEDs were to be seen. Good news then! Hot plugging controller A back in at this stage with the device powered on resulted in both controllers reporting a healthy status and all the disks and hardware being detected. A final test was done by powering off everything and powering it back up again as it should be from a cold start. Everything worked this time.

 

Here is a photo of the rear of the device once all was resolved showing the solid green FRU OK LEDs on both controllers.

 

 

Bit of an odd one, but it would seem that controllers together were preventing each other from starting up. Removing one then booting up with this seemed to solve the problem, and at the end of the day all hardware was indeed healthy. After this the 24 disks were assigned and carved up into some vdisks to be presented to our ESXi hosts!

 

Twitter poll: how many vCPUs do you think a physical CPU Core can handle with vSphere 5

 

I was reading a document the other day where it was mentioned that you could potentially run up to 25 VMs per Physical CPU core with vSphere. Now first off, this wasn’t worded very well, it should have perhaps read “vCPUs” instead of “VMs”. The assumption of course was that each VM would be a single vCPU machine.

 

Nevertheless this is still a bit of a precarious thing to say though if you do not mention what kind of Physical CPU we are talking about, and what kind of load these vCPUs are going to be bearing. I think the assumption was that the VMs would be running extremely light loads and almost potentially be sitting idle most of the time to achieve that kind of density per Physical CPU core.

 

So with that thinking I decided to ask the question on Twitter. I only had 12 responses to the poll, but nevertheless, here are the results in Pie Chart format. The most sensible answer I got was under the “other” category, which was answered as “until %RDY >=10“, referring to the ESXTOP performance counter for CPU Ready time called %RDY. So this answer was saying keep adding vCPUs per core until you start seeing CPU %RDY (remember to look at this figure for individual vCPUs, not aggregated vCPUs) figures of greater than, or equal to 10%. This is the general figure to watch for, as anything over and above this would usually indicate CPU contention issues. I thought that this was a clever answer, but the other figures are also interesting to look at. The general assumption was that we were talking about low activity VMs with 1 vCPU, and the host system was running on a modern Intel Nehalem / 32nm Xeon processor.

 

Assuming, low activity single vCPU VMs on modern Xeon 32nm physical CPU

 

Not too many results to work off, considering I only got 12 responses, but the general consensus seems to be that 5-10 single vCPU VMs to 1 physical CPU core. Of course this isn’t a recommendation at all, its all down to opinion from an online poll, and the poll doesn’t take into account any types of workloads or specific processor specifications. The key thing to look out for here when considering consolidation ratios like this would be what kind of workload these vCPUs would be running. You would also always want to watch your performance stats (peaks and averages), making sure that you don’t get any high %RDY times, scaling up accordingly.

 

I have opened up another voting results collector to get a new set of results from the above results. Do you have an opinion? Please add your view on this poll, it’s only one question and will literally take a couple of seconds to enter!

Click here to take survey

 

nVidia introduces the worlds “first virtualized GPU”

 

I usually only ever follow nVidia and AMD with regard to their GPU offerings for gamers, this being one of my passtimes, however this press release of the green team’s the other day caught my attention.

 

To summarise, nVidia are unveiling their “VGX” platform, which will allow IT to deliver virtualized desktops with graphics or GPU computing power similar to, or as close to the real deal as possible, for users on any connected device (not necessarily just PCs or Thin clients for example). This VGX platform will consist of a few things, one of which will be the addon cards for servers that are passively cooled and as energy efficient as possible (interesting when considering how much power desktop gaming-grade GPUs generally consume!)

 

Some of the features nVidia are toting for their VGX platform thus far, according to their press release are:

 

  • GPU Accelerated desktops (of course)
  • Ultra-low latency remote display capability
  • Energy efficient, passively cooled hardware. Each board will have
    • 4 x GPUs (each with 192 CUDA architecture cores and a 4GB frame buffer).
    • 16GB memory
    • Industry standard PCI Express interface
  • VGX GPU Hypervisor
    • This is a software layer that should integrate with commercial hypervisors (VMware anyone?), enabling virtualization of the GPU
  • High user densities – shared graphics processing power for multiple users
    • (Up to 100 users to be served from a single server powered by one VGX board apparently)

 

Here are a few videos from the press release:

 

httpv://youtube.com/watch?v=kTnB_9ZgEvg

httpv://youtube.com/watch?v=e6BI2OTM-Hk

 

The article has mention of Citrix technology support, but what about VMware View? I am sure this type of integration should be available – I wonder how PCoIP would work to deliver virtual desktops accelerated by the VGX platform. If the latency reduction claims and acceleration benefits are anything to go by then we should be in for an even better VDI experience!

 

Using Project “Onyx” to find the equivalent PowerCLI script to achieve tasks done in the vSphere Client

 

A few days ago someone dropped a comment on one of my blog posts asking how they could Enter an ESXi host into maintenance without migrating VMs off of it automatically using PowerCLI. The -Evacuate switch for the cmdlet in question (Set-VMHost) didn’t see to be working when assigning the value of $false and hence they were unable to put hosts into maintenance mode without evacuating VMs first with PowerCLI.

Perhaps the cmdlet was being used incorrectly, or there is a better way of doing this, but that is not the point of this post. The point of this post is to show you the power and usefulness of Project “Onyx”. Project “Onyx” is an application released (quite some time ago!) by VMware that is essentially a “script recorder”. It connects to your vCenter instance, and in turn, you connect to it as a proxy using your vSphere client. Communications are not secured when doing this, so everything you do in your vSphere client is able to be recorded. You essentially end up with a “recording” of API calls that can be utilised in PowerCLI. Where this comes in handy is where you are not able to achieve something with PowerCLI’s already huge library of cmdlets. In this case the -evacuate switch of Set-VMHost was not working the way I expected it to work, and so to avoid wasting time trying to figure out what I needed to do, I just fired up Project Onyx, connected to it via the vSphere Client, then told an ESXi host to enter maintenance mode (unticking the migrate powered off / suspended VMs option of course) whilst the Project Onyx application was set to “record” mode.

The console then collected the necessary script, and I just modified it as necessary to create a small script that did the exact same task, but this time in PowerCLI.

 

To use Project “Onyx” simply download it from this page, then run the executable once you have extracted the .zip file. Tell Onyx to connect to your vCenter Server, then use your vSphere Client to connect to the machine that Onyx is running on (IP). Make sure you specify the correct listening port in the vSphere Client connection too – it will be the port listed in the Window Title bar of the actual Project “Onyx” application when it is running. Click the record button in the Application and then perform the required tasks using the vSphere Client.

 

Project Onyx application window with some recorded script. Note the port 1545 in the Window Title Bar.

 

Connecting to Onyx as a proxy