Today I was at a customer and they had an interesting error in their virtualization environment with Hyper-V 2012 and SC VMM 2012 Sp1 (yes I know, it is not R2 but we are working on it). One of their Hyper-V hosts had a hardware memory failure which lead to the host crashing and the VM´s restarting on other hosts..
during this the VM´s that was on that host got an error in the VMM db, looking and handling the VM´s from Hyper-V manager or failover cluster manager was no problem. Every time we tried to refresh the VM´s showed the following error and the job did not succeed.
And when we tried to look at the properties on the VM from VMM the console just died, and that happened every time..
So how could we find the VM´s that was suffering from this, well PowerShell could be used and with the following we could get the amount of VM´s and also easily get a list of the VM´s names.
Get-SCVirtualMachine | where virtualharddisks -eq $null | measure
Get-SCVirtualMachine | where virtualharddisks -eq $null | ft Name
So how could we fix this then,
There is a cmdlet in VMM with a parameter that can be used, Remove-SCVirtualMachine -Force , this will remove the VM but not the virtual hard disk as it cannot find it..
But because we want to keep the VM we will do a bit of a workaround
- Stop the vm in VMM by Stop-SCVirtualMachine own01 (even if the vhd(x) is not seen by VMM it will shut down the actual VM)
- Remove the VM from the cluster in Failover cluster manager or with powershell (this will only remove the cluster instance but not the actual VM)
- Start the VM in Hyper-V manager or with PowerShell Get-VM own01 -ComputerName HV01 | Start-VM
- Remove the VM from the VMM with Powershell using the -force Get-SCVirtualMachine own01 | Remove-SCVirtualMachine -Force (as I have removed the VM from the cluster VMM cannot find it and delete the vm´s xml file etc)
- Add the running VM to the cluster again with hyper-V PowerShell and failover cluster powershell using Get-VM own01 -ComputerName HV01 | Add-VMToCluster (Get-Cluster HVCL30)
- refresh VM´s in VMM and see that now the own01 vm can open properties
The reason I start the VM after removing it from the failover cluster is that VMM should think it is in stopped state, because that makes it easier to remove! And as in Hyper-V 2012 I can add a running VM to a cluster I do not have to keep the VM shut down during the whole process but just to get it into the right state in VMM
Not the easiest way but hey who had said that VMM was self-healing And yes we are in the process of upgrading to R2 and hopefully this error will not reemerge in that version….