Rescue failing deploy from VMM 2012 R2 to Hyper-v hosts

I was contacted by a customer with an issue where he could not use ISO´s or deploy VM´s from his VMM server to the Hyper-V hosts.

Screen Shot 2015-10-13 at 16.36.56

First I started searching the net for solutions and found some that could match:

Move VM error  : suggesting changing BITS port

Deploy VM from VMM : where I updated the hosts certificates on the VMM server,

but they did not help as the error continued, I also tried to reinstall the VMM agents on the hosts if that would have been the cause.

Starting to investigating this issue more thoroughly i noticed some strange things, I could store VM´s from the hosts into the VMM library so that way worked with BITS, So the problem seemed to be related to the VMM Server..

Screen Shot 2015-10-13 at 16.36.39

Talking with my friend and MVP Daniel Neumann he sent me a link to another blog post that describes if you have deleted the VMM certificate (my certificate was there but apparently out of sync in some way) so here is the steps to regenerate the VMM certificate:

1. Launch the VMM PowerShell on the Virtual Machine Manager server.

2. Type the following and press enter:

$credential = get-credential

3. Type the username and password that is a local admin on the VMM server.

4. Type the following and press enter:

Get-VMMManagedComputer -ComputerName “” | Register-SCVMMManagedComputer  -Credential $credential

5. Ensure that the VMM job completes successfully under the Jobs tab in the VMM console.

Note: In my example above, is the FQDN of the VMM server.

And after this the VMM deploy works again!


Moving SCVMM 2012 R2 Server going bad

I have helped two customers moving their System Center VMM 2012 R2 servers to a Hyper-V VM..  Instead of carrying legacy stuff we installed a new Generation 2 VM in Hyper-V with Windows Server 2012 R2.

Easier said than done… or?


So what went wrong at both customers and how did I solve it?

We copied the library and the database backup from the old server. Did a shutdown of that one and then started the new one and added that to the domain and then installed the VMM server.

Patched it to UR7 with windows update and after that we did a restore of the db from the old system with the binary scvmmrecover.exe -path <db-backupfile>.

After that I started the console and trying to check things in properties and stuff and the console crashed  the service got a dump:

Screen Shot 2015-10-08 at 15.48.05

Looking at the dump, I could see that not everything was great with the database, (the old VMM server was patched to UR7 before I did a database backup). Based on the log file something is missing in the restored database…

Screen Shot 2015-10-09 at 10.35.49

So how did I solve it? I uninstalled the UR7 on the VMM server and then reinstalled it and voila, no more crashes!


Using Azure Automation and Hybrid Worker to automate SCVMM tasks


Azure Automation and Hybrid Runbook workers are fun to play with and today I wanted to try something like automating System Center VMM tasks,

I read Markus Lassfolk´s post about changing VM´s network adapters MAC from dynamic to static which is the prefered setting that you should use for your Hyper-V VM´s. So How could I utilize this with Azure Automation runbooks instead of an script that is run on the VMM server?

In my runbook I have a VMM Automation account declared as a credential and I connect to the VMM server with that to be able to reconfigure the VM´s. If I do not use a -PSCredential for the inlinescript the Runbook worker will try to use the system account of the Worker server and that does not work so well on the connection to the VMM server,

Screen Shot 2015-09-09 at 15.24.12

Either start it from powershell console if you have Azure PowerShell module installed or within the GUI and I used only one input parameter and that can be used for an explicit VM or “All” VM´s.

Screen Shot 2015-09-09 at 15.04.50

And as you can see in the VMM log I have changed the VM´s NICS, and also if a VM have two or more NICS all of them get a static MAC configured :

Screen Shot 2015-09-09 at 15.03.07

Hope that you see the potential in Azure Automation and Happy automating!


Download SysCtr TP3 evaluation VHD´s with PowerShell

Earlier this week Microsoft released Windows Server Technical Preview 3 and System Center Technical Preview 3 to the masses.

There is also a way to easily test the System Center bits instead of installing everything with prereqs and stuff and that is to use the preinstalled VHD´s that Microsoft and the System Center team provides.

You can find the eval VHD´s here on the download site:

Or you can use my powershell script to download them and import into your Hyper-V server and start playing once downloaded :-D. If there are issues during download you can just start the script again as I check if the file already been downloaded so you will only download each file once and as I use the BITS engine the file appears in the download folder when it is completely downloaded….

Happy playing!!

New sponsor: VirtualMetric – cool monitoring and reporting for hyper-v

I am very proud to tell you that I have a new sponsor on my blog and that is VirtualMetric.

Have you not heard about them then it is time to go and check out their website at to learn more because they have an awesome monitoring and reporting platform for your Hyper-V environment!. It is agentless and it also reports Hardware status for the hosts via IPMI.


There will be a more thorough post about how to get the platform up and running and how to use it later on!

Hyper-V Cluster Heartbeat for MDT ref VM goes bananas or?

I have been helping a customer with their environment and we had a problem that took me a while to figure out.

They were baking reference images for their SCCM environment and the best and easiest way is to use VM´s of course. The problem that occurred was that when the image was being transferred back to the MDT server the VM rebooted after half of the image had been uploaded….

So what was doing this crazy behavior? It took me a little while before I realized what it was all about and it had to do with the Hyper-V cluster platform and resilience and heartbeat functionality!

So at first the build VM boots from the MDT image, no integration tools yet then but then it restarts to install applications and stuff within the OS and as the customer works on a Windows 7 image you can see it starts to send heartbeat to the host.


As you might know, clients and servers since Windows Vista and 2008 have integrational services by default in them although best practice is to upgrade them as soon as possible if the VM shall continue to reside in Hyper-V.

The interesting part in this case was that the OS rebooted within itself when it was finished with sysprep to start the MDT image for transferring the WIM back to the MDT server and the cluster/hyper-v did not notice this and thus it thought that the heartbeat stopped.


And as it was a cluster resource this heartbeat loss was handled by default, and guess what, rebooted!

So what settings in the cluster resource does this madness? First of all, the Heartbeat setting on the cluster vm resource properties



This can be read on the technet site about Heatbeat setting for Hyper-V clusters:

Screen Shot 2015-06-04 at 10.35.22

And then you have policy what the cluster should do after it thinks the vm has become unresponsive:

Screen Shot 2015-06-04 at 14.14.56 1



There are different ways to stop the cluster from rebooting the machine and one of them is to disable heartbeat check and another is to set the response to failure to do nothing,

The customer uses mostly VMM console and for them when building a new VM for MDT reference builds they can set the integrational services to disable heartbeat check and thus not get their work postponed by unwanted reboots.


During the search for why I checked Host Nic drivers as I thought that it might have something with a transfer error but could not find anything, positively the hosts got up on the latest nic firmware and drivers 😉 . My suspicion that it had to be the cluster was awaken after I had spun up a test VM that was not part of the cluster and that one succeeded in the build and transfer.

This is a rare case and I would say that in 99% of the cases you want the default behaviour to happened as a VM can become unresponsive and then the cluster can try a reboot to regain into operations..

Clarification: If you spin up a VM with a OS or pxe image that does not have integrational services it will not reboot the VM after the cluster timeout, the OS has to start sending heartbeat to the Hyper-V host and then it will be under surveillance and managed by the cluster until properly shut down!

Hope that this helps for someone out there wondering what happens…



Exclude VM´s from dynamic optimization in SC VMM

In a case where we had a VM that was a bit sensitive with an application that does not like the ping loss during live migration between hosts in a cluster we wanted to exclude the vm from the automatic dynamic optimization. It might not be totally clear where you find this setting for the VM´s you want to exclude from this load balancing act.

First of all, do you know where you set up automatic dynamic optimization in System Center VMM 2012 R2? For some reason you set it up on the hosts folder and not on the cluster object in VMM:

Screen Shot 2015-04-16 at 13.00.13


So where do you exclude VM´s from this optimization? If you look under a VM´s properties you can check the actions and there you see the magic checkbox :

Screen Shot 2015-04-16 at 12.55.35

And now when the automatic job runs my sensitive VM will stay on the host.

SC VMM Error 803 after restoring a duplicate VM with alternative name

I was at a customer today testing some backup/restore scenarios with their backup providers software and where we got an interesting error: 803 “Virtual Machine restore-Test5_gen1 already exists on the virtual host other cluster nodes” Recommended Action: “Specify a new name for the virtual machine and then try the operation again”



In the Backup/restore console we wanted to do a restore to alternative place and an alternative name as we had not deleted the original VM (for instance if you need files or just verify some state or something), in the configuration of the restore job we checked that the restore process would create a new VM Id which was the first thing we thought that was why VMM complained.

The thing was that this error only appeared when we did a restore to another hyper-v host, if we restored to the same where the original VM was residing there was no error..

As you can see after the restore both the original VM and the alternate had the same “#CLUSTER-INVARIANT#” id but different VMId´s, and when we tried to refresh VM´s we got the error above.

Screen Shot 2015-03-30 at 20.56.59

The solution was not so farfetched and can be read about in the KB2974441 although that case is about RDS VDI but still, and as can be read about why the ID is in the notes field from the beginning: “VMM adds a #CLUSTER-INVARIANT#:{<guid> } entry to the description of the VM in Hyper-V. This GUID is the VM ID SCVMM uses to track the VM.”

For the VM not showing up in VMM console we just went into the notes field on Hyper-V Manager and removed that specific “#CLUSTER-INVARIANT#” id and after that VMM generated a new for that VM and it appeared in the VM list on the VMM server.

So why was it no problem when we restored to that same host? For some reason VMM managed to see the duplicate residing on the same host and generate a new id in the notes field for that and thus appearing in the VM list without any massage..


Hyper-V Powershell module now with 211 cmdlets in build 10041

I have upgraded my windows 10 to the latest build 10041 and activated Hyper-V.

Screen Shot 2015-03-20 at 09.45.22

In this build I can see that there are now 211 cmdlets in the hyper-v PowerShell module:

Screen Shot 2015-03-22 at 10.58.57

And comparing it to the PowerShell module that was released in Windows 8.1/2012 R2 you can see the following new cmdlets, although the Windows 10 is in preview and things can change before it is released!

Screen Shot 2015-03-22 at 11.32.16

I will dig into the new preview release and maybe there will be a followup post on my findings!

Review of the new 5nine Manager 6.1

I have used both Hyper-V Manager and System Center VMM from the early beginning of Hyper-V in 2008. If you wanted to use a tool with several Hyper-V hosts either that was standalone or in cluster you had to use VMM, especially if using the deploy functionality with templates and OS customization.When System Center 2012 was released they decided to sell it as a whole product with different roles which means you have to buy either a System Center Standard (for single managed server hosting up to 2 VM´s) or a System Center Datacenter (Virtualization hosts using 2pCPU with unlimited managed VM´s on that licensed host). There are though some alternative and one of them is the 5nine Manager.


5nine Manager has been evolving for some version since they first saw daylight and now they have added some really cool features making it a real competitor to VMM and if you do not use the other System Center roles this could be a good buy.

part of the new 6.1 version, these new features exist:

  • Graphical Monitoring Plugin and Alerts 
  • New Optimizer Notifications 
  • Increased Scalability through Multiple Console Instances 
  • Simplified VM Management with Sortable Table Parameters 
  • Enhanced VM Templates 
  • Integrated Sysprep Tool 

The gui can be run on a Hyper-V Core server so no excuse here now to not use that for your virtualization hosts! I actually have had some discussions with customers why they do not want to run core because if the host would be disconnected and they would need to do some debug and analysing they thought that it would be to difficult when using core instead of the full gui. With the 5nine Manager you can manage and check your VM´s in a GUI but within the Core session 🙂

To get it working in your environment you need to install it on a Windows box. The nice thing here is that you can run it on a Windows 8.1/8/7 and connect to the different Hyper-V versions.

Screen Shot 2015-03-01 at 19.42.00

Once installed you need to add Hyper-V hosts/clusters to manage them.

Screen Shot 2015-03-01 at 19.44.39

The GUI in the console is quite easy to use and as you can see, I can right click on a VM and convert it as a template to be able to provision new VM´s. Another really nice feature which you can see in the list is “Set IP address” and this uses the functionality of the integrational services and sets the IP on the vNic inside the VM!!

Screen Shot 2015-03-01 at 20.36.15

 And as you can see there are some more options in the bar with optimizer,monitor, reporting best practices and Hyper-V logs. I really like the GUI and that it is easy to find the different options and tasks not having to jump around between different views forever. Having a single pane of glass when searching Hyper-V hosts for failing VM´s or hardware issues through logs then this Manager is great.


As described above, If you do not use more than System Center VMM in your virtualization environment and the list price for System Center R2 Datacenter is about 3600 $ and the 5nine Manager would cost from 199 $ per host then there is not so much to argue about! There is also a free version of the 5nine Manager that you can use combined with VMM on your Windows Hyper-V Core hosts.