Download SysCtr 2016 TP4 VHD´s with PowerShell

I have been using and publishing my script for easily download all the VHD´s for evaluating and testing the technical previews of System Center 2016 and now it is time for the TP4! These pre-installed VHD´s make it quite easy to spin up some of the System Center Suite as VM´s on your Hyper-V box within minutes once downloaded.

Here is the script for your convenience and start playing with the new release!



Trying to do a Checkpoint on a VM resulting in error 12700

Today I helped a customer that have issues with their new VM´s and doing backup using Veeam and also trying to do Checkpoints within VMM on their Hyper-V 2012 R2 environment.


Looking at the error message from VMM it showed quite clearly that it was just one of the VHDx that was having the issue and looking at where it was residing it showed the reason:

Screen Shot 2015-11-13 at 11.43.59

The file was residing on its own in a CSV volume and directly in the root folder, and as stated in the blog post from the Core Team the VM worker process (VMMS) does not have the relevant permissions on that level and thus getting a access denied error when trying to do a checkpoint.

So how do you solve it? By either manually or with live storage migration moving it into a subfolder which will give the right ACL´s and thus giving the VM worker process rights to create a avhdx file in that folder.

So watch out when you create new VM´s that you actually put all of the virtual disks that belong to it inside folders on those CSV´s!

Moving SCVMM 2012 R2 Server going bad

I have helped two customers moving their System Center VMM 2012 R2 servers to a Hyper-V VM..  Instead of carrying legacy stuff we installed a new Generation 2 VM in Hyper-V with Windows Server 2012 R2.

Easier said than done… or?


So what went wrong at both customers and how did I solve it?

We copied the library and the database backup from the old server. Did a shutdown of that one and then started the new one and added that to the domain and then installed the VMM server.

Patched it to UR7 with windows update and after that we did a restore of the db from the old system with the binary scvmmrecover.exe -path <db-backupfile>.

After that I started the console and trying to check things in properties and stuff and the console crashed  the service got a dump:

Screen Shot 2015-10-08 at 15.48.05

Looking at the dump, I could see that not everything was great with the database, (the old VMM server was patched to UR7 before I did a database backup). Based on the log file something is missing in the restored database…

Screen Shot 2015-10-09 at 10.35.49

So how did I solve it? I uninstalled the UR7 on the VMM server and then reinstalled it and voila, no more crashes!


Using Azure Automation and Hybrid Worker to automate SCVMM tasks


Azure Automation and Hybrid Runbook workers are fun to play with and today I wanted to try something like automating System Center VMM tasks,

I read Markus Lassfolk´s post about changing VM´s network adapters MAC from dynamic to static which is the prefered setting that you should use for your Hyper-V VM´s. So How could I utilize this with Azure Automation runbooks instead of an script that is run on the VMM server?

In my runbook I have a VMM Automation account declared as a credential and I connect to the VMM server with that to be able to reconfigure the VM´s. If I do not use a -PSCredential for the inlinescript the Runbook worker will try to use the system account of the Worker server and that does not work so well on the connection to the VMM server,

Screen Shot 2015-09-09 at 15.24.12

Either start it from powershell console if you have Azure PowerShell module installed or within the GUI and I used only one input parameter and that can be used for an explicit VM or “All” VM´s.

Screen Shot 2015-09-09 at 15.04.50

And as you can see in the VMM log I have changed the VM´s NICS, and also if a VM have two or more NICS all of them get a static MAC configured :

Screen Shot 2015-09-09 at 15.03.07

Hope that you see the potential in Azure Automation and Happy automating!


Hyper-V Cluster Heartbeat for MDT ref VM goes bananas or?

I have been helping a customer with their environment and we had a problem that took me a while to figure out.

They were baking reference images for their SCCM environment and the best and easiest way is to use VM´s of course. The problem that occurred was that when the image was being transferred back to the MDT server the VM rebooted after half of the image had been uploaded….

So what was doing this crazy behavior? It took me a little while before I realized what it was all about and it had to do with the Hyper-V cluster platform and resilience and heartbeat functionality!

So at first the build VM boots from the MDT image, no integration tools yet then but then it restarts to install applications and stuff within the OS and as the customer works on a Windows 7 image you can see it starts to send heartbeat to the host.


As you might know, clients and servers since Windows Vista and 2008 have integrational services by default in them although best practice is to upgrade them as soon as possible if the VM shall continue to reside in Hyper-V.

The interesting part in this case was that the OS rebooted within itself when it was finished with sysprep to start the MDT image for transferring the WIM back to the MDT server and the cluster/hyper-v did not notice this and thus it thought that the heartbeat stopped.


And as it was a cluster resource this heartbeat loss was handled by default, and guess what, rebooted!

So what settings in the cluster resource does this madness? First of all, the Heartbeat setting on the cluster vm resource properties



This can be read on the technet site about Heatbeat setting for Hyper-V clusters:

Screen Shot 2015-06-04 at 10.35.22

And then you have policy what the cluster should do after it thinks the vm has become unresponsive:

Screen Shot 2015-06-04 at 14.14.56 1



There are different ways to stop the cluster from rebooting the machine and one of them is to disable heartbeat check and another is to set the response to failure to do nothing,

The customer uses mostly VMM console and for them when building a new VM for MDT reference builds they can set the integrational services to disable heartbeat check and thus not get their work postponed by unwanted reboots.


During the search for why I checked Host Nic drivers as I thought that it might have something with a transfer error but could not find anything, positively the hosts got up on the latest nic firmware and drivers 😉 . My suspicion that it had to be the cluster was awaken after I had spun up a test VM that was not part of the cluster and that one succeeded in the build and transfer.

This is a rare case and I would say that in 99% of the cases you want the default behaviour to happened as a VM can become unresponsive and then the cluster can try a reboot to regain into operations..

Clarification: If you spin up a VM with a OS or pxe image that does not have integrational services it will not reboot the VM after the cluster timeout, the OS has to start sending heartbeat to the Hyper-V host and then it will be under surveillance and managed by the cluster until properly shut down!

Hope that this helps for someone out there wondering what happens…



Taking the SCVMM 2012 R2 UR6 for a test drive

Noticed this evening that Microsoft released the UR6 for System Center and my interest is in Virtual Machine Manager so I wanted to test-install it  and also connect to an Azure IaaS subscription as this was one of the new added features besides all fixes and also of course the other added feature with Generation 2 Service template support etc.

Screen Shot 2015-04-28 at 20.15.14

Here you can read more about the fixes and also if you do not use Microsoft Update, download the files.

As I had my environment connected to the Internet I could press install,

Screen Shot 2015-04-28 at 20.18.03

Once it was finished a reboot of the server had to be done and I could start to add Azure subscriptions to VMM. Here you have to use a management certificate and that is easily created with makecert if you do not have any other CA available!

Screen Shot 2015-04-28 at 20.59.47

And when that is complete you can see my VM´s in Azure on the subscription and the commands that I can use on them,

Screen Shot 2015-04-28 at 21.03.15

Good luck in your tests of this nice new feature.


Exclude VM´s from dynamic optimization in SC VMM

In a case where we had a VM that was a bit sensitive with an application that does not like the ping loss during live migration between hosts in a cluster we wanted to exclude the vm from the automatic dynamic optimization. It might not be totally clear where you find this setting for the VM´s you want to exclude from this load balancing act.

First of all, do you know where you set up automatic dynamic optimization in System Center VMM 2012 R2? For some reason you set it up on the hosts folder and not on the cluster object in VMM:

Screen Shot 2015-04-16 at 13.00.13


So where do you exclude VM´s from this optimization? If you look under a VM´s properties you can check the actions and there you see the magic checkbox :

Screen Shot 2015-04-16 at 12.55.35

And now when the automatic job runs my sensitive VM will stay on the host.

SC VMM Error 803 after restoring a duplicate VM with alternative name

I was at a customer today testing some backup/restore scenarios with their backup providers software and where we got an interesting error: 803 “Virtual Machine restore-Test5_gen1 already exists on the virtual host other cluster nodes” Recommended Action: “Specify a new name for the virtual machine and then try the operation again”



In the Backup/restore console we wanted to do a restore to alternative place and an alternative name as we had not deleted the original VM (for instance if you need files or just verify some state or something), in the configuration of the restore job we checked that the restore process would create a new VM Id which was the first thing we thought that was why VMM complained.

The thing was that this error only appeared when we did a restore to another hyper-v host, if we restored to the same where the original VM was residing there was no error..

As you can see after the restore both the original VM and the alternate had the same “#CLUSTER-INVARIANT#” id but different VMId´s, and when we tried to refresh VM´s we got the error above.

Screen Shot 2015-03-30 at 20.56.59

The solution was not so farfetched and can be read about in the KB2974441 although that case is about RDS VDI but still, and as can be read about why the ID is in the notes field from the beginning: “VMM adds a #CLUSTER-INVARIANT#:{<guid> } entry to the description of the VM in Hyper-V. This GUID is the VM ID SCVMM uses to track the VM.”

For the VM not showing up in VMM console we just went into the notes field on Hyper-V Manager and removed that specific “#CLUSTER-INVARIANT#” id and after that VMM generated a new for that VM and it appeared in the VM list on the VMM server.

So why was it no problem when we restored to that same host? For some reason VMM managed to see the duplicate residing on the same host and generate a new id in the notes field for that and thus appearing in the VM list without any massage..


Hyper-V VM´s BIN files, to be or not to be in clusters

If you create lots of VM´s with large amount of RAM memory assigned to them and start to wonder why you have used some of the storage on the volumes then this is because if you have set up a VM without changing anything you get a bin file in the VM folder that corresponds to the size of the allocated RAM. This file is used to save the VM`s RAM to disk when the VM is going into saved state!

Screen Shot 2015-02-19 at 12.23.41

In an environment where you have all VM´s as clustered resources you will not need to be able to use the saved state when shutdown the host as you will live migrate the VM´s when doing stuff with the HW.

The setting is easily found in Hyper-V Manager for an already deployed VM:

Screen Shot 2015-02-19 at 12.22.06

It is not so easily found in System Center VMM when checking the VM properties, but when deploying a new VM you find it in the wizard:

Screen Shot 2015-02-19 at 12.29.26

If you want to change the setting for your VM´s running in a cluster via VMM you will have to use PowerShell and it is quite easy to do that with a one-liner, first you see the setting with the PowerShell command Get-SCVirtualMachine and then you can configure it with Set-SCVirtualMachine :

Screen Shot 2015-02-19 at 12.40.18
Screen Shot 2015-02-19 at 13.29.46

And now when checking the folder for the VM the BIN file has magically shrunk to 4KB :-)

Screen Shot 2015-02-19 at 13.44.03

Error dialog when opening VMM console after update to VMM 2012 R2 UR5

I have updated a VMM server with the latest UR5 and now when opening the console the following error appears

Screen Shot 2015-02-18 at 11.17.31

This was quite easily handled as you can see the folder is there but looking a bit further I found that authenticated users of the had no NTFS rights on the particular folder.

Screen Shot 2015-02-18 at 11.21.33

After updating and adding authenticaded users as described in the release notes of the UR5 and giving it access to HostSIdeAdapters I did not get any error messages after starting the console