Rescue DataStore by automating Storage vMotion

Storage VMotion to the Rescue

When a datastore is about to run out of space, the fastest resolution may be to simply migrate virtual disks to another datastore.   VMware Storage VMotion provides that capability with zero downtime for VMs and no disruption to end users.  Fortunately, PowerCLI can perform this feat with ease, thanks to the Move-VM cmdlet.

Let’s take a look at a functional prototype PowerCLI PowerShell script:

Add-PSSnapin VMware.Vimautomation.Core Connect-VIServer localhost $vmToMove = get-vm -Datastore $env:VMWARE_ALARM_TARGET_NAME | select-object -first 1 $destDS = Get-Datastore | where {$_.FreeSpaceMB -gt 50000 -and $_.Accessible -eq $true} | select-object -first 1 if ($destDS) {             move-vm -VM $vmToMove -Datastore $destDS -RunAsync }

This script is a proof-of-concept that is not ready for your production environment as it is — it just picks an arbitrary VM from the nearly-full datastore, finds another datastore with at least 50GB free, and moves the VM disks. More comprehensive selection logic and error checking are needed for a critical task like this.

Save your script on the vCenter Server system somewhere, such as C:\scripts\datastore.ps1.

Create the Datastore Alarm

Create a new alarm at an appropriate level in the vCenter hierarchy, such as a datacenter, and configure like this:

On the Triggers tab, add a “Datastore Disk Usage (%)” triggers to alert at a reasonable percentage — I opted for 93.

Run PowerShell Directly from vCenter Server

For whatever reason, PowerShell.exe does not do well when launched directly by another process — it tends to hang instead of exiting when it is finished.  As a workaround, it can be launched from cmd.exe as long as it receives something on standard input.  To do all that, the necessary code looks like this:

"c:\windows\system32\cmd.exe" "/c echo.|powershell.exe -nologo -noprofile -noninteractive c:\scripts\datastore.ps1"

For an alternate approach, take a look at the intermediate batch file solution described by Carter Shanklin in the link above.

On the Actions tab, add a “Run a command” action and supply the appropriate command.  You also need to decide whether to run one time or repeat the action.

Action!

To test the alarm, either fill up the datastore or temporarily lower the alarm threshold.  When the alarm fires, a Storage VMotion should be seen in the vSphere Client:

Note the “Initiated by” column — that’s the machine account for this vCenter Server.  The PowerCLI script is kicked off from vpxd.exe, which is running as LocalSystem.

Additional information is available by looking at the Tasks & Events tab for the datastore.  Here you can see a sample sequence of events, newest on top:

This automated Storage VMotion recovery alarm is a safety valve that could help you avoid suddenly running out of space on a datastore.  It should not take the place of more proactive storage management, but it sure beats VM downtime.

In case you are wondering: No, you can’t do the same thing with Hyper-V because Hyper-V does not have zero-downtime Storage VMotion.  Just another reason to choose VMware vSphere — as if you needed another reason.