Avoid / Defer Costly Storage Upgrades with Azure File Sync (Part 2)

In my previous post, I showed you how to drastically cut down on the files clogging up your file servers…so you can push off SAN / storage upgrades.

What I didn’t show you were common ways to reclaim the storage for other hosts… that’s what I’ll do now.  I did this process on a VMware-hosted Windows Server, so I’ll show you the steps I used, as well as some other options you can consider.

Shrink the VMDK
(or VHD or whatev..)
– You have Options

The typical way I approach freeing up space for a VM is to compress the disk file – there are several tools (including those from) VMware to do this.  I’m going to start out with options for VMware-hosted VMs, but the principles apply to Hyper-V and other platforms as well.

Depending on your storage subsystem, you likely will want to zero out the now empty portion of your VMDK – that would allow the zeroed blocks to be deduplicated.
To zero out Windows VMs, I use SDelete (https://docs.microsoft.com/en-us/sysinternals/downloads/sdelete) with the “-z” option – something like:

sdelete.exe -z E:

I used the 64bit version that’s in the download, and it took about 6 hours to do the cleaning and purging process – all while the file server was still up and available.

BigFileServer-SDelete3

Again, depending upon your storage platform, you may at this point be able to deduplicate the VMDK and save a ton of space.  If your storage doesn’t support deduplication, then you likely have a few more steps, depending on your ultimate plans and goals.

For a thin provisioned VMDK, you should be able reduce the file size using vmkfstools as outlined here  (and documented on the VMware site).  Thick provisioned VMDKs are trickier… but still something worth tackling.

If your ultimate goal is to move the VM somewhere else – like to Azure (either native IAAS or to VMware in Azure using Azure VMware Solutions) then the migration tools themselves will likely resolve the VMDK size issue as part of the move (using Azure Site Recovery or VMware HCX to AVS). If, however you want the VM to remain on premises, you’ll still want to make it smaller.  You can convert the VMDK from thick to thin provisioned simply use VMware Converter or clone the VM in vCenter.  Either way, the results are the key…. and getting to less storage utilization (or stopping uncontrolled storage growth in your data center) is the goal.

I merely shutdown the existing VM and cloned it – changing the storage to “thin” for the new VM.  This process did take some time (about 2 ½ hours of downtime), but took the data disk down from 1TB to less than 9GB!

Here’s what the files looked like when I started:

VMwareVMDKs

…and after cloning to “thin”:

VMwareVMDKs SHRUNK

Yes, there was downtime as part of this process – but that’s the only server downtime that occurred through the entire process!

If my storage supported de-duplication or I used something like HCX to move the VM, I likely would have seen little if any downtime.

Want Less Downtime?
Don’t Want to Shrink?
Replace the VM Instead!

Another way to avoid downtime would be to create a new VM that’s part of the Azure File Sync setup.  Azure File Sync supports replicating files to multiple hosts from a single Azure File Share – that means you could create a new VM with a smaller, thin provisioned VMDK and use it to host the tiered files.

You’ll want to install Azure File Sync, and for everything to work seamlessly, the new file server will need to be in the same AD domain, same drive letters (easier), same high-level directory juju – and there are a few other things to consider:

  • Server name – you may want the new server to have the same name (respond to requests for files) like the old VM. You could rename the server to match, or use DFS-N, or just use a new server name (not a bad option for some situations!)
  • Shares and share permissions – Azure File Sync replicates directories, files, and permissions, but doesn’t do anything with the actual shares defined on the server. You’ll want to migrate (backup and restore) those shares / permissions to the new server

You could create and manage the new shares manually if there are only a few, but I’ll share some options incase you want to automate the migration of shares.

For me, to make things simple, I used two VMs that were in the same AD Domain, on the same subnet, both with files replicated to the same directory on the D: drive.

Windows Server Migration Tools – Awesomeness Hiding Deep Within

If you are working with Windows Server 2012 or newer, you should be able to export and import the shares using the Windows Server Migration tools as outlined here:  https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/jj863566(v=ws.11).

There’s a lot to go through in that article, but the basic steps that worked for me were to run the following commands on both my source (VM with existing shares) and target (new VM without shares) servers:

Install-WindowsFeature migration
Add-PSSnapin Microsoft.Windows.ServerManager.Migration

Once the feature is installed and the cmdlets are available, you’ll want to check your firewall settings – you’ll need to open port 7000 on the destination server.

Moving the shares to another host is super easy at this point – just one command on each host / VM.  It’s easier since I have the same drive letter and path – what I did here was run the following command on the source system:

send-smigserverdata -computername ws2019 -sourcepath "d:\File Shares" -destinationpath "d:\File Shares" -include share -recurse -verbose

The syntax of the command is simple… the “-computername” is the host you are copying the share information to… note that you want to be sure to include the “-include” so you only copy the share info, as well as the “-recurse” to get all the shares.

On the receiving VM, it’s much simpler… you just must start the receive command:

receive-smigserverdata

You’ll also have to include a super-secret password to be use between the two systems on each side (there’s a prompt).

Once things get going, it just takes seconds – here’s what the Target looks like:

Target

One the source system, you’ll see a little more:

Source

The update is super quick – and suddenly shares that were not there are available on the remote server:

SharesBefore

SharesAfter.jpg

At this point you could rename the server or leverage DFS to update things in your infrastructure…but I’ll leave that up to you.

Other Scripts / Tools

There are a lot of scripts out there to save and restore shares and share permissions, but the Migration tools above worked for me, so that’s what I tested.

Storage Migration Service (SMS):  What’s Coming… and it’s cool!

Microsoft should be adding support for this entire processing out later this year (2020).
When you are migrating from a large set of storage on WS2008, 2012 or whatever to a tiered Azure File Sync (AFS) destination, SMS (Storage Migration Services – not txt messaging) will understand the pause and transfer logic needed to let AFS tier data without overflowing. SMS is going to be much faster to transfer than Internet pipes to Azure, typically, so we will make the experience is pleasant. It’s comin’.  Ned talked about it at Ignite already (you can Bing that if you want!).

Below are some cool images Ned (“Mr. SMB1”) shared with me that lay it all out:

SMS1

SMS2