Extend the Capacity and Life of Existing Storage Infrastructure by Tiering Files to Azure
Data storage requirements continue to increase every year, forcing infrastructure / storage teams to constantly review capacity and search for new ways to reduce the cost curve for storage and retention.
Much of what is being stored and saved resides in files – places on shares and often untouched for months, years, or ever – yet left unchecked files will replicate like tribbles… clogging and cluttering storage subsystems in computer rooms and closets until it’s time for a costly upgrade to increase capacity.
Why not transparently tier old files to lower-cost storage in Azure to free up local storage capacity on Window file servers, and avoid costly server or storage upgrades? You can do just that with Azure File Sync without disrupting users or even (hopefully) a reboot.
Azure File Sync is a service for Windows Server that replicates files to Azure and if desired, tier them / “stub them out” on the file server to reduce required storage capacity.
It is installed via an agent for supported versions of Windows Server (2012 R2, 2016, 2019), and connects to an Azure File Share using a Storage Sync Service in Azure:
The benefit to your storage infrastructure in terms of capacity come from enabling tiering when you connect the agent to the Azure File share (through the Sync Service), which you can do on a per server basis – as well as set a free capacity target and modification date thresholds for keeping files local:
Once files are synced from you server to Azure (and tiered / “stubbed”), file access for users and applications don’t really change, but the space on your server certainly can! Here’s an example where about 800+GB of files were tiered from a file server:
The files still look like they are still local to the file server – since the file metadata is local, but the actual data may be in Azure.
When a user accesses the file, the File Sync Agent transparently recalls the file from Azure and caches it on the local server – very similar to how OneDrive or One Drive for Business might perform.
Imagine the space that could be freed up on constrained branch office servers as well as large file servers… pushing out server replacements or SAN capacity upgrades! The added benefit is backup centralization…with files replicated to Azure, you can manage backups there… taking additional strain off your data center infrastructure – I’ve shown how to install and configure Azure File Sync previously along with using it for backup and recovery.
Now I’d like to show you a real-life example of how to shrink down a VMware-based Windows Server.
I’m going to show you the steps I used to significantly reduce the local storage used for a VMware-based Windows File Server.
My server is a Windows Server 2016 VM with two VMDKs: one 60GB boot disk and a 1TB data disk (thick provisioned):
1TB in that data VMDK is alot of local storage – and I can free most of that up without much effort! The file hold It has the same 800+GB of files I showed earlier, except that they are all local and inside the data VMDK on the E: Drive:
Manually Tiering Files
After installing Azure File Sync and synchronizing the files, I tiered all the local files on the E: drive by running this simple bit of PowerShell inside the VM:
Import-Module “C:\Program Files\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll” Invoke-StorageSyncCloudTiering -Path "E:\File Shares"
I could have waited for the bulk of the files to “stub out” but why, when you can run some PowerShell! This process had significant impact on the storage used inside the VMDK:
WOW! That’s a HUGE reduction in file space!
Still, that doesn’t necessarily clear up space on your storage… you still have to compress the VMDK (or VHD / VHDX).
In my next post – I’ll talk through how to reclaim the space.