How to Enable CSV Cache

How to Enable CSV Cache

Rate This
  • Comments 12

Overview

Cluster Shared Volumes (CSV) Cache is a feature in Windows Server 2012 and beyond which allows you to allocate system memory (RAM) as a write-through cache.  The CSV Cache provides caching of read-only unbuffered I/O.  This can improve performance for applications such as Hyper-V, which conducts unbuffered I/O when accessing a VHD file.  Unbuffered I/O’s are operations which are not cached by the Windows Cache Manager.  What CSV Block Cache delivers is caching which can boost the performance of read requests, with write-through for no caching of write requests.

CSV Cache delivers caching at the block level, which enables it to perform caching of pieces of data being accessed within the VHD file.  This is fundamentally very similar in the value it provides to other solutions on the market you may already be familiar with.  The primary difference is that CSV Block Cache reserves its cache from system memory, where most solutions on the market today deliver cache in the form a PCI card with an SSD or flash which you add to the server.

CSV Cache is completely integrated into the Failover Clustering feature and handles orchestration across the sets of nodes in the cluster.

Deployment Considerations

CSV Cache will deliver the most value in scenarios where VMs are heavy read requests, and are less write intensive.  Scenarios such as Pooled VDI VMs or also for reducing VM boot storms.  Because the applicability of CSV Cache depends on the workload and your specific deployment considerations, it is disabled by default.  The customer feedback on CSV Cache has been overwhelmingly positive and we generally recommend turning it on for all scenarios, including both Hyper-V Clusters using CSV and Scale-out File Servers using CSV.

You can allocate up to 20% with Windows Server 2012 and 80% with Windows Server 2012 R2 of the total physical RAM for CSV write-through cache, which will be consumed from non-paged pool memory. 

  • Hyper-V – Our preliminary testing has found 512 MB to deliver excellent gain at minimal cost, and is the recommend starting point / minimal value if enabled.  Then based on your specific deployment and the I/O characteristics of the workloads in the VMs you may wish to increase the amount of memory allocated.  Since system memory is a contended resource on a Hyper-V cluster, it is recommended to keep the CSV Cache size moderate.  Such as 512 MB, 1 GB, or 2 GB
  • Scale-out File Server – It is recommended to allocate a significantly larger CSV Cache on a SoFS as physical memory is typically not a contended resource, you may want to allocate 4 GB, 6 GB, or even more...

There are two configuration settings that allow you to control CSV Cache.

  • CsvEnableBlockCache – This is a private property of the cluster Physical Disk resource.  It allows you to enable CSV Cache on an individual disk.  This gives you the flexibility to configure cache for read intensive VMs running on some disks, while allowing you to disable and prevent random I/O on other disks from purging the cache.  For example parent VHD’s with high reads you would enable caching on Disk1, and high writes for differencing disks the CSV cache could be disabled on Disk2.  The default setting is 0 for disabled, setting to a value of 1 enables CSV Block Cache on that disk. 
    Note:  In Windows Server 2012 R2 this property has been renamed to EnableBlockCache and is Enabled by default.
  • SharedVolumeBlockCacheSizeInMB – This is a cluster common property that allows you to define how much memory (in megabytes) you wish to reserve for the CSV Cache on each node in the cluster.  If a value of 512 is defined, then 512 MB of system memory will be reserved on each node in the Failover Cluster.  Configuring a value of 0 disables CSV Block Cache.  Note:  In Windows Server 2012 R2 this property has been renamed to BlockCacheSize.

Configuring CSV Cache

Windows Server 2012 R2

1. Open an elevated Windows PowerShell prompt

2. Define the size of the size of the cache to be reserved (example of setting to 1 GB)

(Get-Cluster). BlockCacheSize = 1024

Windows Server 2012

1. Open an elevated Windows PowerShell prompt

2. Define the size of the size of the cache to be reserved (example of setting to 1 GB)

(Get-Cluster). SharedVolumeBlockCacheSizeInMB = 1024

3. Enable CSV Cache on an individual disk (must be executed for every disk you wish to enable caching)

Get-ClusterSharedVolume “Cluster Disk 1” | Set-ClusterParameter  CsvEnableBlockCache 1

Considerations: 

The cache size can be modified with no downtime, however for the Hyper-V root memory reserve in the parent partition to be modified to accommodate the memory allocated to the CSV cache it does require a server reboot with Windows Server 2012.  To ensure resource contention is avoided, it is recommended to reboot each node in the cluster after modifying the memory allocated to the CSV cache.  On Windows Server 2012 R2 the Hyper-V room memory reserve will be dynamically updated and a reboot is not necessary.
Enabling CSV Cache on an individual disk requires that the Physical Disk resource be recycled (taken Offline / Online) for it to take effect. 

Interoperability:

CSV Cache will be disabled on:

  • Tiered Storage Space with heat map tracking enabled
  • Deduplicated files using in-box Windows Server Data Deduplication feature (Note:  Data will instead be cached by the dedup cache) 
  • ReFS volume with integrity streams enabled (Note:  NTFS is the recommended file system as the backend for virtual machine VHDs in production deployments)

Optimizing CSV Cache

The CSV Cache also provides a set of counters you can use to monitor the performance of the cache.  You can leverage the Performance Monitor tool (PerfMon.msc) to add the following counts to monitor different aspects of the CSV Cache.

Open Performance Monitor, and under Add Counters you will find “Cluster CSV Volume Cache” with the following counters.

I/O satisfied from cache:

  • Cache IO Read-Bytes
  • Cache IO Read-Bytes/Sec
  • Cache Read
  • Cache Read/Sec

I/O satisfied from disk:

  • Disk IO Read-Bytes
  • Disk IO Read-Bytes/Sec
  • Disk Read
  • Disk Read/Sec

Total I/O:

  • IO Read-Bytes
  • IO Read-Bytes/Sec
  • IO Read
  • IO Read/Sec

 

Thanks!
Elden Christensen
Principal Program Manager Lead
Clustering & High-Availability
Microsoft

 

Leave a Comment
  • Please add 6 and 4 and type the answer here:
  • Post
  • Many thanks Elden, great post, good explanation

  • when will you provide caching to flash (ssd or pci)? we are waiting for this in sql, file server,etc..

  • I've turned this on in my lab environment, and seen great improvement of read performance in our Server 2012 RC Test Cluster.

    Now--  can you provide a little more details on the resource allocation?  I.E.  If I were to allocate 20% of system RAM to this, would that cache get trimmed back as additional VMs are started (or ramp up their dynamic memory)?

  • I have only one file server and doesn't want to use file server cluster. Can I force file server read/write-back cache enabled regardless hyper-v's unbuffered io? otherwise it is so slow.

  • Is there any reason why this isn't enabled by default?

    Are there any drawbacks to CSV cache except the extra memory utilization?

  • In general we recommend turning on the CSV Cache for any/all Hyper-V or Scale-out File Server deployments.

    The only reason it is not enabled by default, is the default amount of memory may vary from deployment to deployment...  depending on the amount of physical memory in the nodes, the VM density you are trying to achieve, and for a Hyper-V deployment it may be less and for a Scale-out File Server it will be more.  So we just didn't have a good memory allocation number to set by default...  but it is something we are definitely thinking about

  • Great write up and post for Windows Server 2012.  This worked great on our Cisco UCS Blade Chassis deployments.

  • Does anyone have any more info on tuning the size of the cache?  I see the perfmon counters, but not really sure how to interpret them as far as tuning.  Can't seem to find much in any documentation.

  • I suggest you do a utility, which analyzes perflog or something, and offers to enable a certain amount of csv cache to each disk.

  • SAN vendors recommend for tired storage following relation: 5% - 25% - 70% (gold data - silver data - bronze data). This ratio may vary depending on read/writes IO. So, if you have 1 TB data your best size of cache - 50GB. You can use RAM or SSD accelerator card (HP, EMC, NetApp and others vendors have solutions with SSD).

  • Is there such a thing as having too large a cache? I'm running 4 blades with 384gb of memory each, and even on a busy day all our VM's combined don't use 300gb of ram. So I currently have 200gb of csv cache per host but I'm unsure if that could have possible negative side effects.

  • QUESTION: Is this feature available to (Windows 7 Ultimate 64-bit) users running a native VHD environment?

    My OS is installed to \VHD\win7u.vhd. I don't even have a 'recovery' system partition. Not multiboot yet (working on it), designing "differencing disk layers" (conceptually), ie

    SOLID (BASE) READ-ONLY (pre-requisite of {grand}parent :( )

    window

    drivers

    -------------------------

    Parent (the 'thin, winfix' profile)

    Stuff you shouldn't be without, ie autoruns et al, AHK, MAYBE coolMidi, Everything (winblows search replacement) Sandboxie? ...

    -------------------------

    DAW Child 1

    Barebones standalone profile with any connectivity tools (rtpMidi to iphone...)

    POWERuser Child 2

    Default profile, connected to web so avasted, defender/wse? along with office et al...

    TESTbed Child 3

    New apps.

    This is a draft off the top of my head. The main challenge I see, given the fact that, once you deploy a child (my understand) parent VHDs MUST be read-only, cannot be modified? ... makes planning (super) important.

    Is  "CSV" another thing for me to consider?!

    Thanks

Page 1 of 1 (12 items)