Write Buffers on VAST Clusters

Prev Next

VAST Clusters use write buffers to improve write performance. Write buffers are typically implemented in DBox NVRAM.

Write buffers were RAID 1 (mirrored) in VAST Cluster versions before 5.1. From version 5.1, write buffers are configured as RAID 6 (dual parity), which can recover from two hardware faults, and are enabled by default.  This provides efficient write performance with a relatively low write overhead for the RAID.

Enabling Write Buffer RAID on Older Clusters (pre-5.1)

Older versions of the VAST Cluster (pre-5.1) implemented write buffers as RAID 1 (mirrored disks).

You can convert the write buffers from this older layout to the RAID 6 layout used in later versions of the Cluster, and benefit from the increased write performance  and storage efficiency that his layout provides.

The process converts the layout of all write buffers on all NVRAMs in the Cluster. The Cluster, and the write buffers are fully functional during this process. The process, once started, is irreversible and cannot be stopped. It can take several hours, depending on the number of DBoxes.

Prerequisites for Write Buffer RAID Conversion

These conditions must be satisfied in order for the conversion process to be started:

  • Conversion to Write Buffer RAID can only be enabled on clusters that were deployed with VAST Cluster 4.7 or later.

  • No other rewrite operation is in progress (for example, encryption, similarity)

  • No cluster expansion operation is in progress (for example, DBox expansion, migration, or replacement activities)

  • The cluster is not in NDU (except hot fixes)

  • No devices are in a failed state

  • The cluster has at least 4 DBoxes; if DBox HA is enabled, then at least 15 Ceres or 22 Maverick DBoxes are required

  • The cluster does not  have Carrier DBoxes (TLC drives)

Enabling Write Buffer RAID Conversion on the Cluster
  1. Navigate to the Settings page, and select Cluster and Data Management

  2. In the Activate Write Buffer RAID section, enable the switch. This starts the conversion process. Once started, it cannot be stopped, and is irreversible.

Using Flash for Additional Write Buffer Storage

A VAST Cluster can be configured to increase the amount of write buffers available to handle bursts of high bandwidth writes. This  can increase the overall write speed during the burst. The write buffer capacity is dynamically increased by using flash drives for write buffers.

On new installations, this feature is enabled by default.

The following limitation applies:

  • Use of flash write buffers on an EBox with the DBox HA capability enabled requires at least 10 boxes to be present on the cluster.

Benefits and Impact of Using Write Buffers on Flash

Using flash drives to store write buffers can improve the overall write speed for the cluster, especially during bursts for high bandwidth write operations such as, for example, AI checkpointing operations during model training. The improvement can be around 2-2.5X.

The implementation of this feature on a VAST Cluster allows up to a certain proportion of space on flash drives to be allocated for write buffers, if the over use of flash storage remains below a threshold level. Allocation of write buffer stripes on flash storage is done gradually, according to need, up to the limit.

Enabling or Disabling the Use of Write Buffers on Flash from the VAST CLI

Enabling Use of Flash Write Buffers
  • Run this cluster modifycommand to enable the use of write buffers on flash.

    vcli: admin> cluster modify --enable-use-flash-write-buffers
Disabling Use of Flash Write Buffers
  • Run this command to disable the use of write buffers on flash.

    vcli: admin> cluster modify --disable-use-flash-write-buffers

    When disabled, any allocated write buffers on flash are flushed and the allocated space is returned.