VAST Clusters use data reduction mechanisms to maximize storage efficiency and save space. When a write occurs, data is first written synchronously to Write Buffers in NVRAM. Data reduction is then performed on the data in NVRAM before it is migrated to SSDs. VAST implements three distinct reduction techniques: deduplication, compression, and similarity-based data reduction. During this migration process, deduplication is executed first, followed by compression or similarity based on which yields the highest reduction. Finally, the reduced data is written to the SSDs, and the original data is deleted from NVRAM.
Deduplication identifies identical blocks of data during migration. If the exact same data already exists on the SSDs, writing the duplicate block is avoided. Instead, only a pointer to the existing data is stored, effectively saving the entire size of the duplicate block.
Compression reduces the size of migrating data by identifying repeating patterns. A specialized algorithm is used to compress these patterns before the data is written to the SSDs. The total storage savings achieved through this method depend on the specific data patterns and the compression algorithm.
Similarity-based data reduction is an optional feature that detects data blocks that are similar but not identical. When similarity is detected, the existing similar block is decompressed and both blocks are compressed together, storing only the unique changes. The similar blocks are linked together in clusters, which can be further optimized later through re-clustering.