Overview
This KB covers the relevant information for the following alerts:
Cluster MD Block usage is above 88% /100%
The alerts usually mean the system is approaching its MD Block limits.
Background
The system contains DBoxes; each DBox has a limit on the number of MD blocks, which depends on the DBox model and the SCMs.
In some scenarios, the system can reach the MD block limit before reaching the capacity/handle limits of the system. Some features that can consume many MD blocks are snapshots (especially the snapshot policy for long-term storage) and a vast catalog.
Monitor
For looking into the MD Block usage, it is shown in the main dashboard in the VMS GUI - in this case, the MD Block used space is 66.22%(relevant metric for MD Block usage in Analytics is MD Usage percent, this metric is a combination of the MD Block and Token mapper usage)
Data reduction UI screenshot
For tracking MD usage on the cluster, please create a graph in Analytics called MD usage that includes the following metrics: Capacity,md_total_blocks, Capacity,md_used_blocks, and Capacity,md_used_blocks_percent.
In this example, the total MD blocks of the cluster are 1077004800, the used MD Blocks in the system are 649013413, MD usage percent is ~60.2% → This percent is according to the total MD usage of the system → since 4.7.0-sp18, the MD Block usage limit is normalized to 100% according to the limit of 91%, the calculation for normalizing the MD usage in such case is: (649013413 /(1077004800* 0.91)) * 100 = ~66.22%
MetaData usage graph
Troubleshooting
A new feature introduced in 5.1 may help reduce the system's MD block usage.
If the upgrade to 5.1 is not possible and the VAST Catalog is enabled, our recommendation is to consider disabling the VAST Catalog, since it may consume a lot of MD usage. A new feature was introduced in 5.1 that helps to decrease the MD that is consumed by the VAST Catalog.
If the system is already in 5.1 or above, our recommendation is to consider removing or deleting unnecessary data or snapshots.
In case deleting data or removing snapshots is not an option, one of the options is to consider expanding the system with additional storage in the cluster.
Related Articles
Capacity terms can be found at VAST Cluster 5.3 Administrator's Guide (requires a valid user to access the Vast customer portal) under the Capacity section.