Documentation Index

Fetch the complete documentation index at: https://kb.vastdata.com/llms.txt

Use this file to discover all available pages before exploring further.

Block Storage on VAST

Prev Next

Overview

VAST Cluster supports the block storage protocol. The block storage service lets you  set up and manage paths on the cluster like block storage systems and split up the cluster’s space into block storage volumes under those paths.

VAST Cluster supports connectivity to block storage using Nonvolatile Memory Express over Transport Control Protocol (NVMe/TCP).

Block storage on the VAST platform is configured through subsystems (block-enabled views) that contain one or more volumes (NVMe namespaces). These volumes are mapped to initiator hosts using their NQN and accessed via a VIP pool, supporting multipath I/O for high availability and load balancing, similar to VIP usage for NFS in a multi-tenant environment.

Supported Client Environments

  • Linux: RHEL 8.8, RHEL 9.4, Ubuntu 24.04 LTS

  • VMware vSphere ESXi 7.0 U3, ESXi 8.0

Host Connectivity

  • NVMe/TCP (only)

Multipath Support

VAST supports native NVMe multipathing on Linux:

  • Enabled by default on RHEL 9+, optional in RHEL 8 (nvme multipath).

  • Clients should connect to all available VIPs in the pool.

  • VAST returns a maximum of 16 VIPs per subsystem on discovery

  • block-protocol-considerations

Note: Up to 16 paths (one path per virtual IP). During path discovery, up to 16 virtual IPs are chosen from the list of all possible virtual IPs across all virtual IP pools belonging to the relevant tenant.

BLOCK_MAX_HOST_TO_SUBSYSTEM_VIPS - 16 (min 1, max max-number-of-vips). Limit the number of vips to which a host connects to a subsystem.

If you want to connect to more than 16, you can, but you will need to connect manually to each vip.

e.g.

for i in {1..32}; do sudo nvme connect -a 172.21.192.$i -t tcp -q <host_nqn> -n <subsystem_nqn>; done

Configuring an NVMe/TCP Client on Linux for VAST Cluster Block Storage

Installing the Client

To configure client block hosts for interacting with the cluster as a remote NVMe device, install the NVMe CLI tool

[root@centos8-var-1 ~]# sudo yum install nvme-cli
/usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.20) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)

Last metadata expiration check: 1:12:01 ago on Friday 01 August 2025 03:39:12 AM GMT.
Package nvme-cli-1.14-3.el8.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
[root@centos8-var-1 ~]#

Verify the NVMe Version

[root@centos8-var-1 ~]# nvme version
nvme version 1.14
[root@centos8-var-1 ~]#

Obtaining the Host NQN: Host NQN stands for Host NVMe Qualified Name. It's a unique identifier used by a Linux (or other OS) system when connecting to NVMe over Fabrics (NVMe-oF)

[root@centos8-var-1 ~]# cat /etc/nvme/hostnqn
nqn.2014-08.org.nvmexpress:uuid:ec2ace17-a853-4561-86dc-567f491521ab
[root@centos8-var-1 ~]#

Create a Block Storage Subsystem View

The components of a block storage configuration are:

  • Subsystems:  A subsystem is an NVMe container for volumes or namespaces. It is configured with a unique NVMe identifier, an NQN, which makes it discoverable by hosts as a remote block storage device. A subsystem on the VAST Cluster is configured as a block-enabled view on a dedicated path.

  • Volumes:  Units of capacity that can be provisioned on a subsystem. Each volume has a path relative to the subsystem path. The path can be backed up with snapshots. Snapshots of volumes can also be presented to hosts as volumes.

  • Hosts: NVMe/TCP hosts must be defined on the cluster with their NQN, a unique identifier created on each host by the NVMe/TCP client host configuration.

Block storage subsystems are provisioned on the cluster as views. To create a subsystem, create a view on an empty directory path and enable Block as the view's access protocol. Views that are block-enabled are configured on dedicated paths that are not accessible to other protocols.

To make volumes available to block hosts, they must be assigned or mapped to the hosts.

Procedure

  1. In the VAST Web UI, select Element Store from the left navigation menu and then select Views.
    Click Create View to add a new view.

  2. Select Policy Name

  3. Select Block as a Protocol

  4. SubSystem NQN: This would generate a Post specifying the SubSystem Name and create the Block View.

Up to 16 network paths are supported on client connections to block subsystem views. One virtual IP pool with a protocol role is used per path.

NOTE: The following are the only view policy settings that are relevant to block view configuration:
1. Tenant
2. VIP Pools

All security flavor options are compatible with block-enabled views. None of the remaining view policy settings affect block storage.

Add a Block view

Right-click on the Block View → View. You will see the SubSystem NQN Generated.

View the SubSystem NQN Generated

Create a Block Volume

Navigate to the Volumes Tab in the Element Store. Click on Create Volume.

NOTE: The Volumes and Hosts tabs appear only when at least one block-enabled view is configured on the cluster.

View element store for new volume

You can choose to select Single Volumes or Multiple Volumes at Once. Specify the Volume Name and Capacity. Tags are Optional. Click Create

Create new volume

The Volume will be created

Newly created volume

Creating Block Host

Navigate to Element Store → Host and Click Create Host.

Adding a block host

  • Select either Single host to create one host or Multiple hosts to create multiple hosts.

  • Select Tenant

  • Enter the Name. The hostname must be unique on the Tenant.

  • Enter the host's NVMe Qualified Name (NQN), a unique identifier used to identify the host in NVMe operations.

NOTE: You can add the same host to multiple Tenants using its identifier.

Optionally, you can specify the tag

Click Create

Enter Host's NQN

The Host is added

View new created host details

Assigning Block Hosts to Volumes

Assigning a block host to a volume enables the host to discover the volume. Assignments are managed by a mapping of hosts to volumes on the same tenant.

Navigate to the Volumes Tab in the Element Store. Right-click the Volume and select Manage Mapping.

Assign host to volume

Select any hosts that you want to assign to the volume. Deselect any hosts that you want to unmap from the volume. Click Save Mapping.

Select host to be assigned to volume

You can view this on the Activities Page.

View details on Activites page

Once completed, you can see that the Volume is mapped to the Host.

View Host to volume mapping in Element Store -> Volumes

CLI processes

You can list the Block Hosts as well as Volumes that were created using vcli

vcli: admin> blockhost list
+----+-----------+---------------+----------------------------------------------------------------------+-----------------+-------------+------------------------+---------------------+
| Id | Tenant-id | Name          | Nqn                                                                  | Tags            | Tenant-name | Mapped-volumes-preview | Mapped-volume-count |
+----+-----------+---------------+----------------------------------------------------------------------+-----------------+-------------+------------------------+---------------------+
| 1  | 1         | centos8-var-1 | nqn.2014-08.org.nvmexpress:uuid:ec2ace17-a853-4561-86dc-567f491521ab | {'Dept': 'Dev'} | default     | Volume1 (id=1)         | 1                   |
+----+-----------+---------------+----------------------------------------------------------------------+-----------------+-------------+------------------------+---------------------+

vcli: admin> volume list
+----+---------+---------+----------+-----------------+--------------+--------------------------------------+--------------------------------------+--------------+-----------------------------+-------------+----------------------------+-------------------------+
| Id | View-id | Name    | Size(GB) | Tags            | Namespace-id | Nguid                                | Uuid                                 | Capacity(GB) | Snapshot-data               | Tenant-name | Mapped-block-hosts-preview | Mapped-block-host-count |
+----+---------+---------+----------+-----------------+--------------+--------------------------------------+--------------------------------------+--------------+-----------------------------+-------------+----------------------------+-------------------------+
| 1  | 85      | Volume1 | 1.000    | {'Dept': 'Dev'} | 1            | 1257a64f-ff95-447f-8c87-26acc6fc700b | 1257a64f-ff95-447f-936c-a7acc6fc700b | 0.000        | {'id': None, 'name': 'N/A'} | default     | centos8-var-1 (id=1)       | 1                       |
+----+---------+---------+----------+-----------------+--------------+--------------------------------------+--------------------------------------+--------------+-----------------------------+-------------+----------------------------+-------------------------+

To view the mapping of Volumes to the Block Host using vcli

vcli: admin> blockmapping list
+----+-----------+--------------+-----------------------------+
| Id | Volume id | BlockHost id | Snapshot-data               |
+----+-----------+--------------+-----------------------------+
| 1  | 1         | 1            | {'id': None, 'name': 'N/A'} |
+----+-----------+--------------+-----------------------------+

vcli: admin> blockmapping show --id 1
+---------------+-----------------------------+
| Id            | 1                           |
| Volume id     | 1                           |
| BlockHost id  | 1                           |
| Snapshot-data | {'id': None, 'name': 'N/A'} |
+---------------+-----------------------------+
vcli: admin>

Connecting to Mapped Volumes from Host

After volumes have been mapped to the host via VMS, you can connect to them as follows.

Load the necessary kernel modules to enable NVMe over Fabrics.

[root@centos8-var-1 ~]# sudo modprobe nvme
[root@centos8-var-1 ~]# sudo modprobe nvme-fabrics
[root@centos8-var-1 ~]#

To have the modules load automatically on reboot:

Create a file /etc/modules-load.d/nvme.conf and add the below entries inside.

nvme.conf
nvme
nvme-fabrics

Run below.

The command sudo dracut -f in CentOS (or RHEL-based systems) is used to regenerate the initramfs (initial RAM filesystem) for the current kernel.

[root@centos8-var-1 ~]#  sudo dracut -f
[root@centos8-var-1 ~]#

Discover available NVMe subsystems over TCP

[root@centos8-var-1 ~]# sudo nvme discover -t tcp -a 172.27.33.1 -s 8009

Discovery Log Number of Records 4, Generation counter 47
=====Discovery Log Entry 0======
trtype:  tcp
adrfam:  ipv4
subtype: nvme subsystem
treq:    not required
portid:  1
trsvcid: 4420
subnqn:  nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1
traddr:  172.27.33.1
sectype: none
=====Discovery Log Entry 1======
trtype:  tcp
adrfam:  ipv4
subtype: nvme subsystem
treq:    not required
portid:  1
trsvcid: 4420
subnqn:  nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1
traddr:  172.27.33.2
sectype: none
=====Discovery Log Entry 2======
trtype:  tcp
adrfam:  ipv4
subtype: nvme subsystem
treq:    not required
portid:  1
trsvcid: 4420
subnqn:  nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1
traddr:  172.27.33.3
sectype: none
=====Discovery Log Entry 3======
trtype:  tcp
adrfam:  ipv4
subtype: nvme subsystem
treq:    not required
portid:  1
trsvcid: 4420
subnqn:  nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1
traddr:  172.27.33.4
sectype: none
[root@centos8-var-1 ~]#

Connect to a subsystem:

  1. Obtain the subsystem NQN from VMS:

    1. In the VAST Web UI, open the Views tab on the Element Store page.

    2. Find the subsystem view.

    3. Right-click the view and select View to see its configuration.

      The NQN is displayed in the Subsystem NQN field.

Get the subsystem NQN for the host

nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1

Establish Connection to Subsystem

[root@centos8-var-1 ~]# sudo nvme connect -t tcp -n nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1 -a 172.27.33.2
[root@centos8-var-1 ~]#

Run the connect-all command to connect all paths:

[root@centos8-var-1 ~]# sudo nvme connect-all -t tcp -a 172.27.33.1 -s 8009

To List the Subsystem Paths

[root@centos8-var-1 ~]# sudo nvme list-subsys
nvme-subsys0 - NQN=nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1
\
 +- nvme0 tcp traddr=172.27.33.2 trsvcid=8009 live
 +- nvme1 tcp traddr=172.27.33.1 trsvcid=4420 live
 +- nvme2 tcp traddr=172.27.33.2 trsvcid=4420 live
 +- nvme3 tcp traddr=172.27.33.4 trsvcid=4420 live
 +- nvme4 tcp traddr=172.27.33.3 trsvcid=4420 live
[root@centos8-var-1 ~]#

To see a list of connected NVMe volumes, use the nvme list command:

[root@centos8-var-1 ~]# sudo nvme list
Node                  SN                   Model                                    Namespace Usage                      Format           FW Rev
--------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1          VastData             VastData                                 1           0.00   B /   1.00  GB    512   B +  0 B   24.05
/dev/nvme1n1          VastData             VastData                                 1           0.00   B /   1.00  GB    512   B +  0 B   24.05
/dev/nvme2n1          VastData             VastData                                 1           0.00   B /   1.00  GB    512   B +  0 B   24.05
/dev/nvme3n1          VastData             VastData                                 1           0.00   B /   1.00  GB    512   B +  0 B   24.05
/dev/nvme4n1          VastData             VastData                                 1           0.00   B /   1.00  GB    512   B +  0 B   24.05
[root@centos8-var-1 ~]#

To get the list of block devices

[root@centos8-var-1 ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  100G  0 disk
├─sda1        8:1    0  600M  0 part /boot/efi
├─sda2        8:2    0    1G  0 part /boot
└─sda3        8:3    0 98.4G  0 part
  ├─cl-root 253:0    0 60.8G  0 lvm  /
  ├─cl-swap 253:1    0  7.9G  0 lvm  [SWAP]
  └─cl-home 253:2    0 29.7G  0 lvm  /home
sr0          11:0    1  9.3G  0 rom  /run/media/vj/CentOS-8-4-2105-x86_64-dvd
nvme0n1     259:0    0  4.7G  0 disk
nvme1n1     259:1    0  4.7G  0 disk
nvme2n1     259:2    0  4.7G  0 disk
nvme3n1     259:3    0  4.7G  0 disk
nvme4n1     259:4    0  4.7G  0 disk

Why do I see Multiple NVMe Devices here when I have created just 1 Volume on VAST?

The reason you see multiple NVMe devices (like /dev/nvme0n1 to /dev/nvme4n1even though you created only one volume on VAST, it is because of multipathing in NVMe over Fabrics (NVMe-oF).

Each of those entries represents a different path to the same volume. They aren't different volumes. There are multiple connections (paths) to the same backend device

For each device, look for the NQN and Serial Number (SN) — they will be the same across all devices.

[root@centos8-var-1 ~]# sudo nvme id-ctrl /dev/nvme1n1 | grep subnqn
subnqn    : nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1

[root@centos8-var-1 ~]# sudo nvme id-ctrl /dev/nvme2n1 | grep subnqn
subnqn    : nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1

[root@centos8-var-1 ~]# sudo nvme id-ctrl /dev/nvme3n1 | grep subnqn
subnqn    : nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1

[root@centos8-var-1 ~]# sudo nvme id-ctrl /dev/nvme4n1 | grep subnqn
subnqn    : nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1
[root@centos8-var-1 ~]#

Run the command below. This tells us that the NVMe device /dev/nvme0n1 does not currently contain a recognizable filesystem or partition table. It's just raw data

[root@centos8-var-1 ~]# sudo file -s /dev/nvme0n1
/dev/nvme0n1: data

You can now format it with ext4 or the IFS file system

[root@centos8-var-1 ~]# sudo mkfs.ext4 /dev/nvme0n1
mke2fs 1.45.6 (20-Mar-2020)
Discarding device blocks: done
Creating filesystem with 244140 4k blocks and 61056 inodes
Filesystem UUID: 5da976e8-8a76-49c4-a887-5ddc3c99e5a7
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

[root@centos8-var-1 ~]#

Create a Mount Point


[root@centos8-var-1 ~]# sudo mkdir -p /mnt/nvme
[root@centos8-var-1 ~]#

Mount the Volume

[root@centos8-var-1 ~]# sudo mount -o noatime,nodiratime /dev/nvme0n1 /mnt/nvme
[root@centos8-var-1 ~]#
[root@centos8-var-1 ~]# df -h /mnt/nvme
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1    923M  2.4M  857M   1% /mnt/nvme
[root@centos8-var-1 ~]#

Basic NVMe Logs

This shows logs related to NVMe device detection, path creation, timeouts, and multipath activity.


[root@centos8-var-1 ~]# dmesg | grep -i nvme
[3453985.376141] TECH PREVIEW: NVMe/TCP may not be fully supported.
[3453985.390872] nvme nvme0: queue_size 128 > ctrl sqsize 32, clamping down
[3453985.391349] nvme nvme0: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 172.27.33.1:8009
[3453985.529705] nvme nvme0: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
[3454167.516170] nvme nvme0: queue_size 128 > ctrl sqsize 32, clamping down
[3454167.516717] nvme nvme0: creating 2 I/O queues.
[3454167.535001] nvme nvme0: mapped 2/0/0 default/read/poll queues.
[3454167.716701] nvme nvme0: new ctrl: NQN "nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1", addr 172.27.33.2:8009
[3454167.721640] nvme0n1: detected capacity change from 0 to 1000000000
[3454238.117614] nvme nvme1: queue_size 128 > ctrl sqsize 32, clamping down
[3454238.118137] nvme nvme1: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 172.27.33.2:8009
[3454238.119639] nvme nvme1: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
[3454238.323556] nvme nvme1: queue_size 128 > ctrl sqsize 32, clamping down
[3454238.324067] nvme nvme1: creating 2 I/O queues.
[3454238.335719] nvme nvme1: mapped 2/0/0 default/read/poll queues.
[3454238.337852] nvme nvme1: new ctrl: NQN "nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1", addr 172.27.33.1:4420
[3454238.342081] nvme1n1: detected capacity change from 0 to 1000000000
[3454238.346924] nvme nvme2: queue_size 128 > ctrl sqsize 32, clamping down
[3454238.347002] nvme nvme2: creating 2 I/O queues.
[3454238.363025] nvme nvme2: mapped 2/0/0 default/read/poll queues.
[3454238.365607] nvme nvme2: new ctrl: NQN "nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1", addr 172.27.33.2:4420
[3454238.373562] nvme2n1: detected capacity change from 0 to 1000000000
[3454241.446786] nvme nvme3: failed to connect socket: -110
[3454241.456412] nvme nvme3: queue_size 128 > ctrl sqsize 32, clamping down
[3454241.456500] nvme nvme3: creating 2 I/O queues.
[3454241.474347] nvme nvme3: mapped 2/0/0 default/read/poll queues.
[3454241.476391] nvme nvme3: new ctrl: NQN "nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1", addr 172.27.33.4:4420
[3454241.481066] nvme3n1: detected capacity change from 0 to 1000000000
[3454248.644095] nvme nvme4: queue_size 128 > ctrl sqsize 32, clamping down
[3454248.644242] nvme nvme4: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 172.27.33.1:8009
[3454248.645443] nvme nvme4: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
[3454248.655353] nvme nvme4: queue_size 128 > ctrl sqsize 32, clamping down
[3454248.655439] nvme nvme4: creating 2 I/O queues.
[3454248.673276] nvme nvme4: mapped 2/0/0 default/read/poll queues.
[3454248.675387] nvme nvme4: new ctrl: NQN "nqn.2024-08.com.vastdata:5dee4d30-4e49-5d5b-a0cd-6a0325b23984:default:subsystem1", addr 172.27.33.3:4420
[3454248.678838] nvme4n1: detected capacity change from 0 to 1000000000
[3454985.277074] EXT4-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: (null)
[root@centos8-var-1 ~]#

Check for Device Recognition

[root@centos8-var-1 ~]# dmesg | grep -i 'nvme[0-9]n1'
[3454167.721640] nvme0n1: detected capacity change from 0 to 1000000000
[3454238.342081] nvme1n1: detected capacity change from 0 to 1000000000
[3454238.373562] nvme2n1: detected capacity change from 0 to 1000000000
[3454241.481066] nvme3n1: detected capacity change from 0 to 1000000000
[3454248.678838] nvme4n1: detected capacity change from 0 to 1000000000
[3454985.277074] EXT4-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: (null)
[root@centos8-var-1 ~]#

Look for Errors/Timeouts

dmesg | grep -iE 'error|timeout|fail|nvme'

Watch Logs Live while Connecting

dmesg -w | grep -i nvme

Unmount the Volume

[root@centos8-var-1 ~]# sudo umount /dev/nvme0n1

To Expand a Volume on VAST

vcli: admin> volume modify --id 1 --size 5gb

vcli: admin> volume show --id 1
+----------------------------+--------------------------------------+
| Id                         | 1                                    |
| View-id                    | 85                                   |
| Name                       | Volume1                              |
| Size(GB)                   | 5.000                                |
| Tags                       | {'Dept': 'Dev'}                      |
| Namespace-id               | 1                                    |
| Nguid                      | 1257a64f-ff95-447f-8c87-26acc6fc700b |
| Uuid                       | 1257a64f-ff95-447f-936c-a7acc6fc700b |
| Capacity(GB)               | 0.210                                |
| Snapshot-data              | {'id': None, 'name': 'N/A'}          |
| Tenant-name                | default                              |
| Mapped-block-hosts-preview | centos8-var-1                        |
| Mapped-block-host-count    | None                                 |
+----------------------------+--------------------------------------+

Deleting a Volume

NOTE: Deleting a volume also deletes the volume directory and all data on it.

Volumes must be unmapped from hosts before being deleted.

Right-click the volume and select Remove.

Remove volume

You can choose to force unmap when you delete a single volume.

There is no force option for bulk volume deletion. You must unmap hosts from volumes before bulk deleting them.

Confirm deletion of volume

Deleting a Snapshot Volume

To delete a snapshot volume, unmap the volume from any mapped hosts. When you unmap hosts from a snapshot volume, the snapshot volume is automatically deleted, but the volume itself, if it still exists, is not deleted.

As we can see below, the Remove Option for Snapshot Volume would always be greyed out.

Remove Option for Snapshot Volume would always be greyed out

If you try to Delete a Volume via vcli, it will throw an error

vcli: admin> volume delete --id 2
Are you sure you want to delete the volume? [y/N] y
Command REST Error: {"detail":"Volume is mapped to hosts","code":"bad_request"}
vcli: admin>

Deleting a Host

Hosts must be unmapped from volumes before being deleted. You can choose to force-unmap a host to delete it.

Deleting a Host from the VAST Web UI

  1. Right-click the host and select Remove.

Deleting Host from VAST VMS UI

  1. If you want to force the deletion even if the host is mapped to volumes, enable the Force slider.

  2. Click Delete Host.

Confirm Host deletion

Data Protection

Create Protection Policy

A protection policy defines:

  • A schedule of points in time at which to take snapshots

  • Retention time for snapshots on the local cluster

  • A remote peer, which may be a replication peer (async replication) or an S3 replication peer (backup to S3). It is not specified whether the policy is intended for local backup only

Procedure

  1. Navigate to Protection Policy.

  2. Click Create Protection Policy

  3. Enter Policy Name, Tenant, Snapshot Prefix, Time

The name of each snapshot will be <prefix>_<timestamp>, where <prefix> is the prefix specified here and <timestamp> is the time the snapshot is created, in the format yyyy-mm-ddTHH:MM:SS.SSSSSSzzz (T denotes time and doesn't represent a value, zzz is the timezone, and the time is accurate to the microsecond). For example, if the prefix is dev, a snapshot taken at 8:15 pm UTC on 20th November 2024 would be named dev_2024-11-20T20:15:06.144783UTC.

If you want to make the protection policy indestructible, enable the Indestructible setting. This setting protects the policy and its snapshots from accidental or malicious deletion. Refer  indestructibility-overview

NOTE: After saving the protection policy, you won't be able to delete the policy or disable its indestructibility without performing a procedure for authorized unlocking of the cluster's indestructibility mechanism

Add Protection Policy

vcli: admin> protectionpolicy list
+----+-------------+--------------------------------------+------------------+-----------------+------------+----------------+-------------+--------------------+
| ID | Name        | Schedule                             | Replication Peer | Snapshot Prefix | Clone Type | Indestructible | Tenant name | Remote tenant name |
+----+-------------+--------------------------------------+------------------+-----------------+------------+----------------+-------------+--------------------+
| 4  | BlockBackup | [{'every': '15m',                    | N/A              | Dev             | LOCAL      | False          | default     | None               |
|    |             |  'keep_local': '1H',                 |                  |                 |            |                |             |                    |
|    |             |  'keep_remote': '0s',                |                  |                 |            |                |             |                    |
vcli: admin> protectionpolicy show --id 4
+--------------------+---------------------------------------------------------------------------------------------------+
| ID                 | 4                                                                                                 |
| Name               | BlockBackup                                                                                       |
| Schedule           | [{'every': '15m',  'keep_local': '1H',  'keep_remote': '0s',  'start_at': '2025-08-01 07:30:00'}] |
| Replication Peer   | N/A                                                                                               |
| Snapshot Prefix    | Dev                                                                                               |
| Clone Type         | LOCAL                                                                                             |
| Indestructible     | False                                                                                             |
| Tenant name        | default                                                                                           |
| Remote tenant name | None                                                                                              |
+--------------------+---------------------------------------------------------------------------------------------------+

Create Protected Path

Navigate to Data Protection → New Local Protected Path

Add a Protection Path

Select the Tenant, Name, Path to be Protected, and Protection Policy. Click Create

Select the Tenant, Name, Path to be Protected, and Protection Policy

vcli: admin> protectedpath list
+----+---------------+-------+--------+-------------+----------------+-------------------+------------------------+----------+-----------------+--------+----------------------+-------------+--------------------+---------------------+------------------------+--------------------------------+-------------------+----------------------------------------------+
| ID | Name          | Role  | State  | Path        | Path On Remote | Peer-cluster-name | Protection-policy-name | Is-local | Bandwidth(MB/s) | Health | Aggregated Usage(GB) | Tenant-name | Remote-tenant-name | Replication-streams | Sync Interval(Seconds) | Lease expiration time(Seconds) | Mode              | Sync replication connection timeout(Seconds) |
+----+---------------+-------+--------+-------------+----------------+-------------------+------------------------+----------+-----------------+--------+----------------------+-------------+--------------------+---------------------+------------------------+--------------------------------+-------------------+----------------------------------------------+
| 2  | S3LocalBackup | Local | Active | /abhinavs3  | None           | None              | S3Backup               | True     | 0.000           | OK     | None                 | default     | None               | ['S3LocalBackup']   | None                   | None                           | ASYNC_REPLICATION | None                                         |
| 4  | Block         | Local | Active | /test-block | None           | None              | BlockBackup            | True     | 0.000           | OK     | 0.0                  | default     | None               | ['Block']           | None                   | None                           | ASYNC_REPLICATION | None                                         |
+----+---------------+-------+--------+-------------+----------------+-------------------+------------------------+----------+-----------------+--------+----------------------+-------------+--------------------+---------------------+------------------------+--------------------------------+-------------------+----------------------------------------------+
vcli: admin> protectedpath show --id 4
+----------------------------------------------+-------------------+
| ID                                           | 4                 |
| Name                                         | Block             |
| Role                                         | Local             |
| State                                        | Active            |
| Path                                         | /test-block       |
| Path On Remote                               | None              |
| Peer-cluster-name                            | None              |
| Protection-policy-name                       | BlockBackup       |
| Is-local                                     | True              |
| Bandwidth(MB/s)                              | 0.000             |
| Health                                       | OK                |
| Aggregated Usage(GB)                         | 0.0               |
| Tenant-name                                  | default           |
| Remote-tenant-name                           | None              |
| Replication-streams                          | ['Block']         |
| Sync Interval(Seconds)                       | None              |
| Lease expiration time(Seconds)               | None              |
| Mode                                         | ASYNC_REPLICATION |
| Sync replication connection timeout(Seconds) | None              |
+----------------------------------------------+-------------------+

If you now navigate to the Snapshots Tab in Data Protection, you will see a Snapshot being triggered as per the Schedule.

The Name of the Snapshot has a prefix followed by a timestamp.

The Name of the Snapshot has a prefix followed by a timestamp

To check the Snapshot List via vcli.

We have 1 Snapshot here for the Protected Block Path


vcli: admin> snapshot list
+----+------------------------------+--------------+-------------+-------+----------------+-----------------------------+----------------------+----------------------+------------------+-----------+-------------------+
| ID | Name                         | Path         | Policy      | Type  | Indestructible | Created                     | Expiration-time      | Aggregated Usage(GB) | Unique Usage(GB) | Tenant-id | Subsystem-related |
+----+------------------------------+--------------+-------------+-------+----------------+-----------------------------+----------------------+----------------------+------------------+-----------+-------------------+
| 2  | Dev_2025-08-01_06_40_00_UTC  | /test-block/ | BlockBackup | local | False          | 2025-08-01T06:40:00.006000Z | 2025-08-02T06:40:00Z | 0.0                  | 0.0              | 1         | True              |
| 1  | test_2025-08-01_06_30_00_UTC | /abhinavs3/  | S3Backup    | local | False          | 2025-08-01T06:30:00.010000Z | 2025-08-03T06:30:00Z | 0.0                  | 0.0              | 1         | False             |
+----+------------------------------+--------------+-------------+-------+----------------+-----------------------------+----------------------+----------------------+------------------+-----------+-------------------+
vcli: admin>
vcli: admin> snapshot show --id 2
+----------------------+-----------------------------+
| ID                   | 2                           |
| Name                 | Dev_2025-08-01_06_40_00_UTC |
| Path                 | /test-block/                |
| Policy               | BlockBackup                 |
| Type                 | local                       |
| Indestructible       | False                       |
| Created              | 2025-08-01T06:40:00.006000Z |
| Expiration-time      | 2025-08-02T06:40:00Z        |
| Aggregated Usage(GB) | 0.0                         |
| Unique Usage(GB)     | 0.0                         |
| Tenant-id            | 1                           |
| Subsystem-related    | True                        |
+----------------------+-----------------------------+
vcli: admin>

Presenting Volume to Snapshots

You can back up a block volume accessible to hosts by mapping the host to a snapshot of the volume. This can be any snapshot on the volume path or on a path that includes it. It can be a snapshot taken manually or from a local protected path.

When you assign hosts to a snapshot of a volume, VMS generates a globally unique UUID and a namespace ID unique within the subsystem, and assigns them to the mapping, enabling the assigned hosts to access the snapshot as a read-only volume. (These identifiers are destroyed if all hosts are unmapped from the snapshot.)

Snapshots that are mapped to hosts cannot be deleted.

If a volume is deleted, but you have the volume backed up by a snapshot, you can make the data available by mapping the volume to the host from the snapshot. You need to know the name of the deleted volume.

Procedure

  1. In the Volumes tab, right-click the volume and select Manage Mappings.

  2. Select the hosts that you want to map to the snapshot.

  3. Click Available volume snapshots.

    All available snapshots of the volume are shown. You can see the ID, Name, Path, and Creation Time of each snapshot.

  4. Select the snapshot from which you want to map the volume to the selected host(s).

  5. Click Save And Go To Mapping.

  6. Click Save Mapping.

    The mapping task begins, and you can track it on the Activities page.

Map volume to hostSnapshots for that volume

Mapping a Volume from a Snapshot

  1. In the Data Protection page, select the Snapshots tab.

  2. Find the snapshot that you want to use. This can be a snapshot of the volume path, the subsystem path, or a path under the subsystem that includes the volume.

  3. Right-click the snapshot and select Map Snapshot Volumes.

    The Map Volumes dialog suggests volumes.

Right-click the snapshot and select Map Snapshot Volumes

  1. In the Map Volumes dialog, select the volume you want to map from the snapshot.

select the volume you want to map

  1. Click Next.

  2. Select the volume path on the left and the host(s) on the right.

  3. Click Save Mapping.

    The mapping task begins, and you can track it on the Activities page.

mapping task begins, and you can track it on the Activities page

If you navigate back to Volume in Element Store, you will see a snapshot of Volume1 being created.

see a snapshot of Volume1 being created

You  can see the Volume1 Snapshot entry via vcli as well. Under Snapshot Data, you would see the Snapshot used to Map to the Volume [ {'id': 2, 'name': 'Dev_2025-08-01_06_40_00_UTC'} ]

vcli: admin> volume list
+----+---------+---------+----------+-----------------+--------------+--------------------------------------+--------------------------------------+--------------+--------------------------------------------------+-------------+----------------------------+-------------------------+
| Id | View-id | Name    | Size(GB) | Tags            | Namespace-id | Nguid                                | Uuid                                 | Capacity(GB) | Snapshot-data                                    | Tenant-name | Mapped-block-hosts-preview | Mapped-block-host-count |
+----+---------+---------+----------+-----------------+--------------+--------------------------------------+--------------------------------------+--------------+--------------------------------------------------+-------------+----------------------------+-------------------------+
| 1  | 85      | Volume1 | 5.000    | {'Dept': 'Dev'} | 1            | 1257a64f-ff95-447f-8c87-26acc6fc700b | 1257a64f-ff95-447f-936c-a7acc6fc700b | 0.003        | {'id': None, 'name': 'N/A'}                      | default     | centos8-var-1 (id=1)       | 1                       |
| 2  | 85      | Volume1 | 5.000    | {'Dept': 'Dev'} | 2            | a004fbab-2d88-486b-8c87-2676c4c38dfa | a004fbab-2d88-486b-a00f-1a76c4c38dfa | 0.003        | {'id': 2, 'name': 'Dev_2025-08-01_06_40_00_UTC'} | default     | centos8-var-1 (id=1)       | 1                       |
+----+---------+---------+----------+-----------------+--------------+--------------------------------------+--------------------------------------+--------------+--------------------------------------------------+-------------+----------------------------+-------------------------+
vcli: admin> volume get_snapshots --id 1
+----+---------+------+----------+------+--------------+-------+------+--------------+---------------+-------------+----------------------------+-------------------------+
| Id | View-id | Name | Size(GB) | Tags | Namespace-id | Nguid | Uuid | Capacity(GB) | Snapshot-data | Tenant-name | Mapped-block-hosts-preview | Mapped-block-host-count |
+----+---------+------+----------+------+--------------+-------+------+--------------+---------------+-------------+----------------------------+-------------------------+
| 2  | N/A     | N/A  | N/A      | N/A  | N/A          | N/A   | N/A  | N/A          | N/A           | N/A         | N/A                        | N/A                     |
| 3  | N/A     | N/A  | N/A      | N/A  | N/A          | N/A   | N/A  | N/A          | N/A           | N/A         | N/A                        | N/A                     |
+----+---------+------+----------+------+--------------+-------+------+--------------+---------------+-------------+----------------------------+-------------------------+
vcli: admin>

Mapping a Volume from a Snapshot after the Snapshot was Deleted

  1. In the Data Protection page, select the Snapshots tab.

  2. Find the snapshot that you want to use. This can be a snapshot of the volume path, the subsystem path, or a path under the subsystem that includes the volume.

Locate the snapshot that you want to use

  1. Right-click the snapshot and select Map Snapshot Volumes.

Map Snapshot Volumes

  1. In the Map Volumes dialog, click Add volumes manually.

Add volumes manually

  1. From the Subsystem dropdown, select the subsystem under which the volume existed before it was deleted.

  2. In the Volume Name field, enter the volume name.

  3. Click Add to table.

select the subsystem under which the volume existed before it was deleted

  1. Select the newly added row in the table.

  2. Click Next.

Select the newly added row in the table

  1. Select the volume path on the left and the host(s) on the right.

  2. Click Save Mapping.

NVMe Block Commands

Command

What it Does

When / Why You Use It

nvme list

Lists all NVMe devices and namespaces

First command to verify NVMe devices are detected by the OS

nvme list -o json

Lists NVMe devices in JSON format

Useful for automation, scripts, or structured debugging

nvme id-ctrl /dev/nvme0

Shows NVMe controller information

Check vendor, firmware, capabilities, queue depth

nvme id-ns /dev/nvme0n1

Shows namespace (block device) details

Verify size, LBA format, sector size

nvme smart-log /dev/nvme0

Displays SMART / health data

Check temperature, wear level, media errors

nvme error-log /dev/nvme0

Shows recent NVMe errors

Debug I/O failures, timeouts, or controller errors

nvme get-log /dev/nvme0 --log-id=2

Retrieves log pages manually

Deep debugging when standard logs aren’t enough

nvme fw-log /dev/nvme0

Shows firmware slot and status

Verify active firmware and available slots

nvme format /dev/nvme0n1

Formats the namespace

Destructive – used for reprovisioning or testing

nvme sanitize /dev/nvme0n1

Secure erase of device

Compliance or complete data wipe

nvme reset /dev/nvme0

Resets the NVMe controller

Recover from hung or unresponsive NVMe device

nvme ns-rescan /dev/nvme0

Rescans namespaces

When namespaces are added/removed dynamically

nvme list-subsys

Shows NVMe subsystems

Debug multipath or NVMe-oF setups

nvme discover

Discovers NVMe-oF targets

Used in NVMe over Fabrics (TCP/RDMA)

nvme connect

Connects to NVMe-oF target

Attach remote NVMe storage

nvme disconnect

Disconnects NVMe-oF target

Clean up fabric connections

nvme top

Real-time NVMe performance view

Live monitoring of I/O activity

nvme version

Shows nvme-cli version

Ensure compatibility with kernel/features

Command

Purpose

lsblk -o NAME,SIZE,TYPE,MOUNTPOINT

View block device hierarchy

lsblk -N -O -J

Full block metadata (great for CSI debugging)

blkid

Identify filesystem / UUID

udevadm info --query=all --name=/dev/nvme0n1

Udev attributes and symlinks

cat /sys/block/nvme0n1/queue/*

Inspect I/O scheduler and queue depth

`dmesg

grep nvme`

Metrics

Predefined Analytics Metrics

image-20250801-060131.png

For Official Documentation on Block Support, refer to: Overview of block storage on VAST Cluster