PROTOCOLS
MULTI-TENANCY
QUALITY OF SERVICE
VAST DATA ENGINE
VAST DATABASE
GLOBAL ACCESS
AUTHENTICATION & AUTHORIZATION
VMS
PLATFORM & CONTROL
VAST DRIVERS
Block Storage Support
VAST Cluster 5.3.0 adds support for block storage devices based on NVMe/TCP.
To expose a block device, create a new Element Store view and enable the new Block access protocol for it. The view acts as an NVMe subsystem. You can create volumes (NVMe namespaces) on that view and associate (map) them with hosts that you define.
The following user controls have been added for this feature:
In VAST Web UI:
Option to enable the Block access protocol in the Protocols field of view settings (Element Store -> Views -> choose to create a view -> General tab)
The new Block tab in view settings to manage block protocol configuration
New grid pages to view and manage volumes (Element Store -> Volumes) and hosts (Element Store -> Hosts)
In VAST CLI:
A new BLOCK value for the
--protocolsoption on theview createcommandVolume management commands:
volume create,volume modify,volume delete,volume list,volume show,volume get_snapshots,volume fetch-capacityHost management commands:
blockhost create,blockhost modify,blockhost delete,blockhost list,blockhost showCommands to associate block volumes with host:
blockmapping show,blockmapping list,blockmapping map_host_to_volumes,blockmapping unmap_host_volumes,blockmapping map_volume_to_hosts,blockmapping unmap_volume_hosts,blockmapping map_volume_path,blockmapping unmap_volume_pathThe
--set-is-default-subsystemand--reset-is-default-subsystemoptions on theview createandview modifycommandsThe
--nqnfilter on theview listcommand to list block-enabled views
The following requirements and limitations apply:
For Rocky Linux-based clients, VAST recommends that the client uses Rocky Linux 9.4 or later.
If a host defined on the VAST cluster does not have any volumes mapped to it, NVMe auto-discovery does not show this subsystem.
A view that is used to expose block storage cannot have other storage protocols enabled.
You cannot enable or disable block storage support on an existing view. Block storage support can only be enabled for a view during view creation and cannot be disabled afterwards.
Block devices can be created on empty directories only.
Nesting of a block view inside an existing block view is not allowed.
The host NQN cannot be modified. To change the NQN, you need to remove the host and then add and map it anew.
When using the VAST Web UI or CLI options to bulk create volumes or hosts, the number of items to be created cannot exceed 256. When mapping hosts to volumes, up to 256 items can be mapped at a time.
The following VAST capabilities are not available with block views:
Access control features (such as ABE, ABAC, WORM)
VAST Audit Log
Replication to a remote peer
Global Access
Remote global snapshot clones
Snapshots on local protected paths are allowed but replication on non-local protected paths is not supported.
An attempt to remove a volume that has snapshots may cause errors for volume objects of snapshots of that volume, if they exist.
The maximum IO block size is limited to 1MB (4GB for unmap).
Known issues include:
ORION-245989: Bulk operations on volumes performed by a cluster admin, cannot be tracked by a tenant admin in the Activities page of VAST Web UI.
ORION-237444: An attempt to create a block view with the same name as a previously deleted block view fails with a
CreateDirResultCode.ALREADY_EXISTSerror.
For more information about this feature, see VAST Cluster Administrator's Guide.
S3 Indestructible Object Mode
Indestructible Object Mode protects objects in an S3 bucket from being altered (including any change in the object's metadata) or removed during a specified period of time.
After this capability has been enabled for the cluster, Indestructible Object Mode can be enabled per view. To disable indestructible object configuration or modify its effective period, the cluster needs to be unlocked.
The following user controls have been added for this feature:
In VAST CLI, the the
--enable-indestructible-objectand--indestructible-object-durationoptions on thecluster modifycommandIn VAST Web UI, a new pane in view settings, Indestructible Object Mode, where you can enable the mode for the view and set the mode effective period (Element Store -> Views -> choose to create or edit a view)
In VAST CLI, the
--enable-indestructible-objectand--indestructible-object-durationoptions on theview createand/orview modifycommands.
The following limitations apply:
An S3 Bucket view with Indestructible Object Mode cannot have other protocols enabled.
Indestructible Object Mode cannot be set for a view that points to / (root directory).
It is not allowed to have views under the view in Indestructible Object Mode, or at the same path as the Indestructible Object Mode view.
Indestructible Object Mode cannot be used together with S3 Object Locking or S3 Object Versioning.
Indestructible Object Mode cannot be set for a view that exposes the protocol audit log directory.
Views in Indestructible Object Mode are not subject to replication or Global Access.
For more information about this feature, see the VAST Cluster Administrator's Guide.
Storage Administration by VAST Tenants
VAST Cluster 5.3 offers a new level of multi-tenancy support. Each VAST tenant is provided with an ability to monitor and manage their storage resources through the cluster's VMS. The tenant admin capabilities include:
Monitor tenant storage using VAST capacity reports, Top Actors analytics, and Data Flow visualizations
Manage users that belong to the tenant and monitor their activities
Manage protected path and protection policies
Set up and run replication and Global Access (based on cluster peer configuration created by the cluster admin)
Manage the tenant's VAST Databases
Manage tenant's VAST Catalog and VAST Protocol Audit logs
Note
Managing the cluster hardware is not included in tenant admin capabilities.
To access the new self-manage capabilities, the tenant admin needs to log in to the VMS using their own credentials, which can be set up by the cluster admin.
The cluster admin manages the cluster as a whole, including the following:
Create and manage cluster tenants
Set capacity and performance quotas for the tenants
Configure cluster networking (virtual IP pools, DNS)
Set up replication peers for the cluster and map tenants between clusters
The following user controls have been added for the cluster admin to manage cluster and tenant admins and their permissions:
To designate Active Directory or LDAP user groups as groups of cluster admins:
In VAST Web UI, the Cluster admin groups option in the provider settings (User Management -> Active Directory or LDAP -> choose to create or edit a provider -> Advanced tab)
In VAST CLI, the
--super-admin-groupsoption on theactivedirectory create,activedirectory modify,ldap create,ldap modifycommands
To designate an Active Directory or LDAP user group as a group of tenant admins for a specific tenant:
In VAST Web UI, the Admins group field in tenant settings (Element Store -> Tenants -> choose to create or edit a tenant -> VMS Login tab)
In VAST CLI, the
--tenant-admins-group-nameoption on thetenant createandtenant modifycommands.
To designate a VMS user as a cluster or tenant admin:
In VAST Web UI, the Cluster Admin and Tenant Admin options in manager settings (Administrators -> Managers -> choose to create a manager)
In VAST CLI, the
--user-typeoption on themanager createandmanager deletecommand.
To associate a role with an admin type and a tenant:
In VAST Web UI, the Cluster Admin and Tenant Admin options and the User Type field in role settings (Administrators -> Roles -> choose to create a role)
In VAST CLI, the
--tenant-idoption on therole createcommand.
To associate a realm with a tenant:
In VAST Web UI, the Tenant field in realm settings (Administrators -> Realms -> choose to create a realm)
In VAST CLI, the
--tenant-idoption on therealm createcommand.
To set Tenant Privacy mode, in which cluster admins cannot view details (views, users, data protection configurations, and so on) about any tenants, except for the default tenant:
In VAST Web UI, the Enable tenant privacy option in VMS settings (Settings -> VMS -> General -> Tenant Privacy pane).
Tenant Capacity Limits
You can limit storage capacity available for a tenant by specifying soft and hard capacity limits, expressed in capacity units of measure such as GB or TB, and also by setting a limit on the number or files and directories that can be created for this tenant.
The following user controls have been added for this feature:
In VAST Web UI, capacity limits can be set in the Capacity rules pane of the new Tenant Limits tab in tenant settings (Element Store -> Tenants -> choose to create or edit a tenant).
In VAST CLI, the
--capacity-rulesoption on thetenant createandtenant modifycommands.
Collection of Tenant Client Metrics
VAST Cluster can collect statistics on tenant’s NFS client operations based on user-defined metadata and present the collected data for analysis. The information is collected by the VAST NFS Collector that is deployed on the client. Collected data is stored in a VAST Database table and can be analyzed as a graph in VAST Web UI or by querying the table directly.
This feature can be enabled or disabled per tenant.
The following user controls have been added for this feature:
In VAST Web UI, the new Client Metrics page (Analytics -> Client Metrics) where you can enable the feature for one or more tenants, set up the metrics to be collected, and view the metrics graph(s).
In VAST CLI, the
tenant show-client-metricsandtenant update-client-metricscommands
The following limitations apply:
NFSv3 and NFSv4 are the only access protocols supported.
For more information about this feature, see VAST Cluster Administrator's Guide.
Workload Prioritization
VAST Cluster provides options that help prioritize workloads and eliminate unexpected performance degradation among views controlled with QoS policies:
You can define a cluster-wide maximum write bandwidth to help prevent situations where workloads do not achieve the expected QoS because of extensive media consumption by other workloads.
The recommended cluster-wide maximum is 70% of the cluster’s total write bandwidth.
To define a cluster-wide maximum write bandwidth:
In VAST Web UI, go to the Global Write BW Limit pane in cluster settings (Settings -> Cluster -> General Cluster Setup and Actions tab). Select Set Manually and enter the write bandwidth limit in the provided fields.
In VAST CLI, run the
cluster modifycommand with the--max-cluster-write-bw-mboption specified.
You can set the prioritization flag for a view QoS policy in order to prioritize the workloads in contention for both CPU and memory resources beyond the limits defined for the tenant and/or for the cluster.
To set the prioritization flag for a QoS policy:
In VAST Web UI, enable the Prioritize policy over cluster or tenant limitations option in QoS policy settings (Element Store -> QoS Policies -> choose to create or edit a QoS policy -> View tab).
In VAST CLI, run the
qospolicy createorqospolicy modifycommand with the--is-goldoption specified.
Tip
Since prioritized workloads are not restricted by the tenant or cluster-wide limits, it is recommended to set a maximum limit for such workloads within the QoS policy.
The following limitations apply:
The prioritization flag is supported for view QoS policies. It cannot be set for user QoS policies.
S3 (including Kafka and VAST Database) and block storage I/Os are not calculated as part of the cluster-wide maximum write bandwidth limit.
Some high-priority optimizations are applied to NFSv3 only.
When the cluster-wide maximum write bandwidth is set, the actual performance may be ±15% of the expected performance.
Use of QoS with RDMA is not supported.
For more information about QoS and workload prioritization, see VAST Cluster Administrator's Guide.
VAST Event Broker
In addition to publishing events to third-party event brokers such as Apache Kafka, VAST Cluster now features its own broker implementation, the VAST Event Broker.
VAST Event Broker is based on the VAST Database, which allows for querying Kafka topics via various database APIs.
The VAST implementation of the Kafka protocol supports a basic subset of the Kafka APIs to allow clients to publish and consume events from the VAST Event Broker:
Producer API
Consumer API
Consumer groups
Database queries on topics
Admin API (create topics, delete topics and consumer groups)
You set up Kafka per view. Each Kafka-enabled view has to have one virtual IP pool assigned to it. The virtual IP pool must have enough virtual IPs so that there is at least one virtual IP per CNode.
The following user controls have been added for this feature:
In VAST Web UI, in view settings (Element Store -> Views -> choose to create a view):
The Kafka protocol can now be selected for a view (General tab -> Protocols field).
The new Kafka tab lets you assign a virtual IP pool to the view.
In VAST CLI:
The
KAFKAkeyword for the--protocolsoption on theview createcommandThe
--kafka-vip-poolsoption on theview createcommandNew commands to manage event topics:
topic create,topic modify,topic delete,topic list,topic show.
The following limitations apply:
Producer API:
Messages are limited to 1MB.
In the event record, the key is limited to 126KB and the value is limited to 126KB.
Access to topics by UUID is not supported.
Idempotent producing is not supported.
Automatic creation of topics is not supported.
Consumer API:
No more than 256 consumer groups per view (broker)
The following is not supported:
Consumer group stickiness parameters (such as
group.instance.id)READ UNCOMMITTED isolation level
Cooperative rebalancing
Client rack awareness
Fetch sessions (only full fetch will be applied), delayed fetch parameters
Seek by time
Admin API:
Supported APIs include the APIs to create topics, delete topics, and to delete groups.
Only the following topic parameters are supported:
Number of partitions
Topic retention period
The following Kafka capabilities are not supported:
Over-the-wire compression of messages
Tip
VAST compression of data is supported.
(RESOLVED IN 5.4.0) Topic compaction
Automatic creation of topics
(RESOLVED IN 5.4.0) Authentication and authorization
(RESOLVED IN 5.4.0) SSL
Transactions
Only one virtual IP pool can be associated with a Kafka-enabled view, providing at least one virtual IP per CNode. Once the view has been created, the virtual IP pool cannot be replaced by another one (but it can be modified if needed).
A topic can have up to 1000 partitions. The number of partitions in a topic cannot be changed after the topic has been created.
Event queries based on the topic partition are not supported.
When listing consumer groups, the response is limited to 256 groups per Kafka-enabled view.
VAST replication of consumer groups is not supported.
Event publishing and consuming operations, as well as topic management operations are not subject to VAST Protocol Auditing or Quality of Service (QoS).
For more information about VAST Event Broker, see VAST Cluster Administrator's Guide.
Support for Database Views
VAST Cluster 5.3 supports creation and management of database views (stored results of queries that present part of the database and can be queried). Clients connecting to a VAST Database can now create database views, list views in the database, get details of a view, redefine a view (Spark only), query a view, and rename a view. Permissions for database view operations are managed through identity policies.
The following limitations apply:
View properties are not supported.
Queries to a view must include full table names.
Redefining a view is supported for Spark clients only.
User-defined column names and comments are lost if the schema of the query changes when redefining a view.
Nested data types are not supported.
For more information about this feature, see VAST Cluster Administrator's Guide.
Global Access for S3 Buckets
VAST Cluster 5.3 adds support for the S3 protocol in Global Access, subject to the following rules and limitations:
Identity policies must be enabled at the cluster to which they get replicated.
The following VAST capabilities are not supported on destination buckets:
S3 event notifications
S3 Indestructible Object Mode
Lifecycle policies
Write Once Ready Many (WORM)
Bucket logging is only supported if both the source and destination buckets are in the same protected path.
Bucket replication between two clusters is only supported when the bucket is associated with the default S3 view policy.
S3 endpoints are not replicated.
Note
Before configuring Global Access for S3, it is recommended to enable bucket replication on both origin and satellite clusters (Settings -> S3 -> enable Bucket Replication). Note that once enabled, bucket replication cannot be disabled.
For more information about this feature, see VAST Cluster Administrator's Guide.
Multiple Local Authentication Providers
Local providers let you manage users and groups on the VAST cluster without the need to connect an external authentication and authorization provider, such as Active Directory or LDAP. VAST Cluster 5.3 offers an ability to create multiple local providers and associate each provider with one or more tenants.
During upgrade to VAST Cluster 5.3, a default local provider is automatically created to include all local users and groups that existed prior to the upgrade. All new local users and groups must be associated with a local provider when they are created. When a new tenant is created, VAST Cluster automatically creates a local provider associated with it.
To manage local providers:
In VAST Web UI, choose User Management -> Local Providers to open the Local Providers page where you can view, create and modify local providers, and also view users and groups associated with a provider.
In VAST CLI, use the
localprovider create,localprovider modify,localprovider delete,localprovider listandlocalprovider showcommands.
To associate a local provider with a tenant, select Vast Provider in tenant settings in VAST Web UI (Element Store -> Tenants -> choose to create or edit a tenant -> Providers tab), or run the VAST CLI tenant create or tenant modify command with the --local-provider-id option specified.
To associate a user or group with a local provider, select a provider in the new Local provider field in user or group settings in VAST Web UI (User Management -> Users or Groups -> choose to create a user or a group), or run the user create , user modify, group create or group modify command with the --local-provider-id option specified.
To copy user accounts from one local provider to another, use the VAST CLI user copy command or the VAST REST API endpoint /users/copy/.
For more information about this feature, see VAST Cluster Administrator's Guide.
User Impersonation
User impersonation lets you handle client users' requests against an NFS export or SMB share using with a preconfigured impersonator user account. When a client user creates or accesses a file or directory stored on the VAST cluster, the operation is handled as though it is performed by the impersonator.
The following user controls have been added for this feature:
In VAST Web UI, the User Impersonation tab in view settings (Element Store -> Views -> choose to create or modify a view)
In VAST CLI, the following options on the
view createandview modifycommands:--enable-user-impersonationand--disable-user-impersonation--user-impersonation-username,--user-impersonation-identifierand--user-impersonation-identifier-type
The following limitation applies:
ORION-216379: When VAST protocol auditing is enabled on a user-impersonated view, only UID of the original user is included in the log. The user's login name and SID are not included.
For more information about this feature, see VAST Cluster Administrator's Guide.
VMS API Token-Based Authentication
VAST Cluster 5.3 supports token-based authentication for VMS manager users accessing the VMS API. With token-based authentication, requests are authenticated with a unique token included in the HTTP Authorization header as Api-Token.
VMS provides an ability to create, modify and revoke authentication tokens, as well as to display effective and archived tokens. Token-related activities can be tracked using VAST Audit logs.
API tokens can be managed using the following VAST CLI commands: apitoken create, apitoken modify, apitoken revoke, apitoken list, apitoken show. To set a maximum number of tokens per VMS manager user, use the vms set_max_api_tokens_per_user VAST CLI command.
The following limitation applies:
ORION-212118: If a wrong VMS authentication token is passed, the cluster responds with
403 FORBIDDENbut not with401 UNAUTHORIZED.
For more information about this feature, see VAST Cluster Administrator's Guide.
Conversion to Write Buffer RAID
For clusters upgrading from a pre-5.1 release, VAST Cluster 5.3 offers an option to convert DBox write buffers from mirrored layout (an older layout used on pre-5.1 clusters) to RAID-6, which provides increased write performance and storage efficiency.
The conversion is a one-time activity that does not interfere with normal cluster operation. Once started, the conversion cannot be stopped and is irreversible. It may take several hours, depending on the number of DBoxes.
Before starting the process, ensure that the cluster is not busy with any rewrite operations (such as encryption or similarity-based reduction), cluster expansion, DBox migration or replacement activities, or cluster upgrades.
The following user controls have been added for this feature:
In VAST Web UI, the Activate Write Buffer RAID option in cluster settings (Settings -> Cluster -> Data Management)
In VAST CLI, the
--enable-write-buffer-raidand--disable-write-buffer-raidoptions on thecluster createandcluster modifycommandsIn VAST CLI, the
--force-wbr-rewriteflag on thecluster createcommand to forcefully run the rewrite process even when there are failed devices.
The following limitations apply:
Conversion to Write Buffer RAID can only be enabled on clusters that were deployed with VAST Cluster 4.7 or later.
This capability is not supported for clusters with TLC drives, and also for VAST on Cloud clusters.
The cluster must include the following minimum number of DBoxes:
DBox Type
DBox HA enabled
DBox HA disabled
Ceres
15
4
Mavericks
22
4
For more information about this feature, see VAST Cluster Administrator's Guide.
New VAST Drivers to Enable Block Storage Access
VAST Block CSI Driver
When used with VAST Cluster 5.3 or later, VAST CSI Driver lets you dynamically provision block storage for your Kubernetes pods. This functionality is referred to as the VAST Block CSI Driver.
To let your applications use block storage on a VAST cluster, you define a block storage class and reference it in the PVC. Based on the storage class definition, the driver provisions a block volume for the PVC using a preconfigured view on the VAST cluster.
For deployment and usage guidelines, see the VAST Block CSI Driver Administrator's Guide.
VAST Driver for Cinder
With this release, VAST provides an implementation of the VAST driver for the OpenStack Block Storage service (Cinder). VAST Driver for Cinder enables you to use VAST NVMe storage as a backend for data volumes managed through Cinder.
For deployment and usage guidelines, see VAST Driver for Cinder documentation.