VAST Cluster 5.0.0 Release Notes

VAST Cluster 5.0 is released to General Availability as of December 29, 2023. It is a major release with new capabilities.

To access this release, contact the VAST Customer Success team via a support ticket, Slack, or by sending an email to support@vastdata.com.

⚠️ Upgrade to VAST Cluster 5.0 is supported from VAST Cluster 4.7.0 and service packs of 4.7.0. Direct upgrade from pre-4.7 versions is not supported. To upgrade from a pre-4.7 version, begin by upgrading to VAST Cluster 4.7, and then upgrade to VAST Cluster 5.0. Upgrading to VAST Cluster 5.0

For scale-related information, see VAST Cluster Scale Guidelines.

New Features in 5.0.0

Global Synchronization Enables Seamless NFSv3 Failover

VAST Cluster 5.0 features a new capability for NSFv3 clients to continue running applications against a mounted view without remounting the view if the cluster fails over to a replication peer. This capability is supported by synchronizing file handles between views on a replicated path on two or more replication peers.

To use this feature, you create a new view on the replicated path on each peer in a replication group and you enable a setting in each view called global synchronization. With globally synchronized views, FSIDs and file handles remain consistent across clusters in the event of a failover.

Note that you also need to handle networking on the client side in the event of an actual failover so that the network layer represents the newly appointed cluster in a way that replaces the cluster that was active prior to the failover. That is, since the mount point of an NFSv3 client directs traffic to one of the primary cluster's virtual IPs, in the event of a failover to a different cluster, unless you have the same virtual IP defined in a protocols VIP pool on the newly appointed cluster, you need to take action on the client network to redirect traffic to and from the VIPs on the newly appointed cluster.

For feature documentation, see VAST Cluster Administrator's Guide.

Note
Known issue: In rare cases with large numbers of files and directories, the existence of a globally synchronized view under a protected path can block the removal of the protected path.

Multi-Cluster Management

Multi-Cluster Management is a new graphic interface within the VAST Web UI that enables you to easily connect multiple on-premises and VAST on Cloud (VoC) clusters to each other and then configure replication between the clusters with a simplified and quicker flow. For feature documentation, see VAST Cluster Administrator's Guide.

Additionally, a similar cloud-based Multi-Cluster Manager service enables you to deploy and manage VAST on Cloud (VoC) clusters.

In the current release, only the Multi-Cluster Manager cloud service can deploy VoC clusters, while only the VMS-local Multi-Cluster management page provides the ability to connect clusters and configure replication between them. These services and their features are expected to converge in future releases.

The following limitations apply:

The feature requires that each cluster participating in the inter-connection is running VAST Cluster 5.0.
ORION-135966: The inter-connecting clusters must have connectivity to each other through the clusters' management networks.
ORION-132073: When you remove a VoC cluster from a Multi-Cluster Manager cloud service instance (using the removal button on the cluster's card ()), the VoC cluster is terminated. There is no option to remove a VoC cluster from Multi-Cluster Manager without also terminating it. (In the Multi-Cluster Management page in the VAST Web UI the button removes the VoC cluster from Multi-Cluster Management and does not terminate it. )
ORION-137875: In case of Multi-Cluster Manager failure, VoCs provisioned by the instance cannot be connected to a Multi-Cluster Management instance.

VAST on Cloud (VoC) Persistency

VAST on Cloud (VoC) clusters can now be deployed with resiliency. In the event that a resilient VoC cluster is brought down by AWS, the cluster automatically recovers and rebuilds data from persistent cloud storage resources.

VAST Database Enhancements

VAST Cluster 5.0 introduces the following VAST Database features:

A VAST connector for the Trino version 420 query engine. For installation and configuration instructions, see here.
Note
The VAST connector for Trino 420 supports Trino 420 only when obtained directly from VAST, and not when obtained from the Trino site.
Semi-sorted table projections. These are subsets of a full table in the VAST Database that allow for optimized queries that use only the columns included in the projection. You can select up to four columns to be used to sort the projection, and an unlimited number of additional columns to include in the projection, which are not used to sort the projection.

Hardlink Indexing in VAST Catalog

In previous releases, VAST Catalog indexed a single link to each file. All hardlinks to each file are now indexed.

VAST Catalog and Async Replication Now Supported Together

This release supports async replication simultaneously with the VAST Catalog. The following limitations still apply:

Replication is limited to two peers (group replication is not supported with VAST Catalog).
VAST Catalog must be disabled before a protected path can be deleted.
(LIFTED IN 5.0.0-SP4) ORION-145127: VAST Catalog must be disabled before you can perform a failover to the destination peer.

Access-Based Enumeration (ABE)

VAST Cluster 5.0 supports Access-Based Enumeration (ABE) for the SMB storage protocol.

ABE hides files and directories that users do not have permission to access. When a client lists an ABE-enabled view, ABE filters the listing so that it contains only files and directories for which the client's user has generic read permissions.

The following user controls have been added for this feature:

In VAST Web UI, the ABE tab for a view (Element Store -> Views -> choose to create or edit a view), which lets you select the SMB protocol in the Protocols dropdown and optionally limit the directory levels at which ABE is enabled using the Max depth field.
In VAST CLI, the --abe-protocols and --abe-max-depth options on the view create or view modify command.

For more information about this feature, see VAST Cluster Administrator's Guide.

Support for SMB Previous Versions

VAST Cluster 5.0 supports the SMB protocol capability that enables Windows Explorer users to access and restore previous versions of a file or directory. The restore is done from an existing VAST snapshot of the file or directory.

For more information about this feature, see VAST Cluster Administrator's Guide.

The following limitations apply:

(RESOLVED IN 5.2.0) ORION-130460: VAST Cluster does not show any previous versions for a file or directory that has the same name as a file or directory that has been deleted and resides in the same directory as the deleted file or directory.
ORION-134730: An attempt to restore a file can fail if after the restore has started, a quota is set on the path where the file resides.

Support for SMB Server-Side Copy

VAST Cluster 5.0 introduces support for SMB Server-Side Copy.

Typically, an SMB client sends an SMB FSCTL_SRV_COPYCHUNK IOCTL request to initiate a server-side copy of data. VAST Cluster performs the copy locally, without requiring data to traverse the network to the SMB client and back. This improves performance and lessens network load on the SMB client.

No additional configuration is required on the clients or the VAST cluster to enable this feature.

The following limitation applies:

(RESOLVED IN 5.2.0) ORION-137905: If an application saves changes to a file by recreating the file, or when the client otherwise deletes a file or a directory and creates a new one with the same name, no previous versions can be displayed for the file or directory. To restore such a file or directory, you need to restore one of its parent directories.

SMB Share Listing per Tenant

Client listings of SMB shares include only shares that are associated with the same tenant as the SMB client. Previously, SMB shares for all tenants were included in an SMB share listing response.

This feature removes the following limitations:

SMB is supported on multiple tenants as long as they are all using the same AD provider.
SMB share names must be globally unique per cluster, rather than only per tenant.

Support of S3 Access To Views with SMB and Mixed Last Wins Security Flavors

VAST Cluster extends its multi-protocol access capabilities to let you manage S3 access through the SMB or Mixed Last Wins security flavors.

When an S3 client attempts to access a view, permissions are evaluated as follows:

If the view is controlled by the SMB security flavor:
- Only permissions for the requested resource are taken into account.
- Permissions on parent directories are ignored.
- S3 clients are denied the ability to set or modify ACLs.
If the view is controlled by the Mixed Last Wins security flavor:
- Permissions for the resource requested and for all parent directories in the path are taken into account.
- S3 clients are allowed to set and modify ACLs.

Support for S3 Presigned POST Uploads

VAST Cluster extends its support of presigned URLs to include S3 presigned POST uploads. You can now use a presigned URL to temporarily authorize users to make a direct upload to an S3 server using security credentials and other parameters obtained from a non-S3 server.

For more information about this feature, see VAST Cluster Administrator's Guide.

The following limitation applies:

An object to be uploaded via an S3 presigned POST request must have only ASCII characters in its name.
A POST policy can be up to 4800 bytes.

Multi-Forest Authentication

VAST Cluster can authorize client access by querying users and groups from one or more trusted forests, in addition to the forest of the cluster's joined domain. When multi-forest authentication is enabled, VAST Cluster automatically discovers all domains in the forest of the cluster's joined domain, and also all domains in forests that have a two-way transitive trust relationship with the cluster's forest.

VAST Cluster can discover up to 10 trusted forests.

The limitations are as follows:

VAST Cluster does not allow adding two different Active Directory configuration records with the same domain name but different settings for multi-forest authentication and/or auto-discovery.
Names of users' domains are not displayed in data flow analytics.
If a trusted domain becomes unavailable and then recovers, SMB clients can use it to connect to the VAST cluster only after a period of time, but not immediately upon domain recovery.
Clients cannot establish SMB sessions immediately after a trusted domain recovers from a domain failure.
If a group exists on an Active Directory domain in a trusted forest and the group scope is defined as DomainLocal, VAST Cluster does not retrieve such a group when querying Active Directory, so members of such a group are denied access despite any share-level ACLs that can rule otherwise.
If TLS is enabled, the SSL certificate has to be a CA-signed certificate that is valid for all of the domain controllers in all trusted forests. If the certificate is not valid for a domain controller, this domain controller is not recognized.
ORION-156168: In a multi-forest environment, after migrating a group account from the forest of the cluster’s joined domain to another forest, information about historical group membership is not kept, so users in the migrated group might not be able to access resources to which they used to have access prior to the migration.

The following user controls have been added for this feature:

In the VAST Web UI, the Enable trusted domains on other forests slider is in the LDAP configuration for Active Directory.
In VAST CLI, the --enable-multi-forest and --disable-multi-forest options on the ldap modify commands.

For more information about this feature, see VAST Cluster Administrator's Guide.

Using Kerberos/NTLM Authentication To Authorize SMB Users from Non-Trusting Domains

A tenant can be configured to authorize SMB client access by using user and group information supplied in the user's Kerberos or NTLM ticket, rather than by retrieving that data in Active Directory. This option is beneficial in one-way trust environments where the VAST Cluster is not allowed to run LDAP queries against some domains.

The following user controls have been added for this feature:

In VAST Web UI: the Use SMB native authentication option in tenant settings (Element Store -> Tenants -> choose to create or edit a tenant -> the Advanced tab)
In VAST CLI, the --enable-use-smb-native and --disable-use-smb-native options on the tenant create or tenant modify commands.

For more information about this feature, see VAST Cluster Administrator's Guide.

The following limitations apply:

ORION-143944: The DOMAIN\username format cannot be used to specify users of remote domains. The username@domain format must be used instead.
ORION-134299: When the tenant is set to use Kerberos/NTLM authentication to authorize SMB users from non-trusting domains, both NFS and SMB must use the native SMB authentication (Kerberos), and not Unix-style UID/GIDs.

Known issues include:

ORION-145961: If you enable, then disable, and then re-enable the Use SMB native authentication option, a User not found error appears when trying to create a new user quota or a new user QoS policy.

Support for Multiple Active Directory Providers for SMB Access

VAST Cluster 5.0 supports up to eight Active Directory providers that can be used to authenticate SMB users. Previously, only one Active Directory provider was supported for SMB.

QoS Policy Limits for Specific User(s)

VAST Cluster 5.0 lets you create a QoS policy to provision performance for one or more users. Previously, QoS policies could only be applied per view.

The following user controls have been added for this feature:

In VAST Web UI, the User tab in the QoS policy settings (Element Store -> QoS Policies -> choose to create or edit a QoS policy)
In VAST Web UI, the User QoS predefined analytics report (Analytics -> Analytics -> choose User report category)
In VAST CLI:
- The policy-type option on the qospolicy create command
- The --is-default option on the qospolicy create and qos policy modify commands
- The qospolicy attach-user and qospolicy detach-user commands

For more information about the two QoS policy types, see VAST Cluster Administrator's Guide.

The following limitation applies:

ORION-139403, ORION-137998: User QoS is blocked for SMB and S3 protocols.

QoS Burst and Credit Limits

In addition to minimum and maximum QoS limits, a QoS policy can enforce a burst limit and a credit limit.

When a burst limit is set, burst credits are accumulated as long as the workload consumes less resources than set by the maximum limit. The credits can later be spent to gain performance that exceeds the maximum limit, up to the configured burst limit. The amount of credits that can be accumulated is capped by a credit limit.

By default, no burst or credit limit is set.

The following user controls have been added for this feature:

In VAST Web UI, the Burst and Credits fields in the QoS policy settings (Element Store -> QoS Policies -> choose to create or edit a QoS policy)

In VAST CLI, the following keywords for the --static-limits option on the qospolicy create or qospolicy modify command:

burst_reads_bw_mb
burst_reads_iops
burst_reads_loan_iops
burst_reads_loan_mb
burst_writes_bw_mb
burst_writes_iops
burst_writes_loan_iops
burst_writes_loan_mb

For more information about burst and credit limits, see VAST Cluster Administrator's Guide.

Terraform Provider

This release introduces the VAST Terraform Provider, which enables you to automate VAST Cluster resource management with Terraform. The Terraform provider supports automation for views, view policies, quotas, users and groups, protected paths, tenants, global snapshots, VIP pools, DNS services, QoS policies, replication peers, S3 replication peers and authorization providers.

For usage information, see VAST Cluster Administrator's Guide.

Support for Encryption Key Rotation

For data encryption with external management of encryption keys on the Thales Group CipherTrust Data Security Platform, VAST Cluster 5.0 supports the rotation of each encryption group's key encryption key (KEK), used to retrieve the data encryption key for the encryption group. KEK rotation can be done on the EKM or from the VAST Web UI or the VAST CLI.

You can now also manually rotate the master key that VAST Cluster generates per cluster. The cluster uses the master key to encrypt the data encryption keys when they are retrieved from the EKM and distributed from the cluster node that hosts the encryption service client to other nodes in the cluster. The master key should only be rotated from the cluster and not directly on the EKM.

For more information about encryption key rotation, see VAST Cluster Administrator's Guide.

Dynamic Enablement and Disablement of Similarity-Based Data Reduction

In 5.0, if similarity-based data reduction is enabled, it is subject to a dynamic mechanism that automatically disables and re-enables the feature based on the amount of data reduction benefit gained in recent writes. This enhancement mitigates the possible read performance degradation that can be present with similarity.

FIPS 140-3 Compliance

VAST Cluster encryption of data at rest is FIPS 140-3 compatible.

VAST Easy Install Support for Multi-Homed CNodes

VAST Easy Install now supports the installation of clusters where CNodes are connected simultaneously to multiple separate client data networks. To support these deployments, you can now set the external network type to a mixed configuration where any given CNode can have differentiated network modes for ports on the same externally connected NIC.

Note
If you set External Network Type for a given CBox to MIX, which enables you to set the External Network Type for each CNode differently, then there are two options for splitting CNode external NICs between Ethernet and IB networks:
IB ETH which supports connection of the left port to an external InfiniBand network and the right port to an external Ethernet network.
ETH IB which supports connection of the left port to an external Ethernet network and the right port to an external InfiniBand network.
With the ETH IB option, the IB port supports either HDR or EDR cable speed. With the IB ETH setting, the IB port is limited to EDR cable speed.

Enhancements in 5.0.0

Networking

ORION-136388: The script used to configure cluster networking (configure_network.py) now features a new keyword, outband_dedicated_ipmi that can be specified on the --auto-ports-ext-iface option to configure the management interface.

Quotas

ORION-112605: Added the following user controls to reset the quota grace period:
- In VAST Web UI, the Reset Grace Period option in the Actions menu for a quota displayed in the Quotas page (Element Store -> Quotas)
- In VAST CLI, the quota reset-grace-period command
- In VAST REST API, the /quotas/{id}/reset_grace_period/ endpoint

Lifecycle Policies

In lifecycle policies, when setting a number of days after which to expire objects, you can now choose which type of timestamp is used to define the start for counting the days until an object expires. The options are creation time, access time and modification time.

Protocols

ORION-132502: Extended support of S3 identity policy permissions when validating access to views that have both NFSv4.1/NFSv3 and S3 protocols enabled, as follows:
- Added mapping of S3 identity policy permissions to NFSv4.1 RPCs:
  - LINK is mapped to S3 PutObject.
  - LOOKUP and LOOKUPP are mapped to S3 HeadObject.
  - READLINK is mapped to S3 GetObject.
- Updated mapping of NFSv3 RPC READLINK to use S3 GetObject. (Previously, it was mapped to HeadObject.)

S3

ORION-122477: Added indication of the S3 bucket name to the text of the error that is displayed when trying to create a bucket with the same name through two endpoints. (Bucket names must be unique across all existing bucket names on the cluster.)
ORION-116787: Extended S3 special character support, which allows use of S3 object names containing character combinations that are not compatible with other access protocols, to include the following new name patterns (in addition to names containing // or /../):
- Names starting with ./ or ../
- Names containing /./
- Names ending with /. or /..

Authentication & Authorization

Added support for Simple Authentication and Security Layer (SASL) authentication. To set SASL as the authentication method that VAST Cluster would use to be allowed to perform queries on a LDAP server:
- In VAST Web UI, go to User Management -> LDAP or Active Directory and choose to create or edit a LDAP configuration or to create an Active Directory configuration. In the dialog that opens, go to the Advanced settings tab and select SASL for Authentication method.
- In VAST CLI, specify --method sasl on the ldap create or ldap modify command.
Added the following controls to allow or prohibit SMB access per Active Directory provider:
- In VAST Web UI, the SMB Allowed option in Active Directory provider settings (User Management -> Active Directory -> choose to create an Active Directory configuration record).
- In VAST CLI, the --allow-smb and --disallow-smb options on the activedirectory create or activedirectory modify command.
Added the following controls to allow or prohibit use of NTLM authentication per Active Directory provider:
- In VAST Web UI, the Enable NTLM option in Active Directory provider settings (User Management -> Active Directory -> choose to create an Active Directory configuration record).
- In VAST CLI, the --enable-ntlm and --disable-ntlm options on the activedirectory create or activedirectory modify command.
When querying a user, -1 is now returned for fields where an empty string was retrieved from the provider.

Data Protection

ORION-129804: Snapshots that async replication creates on the remote replication peer now include the snapshot prefix in their name. This snapshot prefix is defined in the protection policy. Prior to this change, the prefix was used in names of snapshots at the replication source only.

VAST Web UI Usability

A new table layout with a new style of column sorting and filtering and a right-click actions menu.
The new table layout provides the ability to select multiple resources in a table and then perform bulk operations. Select the resources in the new checkbox column and then find supported actions in the now activated Actions menu or the right-click menu:
Interconnected flows to simplify your experience when you create or modify resources that are associated with policies such as view policies, protection policies, identity policies or qos policies. In addition to the ability to select an existing policy from a dropdown, you can now choose to add a new policy:
When you choose this option, the dialog for creating a new policy opens without the current dialog closing. You can add a new policy and close the policy dialog. The new policy is selected in the policy field in the original dialog. You can then continue with the original configuration.
Hyperlinks enabling easy linkage between pages:
Simplified configuration dialogs for Active Directory and LDAP creation, now split into tabs to differentiate between basic and more specific advanced parameters.
ORION-18710: A new toggle switch on the Dashboard provides a choice between MB/s and GB/s for the unit of measurement used to display the Dashboard metrics.
On the Analytics page:
- ORION-79758: The time resolution is now automatically selected according to the time frame (Defined Time) selection.
- We added the Get Latest Data button, which enables you to refresh the display to show the latest data.
The Licenses page in the VAST Web UI now displays a chart that shows which licenses are active per time frame and their cumulative capacity. You can easily adjust the time frame and you can see the current licensed capacity at a glance.
The Create Policy and Update Policy dialogs that are used to configure view policies, now feature new protocol-specific tabs so that options that apply to particular access protocols are grouped together in the same tab.
For resources such as view policies or lifecycle rules, the resource's ID column can now be displayed in the table.
ORION-140836: Added a visual editor that can be used to write an S3 identity policy. To access the visual editor, in VAST Web UI choose User Management -> Identity Policies, click Create Policy and in the Add Policy dialog that opens, click Visual Policy Editor.
ORION-113654: Improved visualization of a failed component in the Hardware page by encircling it with a thick red line.
ORION-55648: Added indication of the drive slot numbers to pages showing the hardware layout so that you do not have to click on the slot to be able to see its number.

VAST CLI

The cluster list-open-protocol-handles command now features a new option named --show-nfsv4.1-only-handles that filters the output to include only handles that can be used by NFSv4.1 clients.
The --s3-object-ownership-rule option on the view create and view modify commands has been replaced with two separate options:
- --disable-acls to set the S3 Object Ownership rule of Bucket Owner Enforced
- --enable-acls to disable S3 Object Ownership for the bucket.

Platform & Control

ORION-106721: Similarity-based data reduction is now enabled only when it provides a certain gain in terms of capacity utilization, as compared to local compression. VAST Cluster is constantly re-evaluating the gain to enable or disable similarity-based-data reduction as needed.

Call Home & Support

ORION-118585: Redesigned the Support -> License page to provide a graph of licensed capacity utilization.

Resolved Issues in 5.0.0

Element Store

ORION-147585: Enhanced internal metadata processing routines to eliminate an issue that could cause CNode containers to restart with the timeout expired for life_type=16,life_gen=773916 (TRAVIS) error.
ORION-131188: Enhanced the internal algorithm used for migration of metadata so that deleting a very large number of objects does not cause the cluster to run in SCARCE mode with the amount of available stripes less than the amount of space available.
ORION-130067: Resolved an issue where after enabling VAST Catalog, the cluster encountered an increase in metadata usage up to the metadata SCARCE state with an ContentDefragMetrics,token_mapper_fullness error raised.
ORION-98531: Eliminated deficiencies in snapshot deletion backlog processing to prevent latency spikes during snapshot deletion.
ORION-87167: Resolved an issue that caused multiple CNode restarts due to a stuck holding generation error.