Enhancements in 5.3.0

Prev Next

VAST Cluster 5.3 enhancement highlights:

Install & Upgrade

  • VAST Cluster 5.3 features a checkpoint mechanism for the cluster deployment procedure. Checkpointing splits the procedure into a series of discrete steps. If one of the steps fails, the process can be resumed from the failed step, without the need to rerun the steps that have already completed.

  • ORION-177163: Added an ability to import and export VAST Cluster Install field values from/to a JSON configuration file. To do so, use the new Export Config and Import Config buttons in the VAST cluster installation utility.

    The following limitation applies:

    • ORION-280966: Having imported a JSON configuration file in VAST Cluster Install, you need to manually verify that all the populated fields have expected values. Sometimes, depending on the cluster configuration and environment, some of the fields are not populated as expected during the import.

  • ORION-235299: Added an ability to determine whether to run automatic upgrade of firmware on non-active drives as part of the cluster upgrade procedure. The following user controls have been added for this purpose to VAST CLI:

    • The --auto-drive-fw-upgrade option on the cluster upgrade command

    • The new cluster set-drive-fw-upgrade command

Cluster Expansion

  • The cluster expansion procedure includes an additional validation step to verify the setup. If you do not want to wait for the validation to complete, you can skip this step by choosing Skip Validation.

Networking

  • ORION-241635: Support of InfiniBand Connected mode has been deprecated.

  • ORION-207871: Added an ability to configure the cluster so that one VAST DNS name can be used for multiple VLANs (virtual IP pools with VLAN tags). If you want clients to be able to connect to any of virtual IPs from any of the VLANs behind a single VAST DNS name, contact VAST Support to turn this feature on.

  • ORION-187391: Added indication of cluster nodes' IP addresses and hostnames to the /etc/hosts file on each node.

  • ORION-195238: The non-disruptive network reconfiguration feature (Settings -> Configure Network) is now available for support and root users only.

Element Store

  • Starting with VAST Cluster 5.3, VMS does not allow deletion of an S3 view or disabling the S3 protocol for a view if there is a non-empty S3 bucket exposed by the view. To overcome this restriction, delete the view via VAST REST API by sending a DELETE request to the /views/<view ID>/ endpoint with the Force option specified.

Multi-Tenancy

  • ORION-236542: VAST Web UI features a new dashboard, the Tenants Dashboard, that lets you monitor performance (capacity, bandwidth and latency) per tenant. To access the new dashboard, choose Dashboard in the left navigational menu and then switch to the Tenants Dashboard tab.

  • You can optionally set a domain name that will be used to build the cluster login URL for the tenant. (If a domain name is not specified, the tenant name is used instead.)

    To specify a tenant domain name:

    • In VAST Web UI, enter the domain name in the Domain field in tenant settings (Element Store -> Tenants -> choose to create or edit a tenant).

    • In VAST CLI, use the --domain-name option on the tenant create and tenant modify command.

    In addition, VAST Web UI lets you preview the resulting login URL in the same dialog.

  • ORION-194717: Increased the maximum allowed number of tenants per cluster from 1000 to 4096.

  • Added predefined analytics to visualize bandwidth, capacity, IOPS and latency per tenant. To access the analytics, select the Tenant category in Analytics -> Analytics -> Predefined Analytics.

  • ORION-238567: If you're deleting a tenant in VAST Web UI (Element Store -> Tenants -> right-click a tenant and select Remove), the confirmation box does not provide an option to force the deletion despite that the tenant directory contains data. The deletion is performed regardless of the presence of tenant data.

  • The --vippool_ids option on the tenant create command has been deprecated.

Encryption of Data at Rest

  • Added support for the following EKM providers:

    • Fortanix Data Security Manager (DSM)

    • HashiCorp Vault Enterprise

  • ORION-197339, ORION-206540: With VAST Cluster 5.3, the Revoke action has been updated to revoke the encryption key or group on the EKM server. To revoke on the VAST cluster only, use the new Deactivate action. A deactivated key can be reinstated on the VAST cluster using the Reinstate action.

    In VAST Web UI, both actions are available in the right-click menu for an encrypted path (Element Store -> Encrypted Paths) or a tenant (Element Store -> Tenants).

    VAST CLI features a new command to deactivate an encryption group, encryptiongroup deactivate-encryption-group, and also a new --deactivate option on the tenant alter-encryption-group-state command.

    Note

    The Revoke operation cannot be undone.

Quotas

  • Extended the list of units of measure that can be selected when setting a quota in VAST Web UI to include terabytes (TB), petabytes (PB), exabytes (EB), zettabytes (ZB), and yottabytes (YB).

  • ORION-157643: The VAST CLI userquota list command features new filters to display group rules, user and group accounting information, as well as default user or group rules.

Quality of Service (QoS)

  • The settings to define minimum QoS have been deprecated.

  • ORION-207047: You can now set QoS static limits for the entire cluster. To do so in VAST CLI, use the --static-limits option on the cluster modify command.

Protocols

  • The S3 security flavor now supports the SMB protocol. This lets you implement policy-based access control for SMB sessions. You can use identity and/or bucket policies to control SMB client access to views associated with a view policy that enforces the S3 Native security flavor.

  • ORION-115966: Added an ability to control the way VAST Cluster sets the owning group when creating files on a view controlled with SMB and Mixed Last Wins  security flavor.

    By default (which is the same as in previous versions), the owning group is determined based on the access protocol: from the user's primaryGroupID for SMB and from the user's POSIX GID for NFS. You can change this behavior to set the owning group based on the POSIX GID of the user for both SMB and NFS. The setting is made per tenant.

    To set the behavior, run the tenant create or tenant modify command with the --preferred-owning-group PROTOCOL_BASED or --preferred-owning-group POSIX_GID option specified.

  • ORION-66805: Added support for applying identity or bucket policies to NFS requests for S3 Bucket views created in a parent NFSv3 view controlled with S3 Native security flavor. In other words, access to object in /nfs_view/bucket_view/object can now be authorized based on S3 policies set for bucket_view.

NFS

  • ORION-212662: Added user controls to determine whether POSIX mode bits are inherited from the parent directory when using a view policy with NFS security flavor:

    • In VAST Web UI, the Inherit ACL from parent option in the Permissions for New Files and Directories tab of view policy settings (Element Store -> View policies -> choose to create or edit a policy).

    • In VAST CLI, the --enable-inherit-parent-mode-bits and --disable-inherit-parent-mode-bits options on the viewpolicy create and viewpolicy modify commands.

  • ORION-179496: Added support for NFS aliases to the VAST implementation of Remote Quota Protocol (rquota).

SMB

  • ORION-223116: Added an ability to configure the way the cluster handles SMB compound requests beginning with a CREATE request when the starting CREATE request gets a STATUS_PENDING response.

    By default, the cluster sends STATUS_PENDING responses to all remaining requests in the compound, which may not be expected by the client.

    You can alter this behavior so that the cluster will first respond with STATUS_PENDING only to the starting CREATE request (skipping the rest of the responses). After the entire compound is executed, responses to all requests in the compound will be sent. To alter the cluster behavior, contact VAST Support.

S3

  • Changed the way the VAST cluster responds to anonymous ListBuckets requests. In version 5.3, the response is HTTP code 200 (OK). Prior to this change, the cluster responded with HTTP code 404 (Not Found).

  • ORION-198542: Added support for S3 conditional writes. If the PutObject or CompleteMultipartUpload request contains the If-None-Match header and the value is '*', the object is uploaded only if there is no existing object with the same key name in the bucket. Conditional writes are not supported for versioned buckets.

  • ORION-191255: Optimized performance when processing S3 PutObject requests for objects with the size of less than 1MiB. The optimization works with S3 Native security flavor only.

  • ORION-187569: Added support for S3 trailing checksums.

    VAST Cluster now recognizes that an S3 request includes a checksum and handles the checksum separately from the uploaded content. Note that VAST Cluster does not perform checksum verification.

  • ORION-159797: Added support for bucket object delimiters other than a forward slash (/).

ABAC

  • ORION-204606: Added support for replication or Global Access where the destination directory has ABAC tags.

  • ORION-204605: Added support for replication and Global Access on a directory that has ABAC tags where the parent directory also has ABAC tags.

Protocol Auditing

  • ORION-222769: Expanded protocol auditing to log the following VAST Database transaction-related operations when protocol auditing is enabled and configured to log session create and close operations:

    • BEGIN_TRANSACTION

    • ROLLBACK_TRANSACTION

    • COMMIT_TRANSACTION

    • START_QUERY

    • QUERY_STATUS

    • FINISH_QUERY

    • LIST_QUERIES

    • GET_DATA

    • FINISH_DATA

Event Publishing

  • ORION-212929: Added an ability to set up event notifications for the events of adding a tag to an object (PutObjectTagging) and removing a tag from an object (DeleteObjectTagging).

    The following user controls have been added for this purpose:

    • In VAST Web UI, the Object Tagging tab in the Trigger pane of view's event notification settings (Element Store -> Views -> choose to create or edit a view -> go to Event Notifications tag)

    • In VAST CLI, the S3_OBJECT_TAGGING_PUT, S3_OBJECT_TAGGING_DELETE and S3_OBJECT_TAGGING_ALL keywords for the --triggers option on the eventnotification create and eventnotification delete command.

VAST Database

  • ORION-215020: Added support for Trino 462.

  • ORION-206485: Added an ability to set user-defined row IDs in VAST Web UI and VAST CLI. Prior to this change, user-defined row IDs could be set through the VAST Connector only.

  • ORION-205367: Added an ability to delete a database from the DataBase -> VAST Database page in VAST Web UI.

  • ORION-194120: VAST Database metrics are now included in Analytics -> Top Actors visualizations.

  • ORION-183605: Added an ability to associate a VAST Database with a non-default tenant. Prior to this change, all VAST databases were associated with the default tenant.

    To supply a tenant when creating or managing databases:

    • In VAST Web UI, use the Tenant selection field in the VAST Database page (DataBase -> VAST Database).

    • In VAST CLI, specify the --tenant-id option on the command.

  • ORION-174344: Added an ability to monitor VAST Database operations in the Analytics -> Data Flow page. To do so, open the page and select Database in the Protocol field.

VAST Data Engine

  • Added support for Spark 3.5.1 with the Spark Thrift server and Spark Connect. You can select the new Spark 3.5.1 with Thrift image tag when adding a managed application in VAST Web UI (Data Engine -> Managed Applications -> choose to create an application).

    If the Spark cluster includes Spark Connect, select the Thrift and Connect checkbox. This checkbox enables support for Spark clusters with TLS encryption. You can upload the SSL certificate and keys using the new Configuration & Security tab in managed application settings (Data Engine -> Managed Applications -> choose to create or edit an application).

  • When running Spark as a managed application, you can upload Spark configuration files (spark-defaults.conf, core-site.xml, hive-site.xml, hdfs-site.xml), SSL certificates and keys in VAST Web UI using the new Configuration & Security tab in managed application settings (Data Engine -> Managed Applications -> choose to create or edit an application). You can also make use of sample configuration files that can be downloaded from this tab.

  • ORION-218020: Added an ability to specify ranges of worker IP addresses when adding a managed application (Data Engine -> Managed Applications -> choose to create an application -> Network tab -> Worker Details).

  • ORION-210269: Added indication of the CNode IP when displaying the CNode state for a running managed application (Data Engine -> Applications -> right-click a running application and choose View CNode State to open the CNode State dialog).

Data Protection

  • ORION-214687: Increased the maximum allowed number of snapshots per protection policy and per protected path from 980 to 1500.

  • ORION-213277: Added an ability to associate a protection policy with a VAST tenant, as follows:

    • In VAST Web UI, use the Tenant and Remote tenant fields in protection policy settings (Data Protection -> Protection Policies -> choose to create a protection policy)

    • In VAST CLI, use the --tenant-id and –remote-tenant-name options on the protectionpolicy create command.

Replication

  • ORION-210450: Added an ability to automatically create configuration for the replicated VAST Database bucket on the destination replication peer:

    • In VAST Web UI, the Bucket DB replication toggle in the cluster's S3 settings (Settings -> S3)

    • In VAST CLI, the --enable-bucket-db-replication option on the cluster modify command

    • In VAST REST API, the enable_bucket_db_replication parameter for the /clusters/{id}/ endpoint

    Once enabled, this capability cannot be disabled.

  • ORION-115311: Added the ability to move files and directories from or to a protected path that is a replication source or replication target. This includes both moving a file or directory from a protected path to a non-protected path and vice versa, and also  moving a file or directory from one protected path to another protected path.

Global Access

  • Added an ability to configure a Global Access protected path and an asynchronous replication remote protected path on the same source path. The configuration requires that all participating clusters run VAST Cluster 5.3 or later and that the destination paths for asynchronous replication and Global Access are be on different replication peers (different clusters).

VAST Dataspace

  • ORION-190960: Enhanced validations run when creating or modifying a replication peer to ensure that the remote cluster has a replication virtual IP pool that is enabled and can be used for replication.

VAST on Cloud

  • You can deploy a VAST on Cloud (VoC) cluster on Google Cloud Platform (GCP).

    VoC deployment in GCP is done through Terraform using a set of Terraform configuration files provided by VAST. For a complete installation procedure, see VAST Cluster Administrator's Guide.

    The following limitation applies:

    • Encryption is not supported with VoC on GCP.

  • ORION-205091: Updated the VoC on GCP cluster removal routines to automatically clean up all static routes created by the cluster.

Authentication & Authorization

  • ORION-175702: Added an ability to set the type of periodic health check that VAST Cluster performs for an Active Directory or LDAP provider configured for the cluster: by pinging the provider or by binding to it.

    The following user controls have been added for this feature:

    • In VAST Web UI, the Ping check and Bind check options in the Periodic health check type pane of advanced provider settings (User Management -> Active Directory or LDAP -> choose to create or edit a provider -> Advanced tab)

    • In VAST CLI, the --monitor-action action on the activedirectory create, activedirectory modify, ldap create and ldap modify commands

    • In VAST REST API, the monitor_action parameter for the activedirectory endpoint.

VMS

  • ORION-231659: VAST Prometheus Exporter lets you export various NIC-related metrics that help monitor physical packet transmission and reception rates. The new metrics can be exported with the /prometheusmetrics/nics and /prometheusmetrics/all endpoints.

  • ORION-225096:  Added an ability to modify the VMS virtual IP, subnet mask and port via VAST CLI. To do so, use the --mgmt-data-vip, --mgmt-data-netmask and --mgmt-data-interface options on the vms modify command.

  • ORION-219419, ORION-208085: Added an ability to set ranges of IP addresses from which users are allowed to log in to the VMS. The settings can be made for the entire cluster and/or per tenant:

    • For the entire cluster:

      • In VAST Web UI, use the new tab named Client Source Address in VMS settings (Settings -> VMS).

      • In VAST CLI, run the cluster modify command with the --access-ip-ranges option specified.

    • Per tenant:

      • In VAST Web UI, use the Source IP Address for Tenant Admin to VMS pane in the VMS Login tab of tenant settings (Element Store -> Tenants -> choose to create or edit a tenant).

      • In VAST CLI, run the tenant create or tenant modify command with the --access-ip-ranges option specified.

  • ORION-216027: Added an indication of the virtual IP pool name to CNode metrics that can be exported with VAST Prometheus Exporter.

  • ORION-196210: Added the DBox state as a metric that can be exported with the VAST Prometheus Exporter.

  • ORION-172950: VMS can monitor the amount of packets pruned or discarded due to TCP socket buffer overruns. The following VMS alerts will be raised when the packet rates get high enough to indicate a potential cable issue:

    • CNode – Hardware,component=node,packets_pruned_socket_buffer_overrun

    • CNode – Hardware,component=node,packets_dropped_socket_buffer_overrun

  • ORION-168916: Fine-tuned the dbox configuration does not match the pci switch type configuration alarm so that it is skipped from alarm listings if the same alarm has already occurred within the day.

VAST Web UI

  • Improved the identity policy visual editor (User Management -> Identity Policies -> choose to create or edit an identity policy) to include:

    • Predefined sets of statements that you can use as building blocks when creating your identity policy from scratch

    • Fields to define conditions under which policy statements take effect.

  • ORION-234678: The right-click menu for a VMS manager entry in the Administrators -> Managers page includes a new option, Clone and Edit, that lets you create a copy of an existing manager account and open it for editing in VAST Web UI.

  • ORION-182749: Added an ability to select and deselect all CNodes when creating or modifying a virtual IP pool (Network Access -> Virtual IP Pools -> choose to create or modify a pool -> Resource Selection tab).

  • ORION-186882: Enhanced the Access Mask column in the File Handles And Byte-Range Locks On A File dialog to provide a user-friendly list of access masks.

  • ORION-110630: Enhanced the Settings -> Certificates page to display certificate expiration dates.

VAST CLI

  • ORION-229365: Added the --tenant-id option that lets you specify the tenant when using the user modify command to update user's S3 permissions.  In addition, the user show command now features the  --tenant-id option to filter the displayed S3 permissions per tenant.

  • ORION-204569: The output of the identitypolicy list and identitypolicy show commands now includes the Enabled field (shows whether the policy is currently in effect) and the Replicated field (indicates whether the policy is replicated from a peer cluster).

  • The tenant create and tenant modify commands now feature a new option, --identity-provider-name, that you can use to specify an authentication provider to authorize access when logging in to VMS.

  • When creating a view in VAST CLI, you can give it a name by specifying the new --name option on the view create command.

VAST REST API

  • ORION-200936: VAST API documentation includes a change log that lists API changes introduced in version 5.3.

Platform & Control

  • ORION-241820: The ebox replace command features a new option, --host-sn, that lets you update the EBox serial number. This option is helpful when troubleshooting EBox discovery issues.

  • ORION-225992: The VAST CLI ebox modify command features a new option, --immediate-phaseout, to power off an EBox without waiting for the drives to finish phasing out.

  • ORION-223443: Root permissions are no longer required when establishing an SSH connection by node name.