VAST Cluster 5.0.0-SP40 Release Notes

Enhancements in 5.0.0-SP40

Resolved Issues in 5.0.0-SP40

Limitations in 5.0.0-SP40

Known Issues in 5.0.0-SP40

ORION-175831: Added an ability to disable SMB leases (client cache) per lease type: read leases, write leases, handle leases.

You can disable SMB leases in view policy settings. The following user controls have been added for this feature:

ORION-175251: Updated the cluster’s networking configuration script (configure_network.py) to use port enp211s0f0 as the B2B IPMI port when creating a B2B IPMI configuration on a cluster with MLK DNodes.

ORION-153041: Resolved an issue that could cause the cluster’s networking configuration script (configure_network.py) to fail with the Command 'ipmitool lan set 1 ipaddr 0.0.0.0' returned non-zero exit status 1 error if run with the IPMI gateway parameter specified.

ORION-196209: Resolved an issue where the cluster encountered a reaching halt_writes splittability issue alert accompanied by very high write latency.

ORION-189280: Made updates to prevent deletion of VAST Database elements even if the S3 lifecycle rules configured on the cluster stipulate the deletion.

ORION-153672: Extended denylisting functionality to support denylisting a single replication stream while keeping all other replication streams operational. Prior to this enhancement, if a stream encountered issues, all replication streams were added to the deny list.

ORION-193898: Updated the CSV file export functionality so that no Query: Please correct the form error occurs when trying to export VAST Catalog query results to a CSV file.

ORION-194719: Updated the logic used to select tenants when creating a virtual IP pool in VAST Web UI so that the All tenants option creates a virtual IP pool for all tenants but not for the default tenant only.

ORION-188310: Resolved an issue that prevented export of Switch Active metrics by VAST Prometheus Exporter endpoints.

ORION-172038: Resolved an issue that could be the cause for false flow_control is False NIC alerts when using VXLAN.

ORION-188866: Resolved an issue that could cause multiple CNode containers to restart with the Child chunk not found in composite for handle error after finishing the upgrade of the last CNode during an NDU process.

ORION-208873: Quotas and quota accounting are not supported on subpaths of a replicated protected path on the destination peer. For example, if a protected path is replicated to a destination directory /dest-dir, you cannot set a quota on /dest-dir/mydir.

(RESOLVED IN 5.3.0) ORION-179496: NFS aliases are not supported with VAST Cluster's implementation of Remote Quota Protocol (rquota).

ORION-148295: QoS should be enabled on all views to avoid performance degradation issues.

ORION-148206: There may be some scenarios in which minimum service levels set by QoS policies are not met. 

ORION-139524: Setting a minimum limit for read operations does not limit write operations on the same view.

QoS provisioning is not supported for S3 clients.

User QoS feature is supported for NFS clients only.

ORION-115336: If one creates an NFSv4.1-only view and mounts it, and then creates its parent view with NFSv3 only, IO operations on the NFSv4.1-only view succeed, but mounts are not allowed.

In rare cases with large numbers of files and directories, the existence of a view with Global Synchronization enabled under a protected path can block the removal of the protected path.

ORION-160323: After updating permissions for an SMB share in Windows Explorer, a duplicate SMB share can be displayed. The duplicate SMB share disappears upon a refresh (F5).

(RESOLVED IN 5.2.0) ORION-130460: VAST Cluster does not show any previous versions for a file or directory that has the same name as a file or directory that has been deleted and resides in the same directory as the deleted file or directory.

ORION-134730: An attempt to restore a file can fail if after the restore has started, a quota is set on the path where the file resides.

(RESOLVED IN 5.2.0) ORION-137905: If an application saves changes to a file by recreating the file, or when the client otherwise deletes a file or a directory and creates a new one with the same name, no previous versions can be displayed for the file or directory. To restore such a file or directory, you need to restore one of its parent directories.

An object to be uploaded via an S3 presigned POST request must have only ASCII characters in its name.

A POST policy (used for S3 presigned POST requests) can be up to 4800 bytes.

The maximum path length supported by VAST Catalog is 1024 characters.

When VAST Catalog is enabled, replication is limited to two peers (group replication is not supported with VAST Catalog). 

VAST Catalog must be disabled before a protected path can be deleted. 

The Multi-Cluster Management feature requires that each cluster participating in the inter-connection is running VAST Cluster 5.0.

ORION-135966: The inter-connecting clusters must have connectivity to each other through the clusters' management networks.  

ORION-132073: When you remove a VoC cluster from a Multi-Cluster Manager cloud service instance (using the removal button on the cluster's card ()), the VoC cluster is terminated. There is no option to remove a VoC cluster from Multi-Cluster Manager without also terminating it. (In the Multi-Cluster Management page in the VAST Web UI the button removes the VoC cluster from Multi-Cluster Management and does not terminate it. )

ORION-137875: In case of Multi-Cluster Manager failure, VoCs provisioned by the instance cannot be connected to a Multi-Cluster Management instance.

ORION-143944: When using Kerberos/NTLM Authentication to authorize SMB users from non-trusting domains, the DOMAIN\username format cannot be used to specify users of remote domains. The username@domain format must be used instead.

ORION-134299: When the tenant is set to use Kerberos/NTLM authentication to authorize SMB users from non-trusting domains, both NFS and SMB must use the native SMB authentication (Kerberos), and not Unix-style UID/GIDs.

ORION-141763: Before enabling or disabling NTLM authentication, you need to leave the cluster's joined Active Directory domain. After NTLM authentication is enabled or disabled, rejoin the domain.

The following limitations apply to Multi-Forest Authentication:

https://<VMS IP>/api/prometheusmetrics/ 

https://<VMS IP>/api/prometheusmetrics/all 

https://<VMS IP>/api/prometheusmetrics/users 

https://<VMS IP>/api/prometheusmetrics/defrag 

https://<VMS IP>/api/prometheusmetrics/views 

https://<VMS IP>/api/prometheusmetrics/devices 

https://<VMS IP>/api/prometheusmetrics/quotas 

When creating a support bundle with the METADATA preset, only one CNode can be selected for the bundle. Selecting any DNode(s) or multiple CNodes together with the METADATA preset results in an error.

(RESOLVED IN 5.0.0-SP60) ORION-189672: An attempt to run a VAST Easy Install process with the Northband option selected for the management network fails with the northband is supported only on cnodes and only those with 2 HCAs error. If you encounter this issue, contact VAST Support for a workaround.

ORION-145815: In some cases, VAST Cluster does not raise an alert on a wrong NIC firmware version during a cluster upgrade.

ORION-175762: In some cases, a DBox expansion procedure run on a cluster with similarity-based data reduction enabled can take longer than expected.

ORION-205395: If, during an HA event on a cluster with InfiniBand internal networking the OpenSM service is found unavailable on a CNode, the CNode may occasionally encounter a failed connecting to the leader's platform error.

ORION-155530: Sometime,s after you run the cluster networking configuration script (configure_network.py) and then rebooted the CNode, the eb1 interface can still be down with the Device ib1 has different MAC address than expected, ignoring error. In this case, rerun the script after the reboot to bring the interface up.

(RESOLVED IN 5.1.0) ORION-114057: A tenant_create returned an error : ObjectCreateResultCode.FAILURE error occurs when attempting to create 256 tenants, each with a unique encryption group, if prior to this attempt, a tenant with per-tenant encryption enabled was created and then deleted.

(RESOLVED IN 5.2.0) ORION-178975: After creating a user quota with the identifier type set to UID, VMS lists this quota under the corresponding username pulled from the LDAP provider but not under the UID specified during quota creation.

(RESOLVED IN 5.1.0-SP50, 5.2.0-SP6) ORION-201538: The lifecycle rule mechanism deletes empty directories that were created through NFS and SMB protocols on the view for which a lifecycle rule is enabled, even when the empty directories are not expired according to the enabled lifecycle rule.

ORION-139913: When applying a QoS policy to NFSv3 access, both data and metadata are taken into account in QoS limit calculations, while with NFSv4.1, only data are considered.

ORION-137986: Enabling a QoS policy for a view on which a mixed (read and write) workload runs, can result in decreased performance for the workload.

(RESOLVED IN 5.1.0-SP60, 5.2.0-SP10) ORION-216774: For views with the SMB and S3 protocols enabled and the Mixed Last Wins or SMB security flavor set, the owner of a child directory in a parent that has no default ACL, may in some cases be set incorrectly.

(RESOLVED IN 5.3.0) ORION-204972: When creating S3 objects on a multi-protocol view controlled with the NFS security flavor, in a directory for which the SGID POSIX modebit is set, the SGID modebit may get propagated to files/objects created in that directory.

(RESOLVED IN 5.1.0-SP50) ORION-193090: The READDIR and READDIRPLUS operations against a directory with a name longer than 255 characters may hang without returning an error.

(RESOLVED IN 5.1.0) ORION-135514: The word percent in the CNode <...> nfs over rdma connections is at <...> percent alert should be read as connections, since the alert shows the number of connections but not a percentage.

(RESOLVED IN 5.3.0) ORION-144020: When use of Kerberos/NTLM authentication to authorize SMB users from non-trusting domains is enabled for the tenant, a Windows client would let you add a new ACE only by searching for a specific user in the list of trusted forest users, instead of locating the user through the list of domains.

ORION-142968: If a quota is exceeded during the process of coping a file to the VAST cluster, the copying process is stopped with a misleading error message: A device attached to the system is not functioning.

(RESOLVED IN 5.1.0-SP60) ORION-217661: If the final part of a multipart upload has a size of 0 (zero), VAST Cluster responds with a 400 Bad Request error.

(RESOLVED IN 5.3.0) ORION-198606: In rare cases, an IO is stuck - should close alert can be raised on a CNode caused by the cluster waiting for completion of an S3 multi-part upload.

(RESOLVED IN 5.1.0) ORION-136816: S3 GET of a symlink is blocked but HeadObject and GetObjectACL operations still succeed.

(RESOLVED IN 5.1.0) ORION-134836: When displaying path details in the VAST Audit log dialog, the phandle field does not show the phandle.

(RESOLVED IN 5.0.0-SP60) ORION-189280: VAST Cluster does not exclude VAST Database elements from the scope of S3 lifecycle rules. This means that in some cases, database elements can get deleted if the S3 lifecycle rules configured on the cluster stipulate the deletion.

ORION-163038: When importing data into a VAST Database table and there is a type mismatch between the column and the data being imported, VAST Cluster produces an ambiguous error message (Failed to get column) instead of pointing to the expected data type.

(RESOLVED IN 5.2.0, 5.1.0-SP50) ORION-196575: An attempt to bulk delete a large number of protected paths may result in a timeout in case an issue occurs during deletion of one of the protected paths.

(RESOLVED IN 5.1.0-SP30) ORION-201982: An attempt to replicate from more than eight source clusters may result in a CNode container restart with the Buffers pool is exhausted error.

ORION-183432: When trying to perform a failover using the protectedpath modify --modify-replication-state VAST CLI command, the replication state remains Standalone, although it is expected to change from Standalone to Source. If you encounter this issue, use VAST Web UI to perform the failover.

(RESOLVED IN 5.2.0) ORION-144137: User quotas for Alternate Data Stream (ADS) children might get miscalculated at the replication destination when the size and/or used attributes of an ADS child are updated due to replication.

ORION-140894: When attempting to delete a protected path from the destination peer after an ungraceful failover, a Failed to delete following streams or similar error occurs. The workaround is to manually change the destination peer's role to STANDALONE and retry the deletion.

(RESOLVED IN 5.1.0) ORION-146029: When sending call home bundles from a VAST on Cloud (VoC) cluster, the Multi-Cluster Manager (MCM) sends the first bundle an hour after the cluster has been registered, and the following bundles are sent according to the user-defined interval.

ORION-196963: The owner for files and folders created on an NFS4.1 view can be occasionally reported as nobody instead of the correct value. This issue can occur if both LDAP and Active Directory providers (with different domain names) are configured for the view’s tenant and the same group exists on both providers, but the user is part of this group on the LDAP provider only.

(RESOLVED IN 5.1.0) ORION-144288: Due to a caching issue, an incorrect user UID can be returned in a user query being retried immediately after the connectivity to the provider has been restored.

(RESOLVED IN 5.0.0-SP60) ORION-206781: A CNode bulk activation task activates only first five CNodes and ignores all the rest nodes in the batch.

ORION-203155: The Unexpected width, actual link width is <...>  alarm message may contain garbage at the end of the message.

(RESOLVED IN 5.0.0-SP60) ORION-182932: The name of the BW (Mb/s) column in the Global Snapshot Clones page (Data Protection -> Global Snapshot Clones) should read BW (MB/s) to denote megabytes per second.

(RESOLVED IN 5.0.0-SP60) ORION-180832: When displayed in VAST Web UI, the definition of the CNode - ProtoMetrics,proto-name=ProtoCommon,latency (ms) event includes ms as a unit of measurement, which is typically used to denote milliseconds. However, in this particular event definition, the threshold is set in microseconds.

(RESOLVED IN 5.2.0) ORION-172811: Some analytics properties that can be selected when creating a customized analytics report, produce a graph that does not precisely correspond to the property name. For example, selecting the NFS Write IOPS property produces a graph showing the write IOPS not only for NFS but for all protocols. In particular, this issue may occur with protocol-specific and replication-related properties that represent bandwidth, IOPS and latency.

(RESOLVED IN 5.1.0) ORION-147658: An attempt to add a user quota for a non-existing user does not  raise an error.

ORION-143717: On a cluster with CNode Port Affinity configured, there is no way to expose the VAST DNS IP on a specific port (left or right).

(RESOLVED IN 5.1.0) ORION-134765: The Rows filtered out and Rows scanned metrics in the VAST DB Row Metrics analytics report show the total number of rows accumulated over time while other metrics in the report show the number of rows per second.

ORION-131386: When there is a parent directory that has a very large number of child directories, a total of children’s capacity values displayed in the Capacity page can exceed the capacity value shown for the parent directory.

ORION-89570: In some cases, capacity analytics for subdirectories cannot be reported due to an internal timeout. This issue occurs when there is an extremely large number of subdirectories to be estimated.

(RESOLVED IN 5.0.0-SP60) ORION-194719: When trying to create a virtual IP pool for all tenants via VAST Web UI (Network Access -> Virtual IP Pools -> choose to create a pool -> select All Tenants in the Tenant field), VAST Cluster creates a pool for the default tenant, instead of creating a pool for all tenants.

(RESOLVED IN 5.0.0-SP60) ORION-184012: An attempt to export VAST Catalog query results to a CSV file results in a Query: Please correct the form error with no CSV file generated.

(RESOLVED IN 5.0.0-SP60) ORION-182932: The name of the BW (Mb/s) column in the Global Snapshot Clones page (Data Protection -> Global Snapshot Clones) should read BW (MB/s) to denote megabytes per second.

ORION-169645: A tip for the Atime Frequency field (Element Store -> View Policies -> choose to create or edit a view policy -> General tab) states that 3600s is the default value for this field, while the actual default is 0 (no atime updates).

ORION-150503: A local user cannot be found when trying to add it as a value in the Database Owner field of the New Database dialog.

ORION-147073: The Database page does not show the actual number of rows and size of objects until the page is refreshed manually.

(RESOLVED IN 5.1.0) ORION-146832:  After an existing VAST Web UI session has timed out, the Multi-Cluster Management page may display a prompt to enter a registration token for a cluster for which the token has already been provided. To eliminate the prompt, refresh the page.

(RESOLVED IN 5.1.0) ORION-146273: After deleting a cluster in the Multi-Cluster Management page, subsequent delete confirmation popups can show the Type DELETE to approve field pre-populated with the DELETE word.

(RESOLVED IN 5.1.0) ORION-143724: Some of the columns in the SSDs tab of the Infrastructure page opened through Multi-Cluster Management may show dm_mock or mock dev values instead of model and firmware version numbers.

ORION-142547: Clicking the Vast catalog policy link in the Policy column of the Snapshots page in Multi-Cluster Management opens an empty Protection Policies page instead of showing a specific policy.

(RESOLVED IN 5.1.0) ORION-141670: Relative file symlinks created through SMB are listed as directory symlinks and require use of rmdir to be deleted.

(RESOLVED IN 5.2.0) ORION-140652: Auto-completion for the Logon name of the privileged domain user field in tenant settings (Element Store -> Tenants -> choose to create or edit a tenant) is not provided.

(RESOLVED IN 5.1.0) ORION-139890: The QoS policy field in the Create View or Update View dialog (Element Store -> Views -> choose to create or edit a view) can list both view QoS policies and user QoS policies, although it does not let you add a user QoS policy to the view.

(RESOLVED IN 5.1.0) ORION-146200: The auto-completion options for the role-assign command do not list all possible parameters.

(RESOLVED IN 5.1.0-SP50) ORION-201905: When trying to retrieve the segments retransmitted metrics with an API call to /api/monitors/ad_hoc_query/, a "detail": "metrics not available" error can occur.

(RESOLVED IN 5.0.0-SP60) ORION-185217: The /users/<user ID>/ endpoint returns an empty JSON in response to an update request, although the requested updates are made as expected.

ORION-178569: The /users/names endpoint always returns only the first 50 entries, regardless of the page size parameter or the total amount of entries to be returned.

ORION-205393: After disconnecting and reconnecting an InfiniBand switch, the cluster might encounter a CNode container restart due to the assertion failed: (!has_verifier(mem_dev->dest().env_id)) Failed performing rpc call! lock_op=HAS_TEMP_REFS error.

ORION-203504: A finished redistribution and still not balanced alert can occur on the cluster when one of the CNode ports is disconnected and thus even distribution of virtual IPs among the platform ports is not possible. If there are no accompanying messages indicative of any issues, this alert can be ignored.

ORION-202806: When handling extreme workloads, CNode containers may occasionally restart with the timeout expired for life_type=16,life_gen=<number> (TRAVIS) error. The error means that the cluster is busy processing the workload. If there are no other symptoms indicative of any issues, no human intervention is required.

(RESOLVED IN 5.3.0) ORION-193956: The leader hogging for <number> us message may occasionally appear in VAST logs. If there are no accompanying messages indicative of a failure, this message can be ignored.

(RESOLVED IN 5.1.0-SP60) ORION-158539: The back view for the CERES DBox in the Hardware Layout page shows the data ports in incorrect positions (e.g. port enp3s0f1 is shown on the right while it should be on the left). To mitigate the issue, refer to the Infrastructure -> NICs page that lists the correct locations for the ports.

ORION-239170: When obfuscating a support bundle, the CNode hostname may not get obfuscated in some of the logs included in the bundle.

(RESOLVED IN 5.1.0) ORION-143381: When the directory used to store call home bundles reaches its size cap, a FileNotFoundError: [Errno 2] No such file or directory error is reported instead of an out-of-space error.