Known Issues in 5.1.0-SP4

(RESOLVED IN 5.1.0-SP30) ORION-187510: On clusters that have the Obfuscated option enabled in call home settings, an attempt to complete a CNode add task may fail with the ssh: Could not resolve hostname; <obfuscated IP> error. If you encounter this issue, contact VAST Support for a workaround.

ORION-145815: In some cases, VAST Cluster does not raise an alert on a wrong NIC firmware version during a cluster upgrade.

(RESOLVED IN 5.3.0) ORION-184301: When adding a CBox to the cluster using VAST Web UI, the external NICs are added even when the option to skip external NICs is selected.

ORION-183372: If a cluster has the Write Buffer RAID capability enabled, cluster expansion is only possible if all the SCMs in all DBoxes are of the same size.

ORION-175762: In some cases, a DBox expansion procedure run on a cluster with similarity-based data reduction enabled can take longer than expected.

ORION-173816: VAST Cluster does not block expansion from CERES 15TB to CERES 60TB, although such expansion is not supported.

ORION-161709: In some cases, capacity values reported during DBox migration do not match the capacity reported once the migration is complete.

(RESOLVED IN 5.1.0-SP60) ORION-214087: On a cluster where both external and internal interfaces are InfiniBand, the VMS may report the Failed to find the current OpenSM master with error: 'NoneType' object has no attribute 'ssh_conn' error if the OpenSM master is outside the cluster.

ORION-185008: Modifying a subnet in an existing virtual IP pool may result in the pool being not accessible by clients. To work around the issue, delete the existing pool and create a new one with the subnet you want.

(RESOLVED IN 5.2.0) ORION-175251: For clusters with MLK DNodes, when running the cluster’s network configuration script (configure_network.py) to create a B2B IPMI configuration, the script does not set enp211s0f0 as the B2B IPMI port, which results in inability to reach DNodes via IPMI (ipmitool).

(RESOLVED IN 5.2.0) ORION-175251: For clusters with MLK DNodes, when running the cluster’s network configuration script (configure_network.py) to create a B2B IPMI configuration, the script does not set enp211s0f0 as the B2B IPMI port, which results in inability to reach DNodes via IPMI (ipmitool).

ORION-155530: Sometimes after you run the cluster networking configuration script (configure_network.py) and then rebooted the CNode, the eb1 interface can still be down with the Device ib1 has different MAC address than expected, ignoring error. In this case, rerun the script after the reboot to bring the interface up.

(RESOLVED IN 5.2.0) ORION-160555: If a leader CNode HA event occurs during encryption group rotation, the rotation cannot be completed, and there is no indication that the rotation was not performed.

(RESOLVED IN 5.1.0-SP60, 5.2.0) ORION-206297: In some cases, quota capacity percentage shown in the Element Store -> Quotas page may not get updated properly to reflect the capacity consumption. If you encounter this issue, use Uplink to view the data.

(RESOLVED IN 5.1.0-SP51) ORION-219774: In some cases, the Identifier field in the Users Accounting tab in quota settings (Element Store -> Quotas -> choose to create or update a quota) displays the user's VAST ID (VID) instead of the user's username or UID.

(RESOLVED IN 5.1.0-SP50, 5.2.0-SP6) ORION-201538: The lifecycle rule mechanism deletes empty directories that were created through NFS and SMB protocols on the view for which a lifecycle rule is enabled, even when the empty directories are not expired according to the enabled lifecycle rule.

ORION-139913: When applying a QoS policy to NFSv3 access, both data and metadata are taken into account in QoS limit calculations, while with NFSv4.1, only data are considered.

ORION-137986: Enabling a QoS policy for a view on which a mixed (read and write) workload runs, can result in decreased performance for the workload.

(RESOLVED IN 5.3.1) ORION-236888: When uploading a file via S3 to a view controlled with the NFS security flavor, when the view is configured so that each new file is to inherit its owning group from the parent directory (--gid inheritance bsd), the resulting file does not inherit the directory’s owner group as expected.

(RESOLVED IN 5.1.0-SP60, 5.2.0-SP10) ORION-216774: For views with the SMB and S3 protocols enabled and the Mixed Last Wins or SMB security flavor set, the owner of a child directory in a parent that has no default ACL, may in some cases be set incorrectly.

(RESOLVED IN 5.3.0) ORION-204972: When creating S3 objects on a multi-protocol view controlled with the NFS security flavor, in a directory for which the SGID POSIX modebit is set, the SGID modebit may get propagated to files/objects created in that directory.

(RESOLVED IN 5.1.0-SP61) ORION-235601: On views controlled with the NFS security flavor with POSIX ACLs enabled, a permission translation issue may occur when a file inherits its ACL from the parent directory, resulting in unexpected Permission denied errors when an NFSv3 user, being a member of a group present in the ACL, tries to access a file created through NFSv4.

(RESOLVED IN 5.1.0-SP56) ORION-216344: If a file is created in a directory controlled with the default ACL by a client that does not overwrite ACLs upon creation (such as the Windows SMB client), NFSv3 access to such file may in some cases be denied for users which are members of the group that owns the file, while the file owner user does have the access.

(RESOLVED IN 5.1.0-SP50) ORION-193090: The READDIR and READDIRPLUS operations against a directory with a name longer than 255 characters may hang without returning an error.

(RESOLVED IN 5.1.0-SP40) ORION-190107: An attempt to create an NFS mount using RDMA over IPv6 is rejected with the Connection refused error.

(RESOLVED IN 5.1.0-SP60) ORION-212855: If the Everyone group entry is added after an existing group entry in a share-level ACL, the above group entry may get duplicated and the Everyone entry may disappear. The workaround is to place the Everyone entry prior to other entries in the ACL.

(RESOLVED IN 5.2.0) ORION-179201: When listing SMB shares that reside on a VAST cluster, the share path field may show the share name but not the complete path.

ORION-155854: If you run a command to release locks twice in a row, the number of locks that VMS reports as successfully released for the second run includes the released locks both from the first run and from the second run.

(RESOLVED IN 5.3.0) ORION-144020: When use of Kerberos/NTLM authentication to authorize SMB users from non-trusting domains is enabled for the tenant, a Windows client would let you add a new ACE only by searching for a specific user in the list of trusted forest users, instead of locating the user through the list of domains.

ORION-142968: If a quota is exceeded during the process of coping a file to the VAST cluster, the copying process is stopped with a misleading error message: A device attached to the system is not functioning.

(RESOLVED IN 5.1.0-SP63) ORION-241388: During an upgrade from VAST Cluster 5.0 to VAST Cluster 5.1, any views which had S3 Object Ownership set to BucketOwnerEnforced mode, are automatically reconfigured to ObjectWriter mode.

(RESOLVED IN 5.1.0-SP56) ORION-227797: VAST Cluster does not support trailing checksums included in a an S3 request with the x-amz-trailer header. If the request contains a trailing checksum, the checksum is not recognized as such and is appended to the request body.

(RESOLVED IN 5.1.0-SP60) ORION-217661: If the final part of a multipart upload has a size of 0 (zero), VAST Cluster responds with a 400 Bad Request error.

(RESOLVED IN 5.3.0) ORION-198606: In rare cases, an IO is stuck - should close alert can be raised on a CNode caused by the cluster waiting for completion of an S3 multi-part upload.

(RESOLVED IN 5.2.0) ORION-189731: When creating an S3 Bucket view with the ACLs disabled option selected, the view is created with ACLs enabled instead, and after a short period of time after saving the changes VMS reports the view as ACLs enabled.

This issue does not occur when modifying an existing view. If you encounter this issue after the view has been created, open the Update View dialog for the view (Element Store -> Views -> choose to edit a view) and set it to ACLs disabled again.

ORION-188749: The view policy option that restricts S3 read/write access based on client IP addresses (in VAST Web UI: Element Store -> View Policies -> choose to create or edit a view policy -> Host-Based Access -> S3 pane -> Read/Write) does not work for bucket-level operations, such as creating or deleting a bucket.

(RESOLVED IN  5.3.0) ORION-182790: VAST Cluster returns an incorrect error code, MalformedACLError, on attempts to put an object or bucket ACL based on a non-existing user email address.

(RESOLVED IN 5.1.0-SP40) ORION-206735: Logging of protocol operations after having enabled the feature to save protocol audit results to a VAST Database table (Settings -> Auditing -> General tab -> the Save audit logs to VAST DB toggle) may impact performance of subsequent VAST Database and VAST Catalog queries.

(RESOLVED IN 5.2.0) ORION-176151: When creating a view with many 50-character ABAC tags in VAST Web UI, an ABAC tags were not set, they were inherited from another view instead error message can be displayed although no ABAC tag inheritance occurs. To avoid this message, try reducing the length of ABAC tags by one character.

(RESOLVED IN 5.2.0) ORION-171054: If an NFSv4.1-enabled view has ABAC tags assigned, an attempt to mount a nested view through NFSv4.1 fails with a Permissions Denied error.

ORION-163697: When an SMB user accesses a file for which the user has ABAC set to read-only, a lock is placed on the file although the user does not have read/write permissions for the file.

(RESOLVED IN 5.1.0-SP60) ORION-216244: In some cases, VAST Catalog can include duplicate entries for the same item (e.g. two identical rows with different row IDs).

ORION-175036: The vast.rpc.import_data RPC, when used on VAST Cluster 5.1.0 with Trino 429, may fail to import data from multiple Parquet files due to the Procedure not registered: rpc.import_data error. If that occurs, try importing with the following command:

INSERT INTO vast."<bucket name>/<schema name>"."<table_name> vast.import_data" ("$parquet_file_path") 
VALUES (<parquet1>), ..., (<parquetN>) 

ORION-163038: When importing data into a VAST Database table and there is a type mismatch between the column and the data being imported, VAST Cluster produces an ambiguous error message (Failed to get column) instead of pointing to the expected data type.

(RESOLVED IN 5.1.0-SP56) ORION-209079: Assigning a VLAN to application CNodes (e.g. Data Engine -> Applications -> choose to create or edit an application -> Network tab -> VLAN field) does not work as expected, resulting in incorrect network configuration.

ORION-196575: An attempt to bulk delete a large number of protected paths may result in a timeout in case an issue occurs during deletion of one of the protected paths.

ORION-190517: VAST Cluster does not delete replicated protection policies when the associated protected path is deleted.

(RESOLVED IN 5.1.0-SP30) ORION-186788: An attempt to create a remote protected path between two non-default tenants in VAST Web UI results in The default tenant may only replicate to a remote default tenant error.

(RESOLVED IN 5.1.0-SP30) ORION-186816: When trying to create a remote protected path with the S3 Replication capability via VAST Web UI, the path is created with a role of Standalone and a state of Local.

ORION-183432: When trying to perform a failover using the protectedpath modify --modify-replication-state VAST CLI command, the replication state remains Standalone, although it is expected to change from Standalone to Source. If you encounter this issue, use VAST Web UI to perform the failover.

(RESOLVED IN 5.2.0) ORION-144137: User quotas for Alternate Data Stream (ADS) children might get miscalculated at the replication destination when the size and/or used attributes of an ADS child are updated due to replication.

ORION-140894: When attempting to delete a protected path from the destination peer after an ungraceful failover, a Failed to delete following streams or a similar error occurs. The workaround is to manually change the destination peer's role to STANDALONE and retry the deletion.

ORION-154973: Prior to making massive deletions (from the on-premises cluster) on a path that is configured to be replicated to a remote cluster, you need to delete the protection policy associated with this path on the on-premises cluster to avoid deletion performance degradation.

ORION-145307: Bulk permission updates are not supported for files and directories on remote clusters.

(RESOLVED IN 5.1.0-SP60) ORION-212816: An attempt to query a user group that has been updated by assigning an identity policy in VAST Web UI may fail with a The policy and the group must belong to the same tenant error. If you encounter this issue, use VAST CLI and specify the tenant ID explicitly on the query group command using the --tenant-id option.

(RESOLVED IN 5.1.0-SP50) ORION-209468: An attempt to get S3 access keys using the <vms_vip>/api/s3keys/ endpoint results in a identifier\n value is not a valid enumeration member validation error.

(RESOLVED IN 5.1.0-SP30) ORION-192018: An attempt to create an Active Directory configuration fails with the Invalid domain configuration. DC should be provided for organizational unit error if the specified organizational unit (OU) contains a white space after the comma separator of OU elements (for example: CN=Computers, DC=AD, DC=com).

(RESOLVED IN 5.1.0-SP30) ORION-190541: An attempt to attach an identity policy to a user via VAST Web UI or VAST CLI results in an validate attached policies() missing 1 required positional argument: 'tenant guid' error.

(RESOLVED IN 5.1.0-SP50) ORION-183026: If you edit a disabled identity policy, the policy is automatically enabled when you save the changes.

(RESOLVED IN 5.2.1) ORION-221021: The CNode write latency event definition contains the word 'read' instead of 'write' in the description text. The event description should read: CNode <CNode> write latency is {threshold} micro seconds.

(RESOLVED IN 5.2.0) ORION-180832: When displayed in VAST Web UI, the definition of the CNode - ProtoMetrics,proto-name=ProtoCommon,latency (ms) event includes ms as a unit of measurement, which is typically used to denote milliseconds. However, in this particular event definition, the threshold is set in microseconds.

ORION-175334: When creating a managed application (Data Engine -> Applications -> choose to create an application), VMS does not block attempts to set port membership for CNodes on a cluster that does not support CNode Port Affinity.

(RESOLVED IN 5.2.0) ORION-172811: Some analytics properties that can be selected when creating a customized analytics report, produce a graph that does not precisely correspond to the property name. For example, selecting the NFS Write IOPS property produces a graph showing the write IOPS not only for NFS but for all protocols. In particular, this issue may occur with protocol-specific and replication-related properties that represent bandwidth, IOPS and latency.

(RESOLVED IN 5.2.0) ORION-168744: Occasionally, some local users that can be seen under User Management in VAST Web UI, cannot be found in relevant fields of other VAST Cluster features.

ORION-143717: On a cluster with CNode Port Affinity configured, there is no way to expose the VAST DNS IP on a specific port (left or right).

ORION-131386: When there is a parent directory that has a very large number of child directories, a total of children’s capacity values displayed in the Capacity page can exceed the capacity value shown for the parent directory.

ORION-89570: In some cases, capacity analytics for subdirectories cannot be reported due to an internal timeout. This issue occurs when there is an extremely large number of subdirectories to be estimated.

ORION-246617: When creating or editing a lifecycle rule (Element Store -> Lifecycle Rules -> choose to view or edit a rule), the field for setting the unit of measure for the Minimum object size and Maximum object size filters is not compatible with the VMS setting that stipulates use of base2 units. It is recommended to switch to base10 before using this field.

(RESOLVED IN 5.1.0-SP60) ORION-224820: The option to acknowledge alarms in the Alarms page (Alarms & Events -> Alatms -> right-click an alarm) does not acknowledge alarms.

(RESOLVED IN 5.2.0-SP6) ORION-217317: Changes made to the default POSIX modebits in the Update Policy dialog (Element Store -> View Policies -> choose to update a policy -> Default POSIX modebits tab) cannot be saved and do not take effect. Use VAST CLI's viewpolicy modify --smb-file-mode or viewpolicy modify --smb-directory-mode commans instead.

(RESOLVED IN 5.2.0) ORION-184012: An attempt to export VAST Catalog query results to a CSV file results in a Query: Please correct the form error with no CSV file generated.

(RESOLVED IN 5.2.0) ORION-175189: When querying a local user using the Aggregated context, the Leading GID and Primary group SID fields in the User Details dialog have a value of -1 instead of an empty string.

(RESOLVED IN 5.2.0) ORION-175071: The number 9223372036854775807 shown in the tooltip for the Files and directories field under Settings -> Cluster -> General -> Global Access Cache Limits refers to the maximum number of files and directories that VMS can handle. When using VAST Web UI, the number that can be saved for this field, may be less depending on the capabilities of your browser.

(RESOLVED IN 5.2.0) ORION-147073: The Database page does not show the actual number of rows and size of objects until the page is refreshed manually.

ORION-142547: Clicking the Vast catalog policy link in the Policy column of the Snapshots page in Multi-Cluster Management opens an empty Protection Policies page instead of showing a specific policy.

(RESOLVED IN 5.2.0) ORION-140652: Auto-completion for the Logon name of the privileged domain user field in tenant settings (Element Store -> Tenants -> choose to create or edit a tenant) is not provided.

ORION-322120: When displaying path names in VAST CLI, the names written in right-to-left languages may appear spelled from left to right.

(RESOLVED IN 5.1.0-SP40) ORION-184206: An attempt to run a cbox list command fails with the Invalid value for `subsystem`, must be a value greater than or equal to `1` error if the cluster uses subsystem 0.

(RESOLVED IN 5.2.0) ORION-175435: An attempt to attach an identity policy to a local user with the user modify --identity-policy-ids command results in a S3 Policies with the following ID don't exist error.

(RESOLVED IN 5.2.0) ORION-174843: An attempt to specify the --tenant-ids option on the role create command results in an unrecognized argument error.

ORION-156628: An attempt to run the viewpolicy show --audit command results in a 'ViewPolicyProtocolsAudit' object has no attribute 'get' error.

(RESOLVED IN 5.2.0) ORION-181114: When listing views through VAST REST API, the /views/ endpoint does not return the S3 object ownership rule correctly, although it is returned when querying a specific view through /views/<id>.

ORION-178569: The /users/names endpoint always returns only the first 50 entries, regardless of the page size parameter or the total amount of entries to be returned.

(RESOLVED IN 5.2.1) ORION-246581: VMS automatically selects the DEFAULT section layout for Solidigm SBFPF2BV307T hard drives, while the correct layout would be THIRTY_TB_DRIVES.

(RESOLVED IN 5.3.0) ORION-193956: The leader hogging for <number> us message may occasionally appear in VAST logs. If there are no accompanying messages indicative of a failure, this message can be ignored.

(RESOLVED IN 5.1.0-SP6) ORION-187685: Service interruption may occur due to an NVRAM RAID issue that may be encountered when the cluster rebalances multiple NVRAMs simultaneously.

(RESOLVED IN 5.2.0) ORION-184359: The minimum number of DBoxes required to implement DBox High Availability is 12 (twelve), which is one DBox more than stated in the documentation.

ORION-173461: Slot numbers displayed in the Slot column of the Infrastructure -> SCMs page in VAST Web UI are not aligned with slot numbers displayed in the Hardware Layout page.

(RESOLVED IN 5.2.0) ORION-162393: In some cases, a SSD failure during DBox migration might result in an failed op=RECOVER_DISK for 1500 times in a row in shard_id={ mega_shard_id=<...> mega_shard_count=<...>}. last failure res=9 will denylist op on this shard alert on a CNode. If you encounter this issue, contact VAST Support to clear the alert and complete the operation.

(RESOLVED IN 5.1.0-SP6) ORION-186057: A false alert stating assertion failed: (cpus->cnt == (USE_OTHER_CORE_FOR_THREAD_POOL ? 1 : 2) may occasionally be encountered on IceLake CNodes.

(RESOLVED IN 5.1.0-SP60) ORION-158539: The back view for the CERES DBox in the Hardware Layout page shows the data ports in incorrect positions (e.g. port enp3s0f1 is shown on the right while it should be on the left). To mitigate the issue, refer to the Infrastructure -> NICs page that lists the correct locations for the ports.

ORION-239170: When obfuscating a support bundle, the CNode hostname may not get obfuscated in some of the logs included in the bundle.

(RESOLVED IN 5.1.0-SP6) ORION-187697: If a proxy is configured in the cluster’s call home settings, the cluster may not be able to establish a connection to send call home bundles.