Following are new features introduced with VAST Cluster 5.2.0:
PLATFORM & CONTROL
ENCRYPTION OF DATA AT REST
REPLICATION
VAST DATA ENGINE
IDENTITY AND BUCKET POLICIES
S3
VAST DATABASE
GLOBAL ACCESS
A New Platform: EBox
We introduce a new type of platform named EBox which is designed for hyperscale use. An EBox includes one CNode container and two active-active DNode containers running in one physical enclosure.
You can manage EBoxes using relevant controls under Infrastructure in VAST Web UI, and also through ebox <action> commands of VAST CLI.
The following limitations apply:
ORION-193794: Power cycling of an EBox where the leader was running may result in significant IOPS degradation until the EBox is up again. Contact VAST Support for a workaround.
DBox migration is not available for EBoxes.
Write Buffers on Flash
A VAST cluster can be configured to increase the amount of write buffers available to handle bursts of high bandwidth writes. This can increase the overall write speed during the burst. The write buffer capacity is dynamically increased by using flash drives for write buffers.
To manage the status of this feature, use the new --enable-use-flash-write-buffers and --disable-use-flash-write-buffers options on the cluster modify command.
The following limitation applies:
Use of flash write buffers on an EBox with the DBox HA capability enabled requires at least 10 boxes to be present on the cluster.
Metadata Triplication
Metadata triplication improves cluster resiliency by allowing it to lose two metadata sections (on SCM or DNode RAM) at the same time without any loss of metadata or access.
Metadata triplication is supported for eBoxes only, where it is enabled by default and cannot be disabled.
CNode OS Boot Drive Encryption
VAST Cluster 5.2 supports optional encryption-at-rest for the OS boot SSDs on CNodes.
The following limitation applies:
ORION-208004: Enabling VAST OS boot drive encryption requires that the node is inactive. Enabling the encryption on an active node may cause a long reboot sequence.
Encrypted Paths
With VAST Cluster 5.2, a tenant can use different EKM encryption keys to protect different encrypted paths. For more information, see Encryption of Data at Rest.
The following user controls have been added for this feature:
In VAST Web UI, the Encrypted Paths page that lets you view, create, modify and delete encrypted paths (Element Store -> Encrypted Paths).
In VAST CLI:
encryptedpath <action>commands to manage encrypted pathsencryptiongroup <action>commands to manage encryption groups.
Synchronous Replication for S3
Synchronous Replication provides resiliency in the case of a disaster (full cluster failure) with no data loss. Replication peers and protected paths are configured as for asynchronous replication. For synchronous replication however, data is continuously replicated between source and destination replication peers so that, in the event of a failover, the destination can take over fully synchronized with source data.
With synchronous replication, clients can have read-write access to the protected path on both the source or destination clusters. A write operation on either is replicated to the other, to maintain full data synchronization at all times.
The following limitations apply:
Synchronous replication in this release is supported for S3 buckets only.
It is not allowed to configure local snapshots, asynchronous replication or Global Access on the same protected path.
Up to 250 replication streams are supported.
S3 lifecycle rules are not replicated.
S3 keys are replicated asynchronously.
Synchronously replicated directories are not subject to bulk permission updates.
See S3 Synchronous Replication for more details.
On-Demand Replication
With this release, you can start a replication of any remote protected path at any time instead of waiting for the next scheduled replication. The snapshot is replicated to all defined destinations for the protected path. For more information about this feature, see Replicating a Protected Path on-Demand.
To replicate on demand, on the source peer:, use the Replicate Now action
In VAST Web UI, choose the Replicate Now action for a protected path (Data Protection -> Protected Paths -> right-click the path you want to replicate)
In VAST CLI, run the
protectedpath replicate-nowcommand.
Bucket Replication
If bucket replication is enabled on the source replication peer, any S3 buckets on or under replicated protected paths are automatically recreated at the replication target path, provided the bucket views are configured on the source peer with the S3 default view policy. This prevents the need to manually create buckets in order to enable S3 access to data at replicated paths.
For more information about this feature, see S3 Access to Replicated Data.
To enable bucket replication:
In VAST Web UI, go to Settings -> S3 in VAST Web UI and click the Enable button next to the Bucket replication option.
In VAST CLI, run the
cluster modifycommand with the--enable-bucket-replicationoption specified.
Note
Bucket replication cannot be disabled.
Event Publishing / S3 Event Notifications
VAST Cluster 5.2 can stream events to a third-party event broker.
In the context of event publishing, an event is a change that has occurred for an element in VAST Element Store, such as creation of a new object in an S3 bucket. When a certain event is encountered, VAST Cluster can send a JSON record of it to the connected event broker, where the information can be consumed by applications.
As of version 5.2, VAST Cluster supports Apache Kafka as the event broker. Event publishing is only available for S3 bucket views.
The following user controls have been added for this feature:
In VAST Web UI:
The External Kafka tab in cluster notification settings where you can view and manage event brokers (Settings -> Notifications -> External Kafka Brokers).
The Bucket Notifications tab in view settings where you can set up S3 event notifications for a view (Element Store -> Views -> choose to create or edit a view).
In VAST CLI:
The event broker management commands:
kafkabroker <action>The event notification management commands:
eventnotification <action>
In VAST REST API:
The
/kafkabroker/and/kafkabroker/<ID>/endpointsThe
event_notificationsparameter for the/views/and/views/<ID>endpoints
S3 bucket notifications can also be managed through S3 API using GetBucketNotificationConfiguration and PutBucketNotificationConfiguration requests.
For more information about this feature, see Event Publishing.
Bucket Policies
VAST Cluster 5.2 adds support for bucket policies.
A bucket policy is attached to a view that has the S3 protocol enabled. It defines actions that particular user(s) and/or group(s) are allowed or denied to perform against this view and files or directories under that view.
Bucket policies can be managed by sending PutBucketPolicy, GetBucketPolicy and DeleteBucketPolicy requests to the VAST S3 server.
For more information about using S3 bucket policies with VAST Cluster, see Managing Bucket Policies.
Policy Conditions
VAST Cluster 5.2 lets you use conditions in identity policies and bucket policies.
A Condition element limits the effect of the policy to only those requests that match the conditions specified. A condition statement includes an operator, a condition key and a key value.
For more information about conditions, see Identity and Bucket Policy Reference.
The following limitation applies:
ORION-152475: An access denied error is returned for NFSv3 or NFSv4 requests if they are checked against an identity or bucket policy with an
s3:ExistingObjectTagcondition statement in it.
S3 Bucket Logging
VAST Cluster 5.2 introduces support for S3 server access logging, a native S3 feature that lets you set up detailed recording of operations on your bucket or bucket objects, with the logs written to another bucket.
When the source bucket has S3 bucket logging enabled, VAST Cluster creates a log entry in AWS log format for each S3 request made to the source bucket, and periodically uploads the log objects to a destination bucket. The format of log object keys can be configured to allow for date-based partitioning of log objects.
Note
Enabling bucket logging may have impact on performance.
For more information about this feature, see S3 Bucket Logging.
The following user controls have been added for this feature:
In VAST Web UI, the Bucket Logging tab in the Create View or Update View dialog.
In VAST CLI, the following options on the
view createandview modifycommands:--bucket-logging-destination-id--bucket-logging-prefix--bucket-logging-key-format--disable-bucket-logging
The following S3 requests are now supported:
PutBucketLoggingGetBucketLogging
The following limitations apply:
(RESOLVED IN 5.2.0-SP20) ORION-213741: Bucket logging cannot be effectively disabled after enabling it for a view. After toggling the Enable bucket logging option off, the feature state changes to disabled for a short time and then goes back to enabled.
ORION-197281: VAST Cluster disables bucket logging set on a bucket from which data is synchronously replicated to another bucket once you set up bucket logging on the replication destination bucket and configure it to use a different logging destination bucket.
Import Files to a VAST Database from the VAST Web UI
You can now import files into a VAST database table from the VAST Web UI. Parquet or CSV files can be imported. As part of the import process, checks are made to prevent duplicate imports of the same files.
See Importing Data to VAST Database for more details.
Prefetching Data to Global Access Peers
In this release, you can cache files and folders from a Global Access source cluster to a destination cluster. This allows the data to be available locally on the destination cluster ahead of its need, and not have to be fetched when needed.
You configure data prefetch on the destination cluster as described in Configuring Global Access. You can specify the path and determine whether to prefetch data or directory structure only. Prefetched data is copied to the cache on the destination cluster, and is subject to availability of storage in the cache and cache size rules.
The following user controls have been added for this feature:
In VAST Web UI, settings in the the Prefetch Data dialog (Data Protection -> Protected Paths -> right-click a path and choose Global Access/Prefetch Data).
In VAST CLI, the following commands and options to view and manage prefetch status:
cluster list-prefetch-folders-infoprotectedpath get-prefetch-statusprotectedpath prefetch-folder-to-cacheprotectedpath prefetch-delete
The following limitation applies:
ORION-194613: If some files have additional hardlinks, the amount of bytes reported as prefetched can be higher than the actual amount prefetched.