Overview
VAST compute clusters are container orchestration clusters that you can deploy on the VAST cluster's own hardware infrastructure and use to handle container orchestration for VAST DataEngine.
VAST DataEngine requires at least one compute cluster, which can be either a VAST compute cluster or an external Kubernetes cluster.
Performance Impact
The deployment of a VAST compute cluster reallocates a significant portion of compute resources from the VAST Cluster to the VAST compute cluster, and can therefore impact VAST Cluster performance. When you configure a compute cluster, you choose which CNodes are allocated to it and you choose from a selection of options how much resources are allocated from each of the CNodes you allocate. The following options are offered. Note the guidelines per option for a number of CNodes to exclude from the resource allocation:
Low Compute. This option allocates approximately 25% of resources to the compute cluster from allocated CNodes. There is no upper limit to the number of CNodes you can allocate with this option.
Balanced Compute and Storage. This option allocates approximately 50% resources from allocated CNodes, in a balanced configuration of CPU and RAM. This option requires you to leave at least one CNode not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST cluster.
Memory Heavy. Similar to Balanced, allocating more RAM and less CPU. This option requires you to leave at least two CNodes not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST Cluster.
High Compute. This option allocates 100% CNode resources from allocated CNodes. It requires you to leave at least two CNodes not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST Cluster.
Provisioning VAST Compute Clusters for Multiple Tenants
Provided you have sufficient resources, you can provision multiple VAST compute clusters on a given VAST cluster, and you can associate each compute cluster with one or more tenants, where each tenant has DataEngine deployed. The DataEngine loads of all tenants associated with any given compute cluster share that compute cluster's resources.
Deployment Permissions and Flow
Creation and management of VAST compute clusters, as well as attaching VAST compute clusters to tenants, require a VMS manager user of type cluster admin and is done on the cluster level.
As a cluster admin user, you can attach VAST compute clusters to tenants. If DataEngine is enabled on a tenant and a VAST compute cluster is attached to the tenant, the VAST compute cluster becomes linked to DataEngine.
A tenant admin user logged into a tenant VMS can see details of VAST compute clusters that are linked to DataEngine on the tenant. When a tenant admin user enables DataEngine on a tenant, an attached compute cluster is automatically linked. The wizard for enabling DataEngine shows the tenant admin user that the compute cluster is linked and allows the user to continue with the DataEngine configuration without linking another compute cluster.
Security
A VAST compute cluster acts as a certificate authority (CA), allocating leaf certificates to internal services and client certificates to clients. Client access to the compute cluster's API server is supported for scraping metrics. You can choose to provide your own CA certificates to the VAST compute cluster or for VMS to generate them for you.
Viewing VAST Compute Clusters
Viewing VAST Compute Clusters Using the VAST Web UI
The Compute Clusters tab on the Data Engine page of the VAST Web UI can show you all VAST compute clusters configured on the VAST cluster or all DataEngine linked and all attached VAST compute clusters for a specific tenant.
From the Data Engine menu, select Compute Clusters.
From the Tenant dropdown, select one of the following:
All VAST Clusters to see all VAST compute clusters configured on the VAST cluster.
Note
You can select a cluster to see properties in the right panel, including which tenants are attached to the compute cluster.
A specific tenant. Selecting a specific tenant displays:
DataEngine Linked Clusters. All compute clusters that are linked to DataEngine on the selected tenant. These can also include external compute clusters.
VAST Compute Clusters. All VAST compute clusters that are attached to the tenant, whether or not DataEngine is enabled on the tenant.
The following properties are displayed:
For DataEngine Linked Clusters:
Cluster name
The name of the compute cluster.
Type
The type of compute cluster:
VAST. A VAST compute cluster.
External. An external compute cluster.
API Server URL
The IP of the server internal to the compute cluster to which you can send API requests, such as to scrape metrics.
Tags
Tenant specific tags associated with the linked compute cluster. These are distinct from the tags defined in the compute cluster's global configuration for the VAST cluster. They are added by editing the DataEngine linked cluster, which you can do by right-clicking the cluster in this list.
Description
A description of the compute cluster, if defined.
For VAST Compute Clusters:
Cluster name
The name of the compute cluster.
Attached Tenants
All tenants attached to the compute cluster.
Status
The status of the compute cluster:
Creating. Shows during compute cluster creation until the cluster creation is complete.
Updating. Shows after a user submitted changes to the compute cluster configuration, until the update task is finished.
Running. Shows in normal running state after cluster creation.
Deleting. Shows while the cluster is being deleted.
Partially provisioned. An error state shown when the compute cluster has been partially created and is not fully created. If this occurs, you can run the computecluster reconcile-create CLI command to attempt to complete the creation of the compute cluster.
Degraded. The compute cluster is running with some allocated CNodes inactive, following Error. The remaining CNodes are being added.
Error. The quorum of CNodes in the cluster has been lost. The cluster is being bootstrapped again from a single node.
Stopped. This status shows if the compute cluster has been intentionally stopped.
Stopping. This status shows if the compute cluster is in the process of stopping.
Starting. This status shows if the computer cluster has been restarted after being intentionally stopped and the starting process is not complete.
API Server
The IP of the server internal to the compute cluster to which you can send API requests.
Resources Limitation
Shows which one of the following settings was applied to allocate compute resources to the compute cluster from allocated CNodes:
Low Compute. This option allocates approximately 25% of resources to the compute cluster from allocated CNodes. There is no upper limit to the number of CNodes that can be allocated to the compute cluster with this option.
Balanced Compute and Storage. This option allocates approximately 50% resources from allocated CNodes, in a balanced configuration of CPU and RAM. This option requires you to leave at least one CNode not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST cluster.
Memory Heavy. Similar to Balanced, allocating more RAM and less CPU. This option requires you to leave at least two CNodes not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST Cluster.
High Compute. This option allocates 100% CNode resources from allocated CNodes. It requires you to leave at least two CNodes not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST Cluster.
Viewing Compute Clusters Using the VAST CLI
To list all VAST compute clusters, use the computecluster list command.
To view details of a specific VAST compute cluster, use the computecluster show command.
Creating a VAST Compute Cluster
Caution
This configuration dedicates a significant proportion of compute resources from the VAST cluster's CNodes to a new compute cluster that will reside internally on the VAST cluster. During the procedure, you will choose which CNodes to allocate and what proportion of resources to allocate. The resources you allocate to the compute cluster are no longer available for other VAST Cluster operations.
The process of creating a VAST compute cluster is a lengthy, rolling process that requires the sequential deactivation of the selected CNodes. Therefore, you will be prompted to confirm your choice before the operation begins.
You cannot create more than one VAST compute cluster in parallel.
Creating a Compute Cluster Using the VAST Web UI
From the Data Engine menu, select Compute Clusters.
Click Create Compute Cluster and then select Create VAST Compute Cluster.
Complete these fields:
Cluster name
A name for the compute cluster. The name must be 2-40 characters long and contain lowercase letters, numbers, and hyphens only.
Assign all tenants
Enable this setting if you want to automatically assign any tenant to the compute cluster whenever DataEngine is enabled on a tenant.
If this setting is enabled:
When you enable Data Engine on a tenant, this compute cluster is automatically linked to the tenant.
Manual assignment of tenants to this compute cluster is disabled
Description
A description of the compute cluster (optional).
Do one of the following to provide root and intermediate certificates for the compute cluster to use to issue certificates to its services:
To generate certificates for the compute cluster, click Auto Generate Certificates.
To provide your own certificates:
Click the Add new button for the Root CA cert.
Either click Upload and browse to the certificate file to upload, or paste the content of the certificate into the text field and click Save Root CA cert.
Repeat the two previous steps for the Intermediate CA cert and the Intermediate Private key, adding and saving each certificate.
If you want to add tags to the compute cluster, click Add tags, enter a key-value pair in the fields provided and click Add to Tags, for each tag that you want to add.
Note
After the compute cluster is linked to a tenant's DataEngine, you can also add tenant-specific tags.
Click Next.
Under Resources Limitation, choose one of the following to determine what resources to allocate to the compute cluster:
Low Compute. This option allocates approximately 25% of resources to the compute cluster from the CNodes you select under CNodes Selection. There is no upper limit to the number of CNodes you can select with this option.
Balanced Compute and Storage. This option allocates approximately 50% resources from the CNodes you select under CNodes Selection, in a balanced configuration of CPU and RAM. This option requires you to leave at least one CNode not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST cluster.
Memory Heavy. Similar to Balanced, allocating more RAM and less CPU. This option requires you to leave at least two CNodes not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST Cluster.
High Compute. This option allocates 100% CNode resources from the selected CNodes. It requires you to leave at least two CNodes not allocated to any compute cluster per every eight and part of eight DBoxes in the VAST Cluster.
Under CNodes Selection, from the list of All possible CNodes, select the CNodes that you want to allocate to the VAST compute cluster. You must select at least three CNodes. See also the previous step for requirements on the upper limit, depending on which Resource Limitation you selected.
Click Next.
Under Network, provide a pool of IP addresses to be statically allocated to the nodes of the compute cluster. These IPs will be used by the compute cluster's nodes to communicate with each other. They should be valid external IPs. Allocate a minimum of one more than the number of CNodes you allocated under CNodes Selection.
To enter the IP pool:
In the Netmask field, enter a netmask for the pool.
In the Start IP field enter the first IP in a range, and then the end IP of the range in the End IP field.
Click Add To Table.
Repeat the previous two steps as needed until you have allocated all the IPs you intend to add.
Under Advanced Configurations, optionally modify the following:
Note
These cannot be changed after creating the compute cluster.
Cluster CIDR
A subnet internal to the compute cluster from which IPs are assigned to pods. Change this from the default if the default CIDR overlaps with IP ranges used by infrastructure or external services, as this can cause incorrect routing and connectivity failures.
Default: 10.64.0.0/12
Server CIDR
A subnet internal to the compute cluster from which IPs are assigned to services. Change this from the default if the default CIDR overlaps with IP ranges used by infrastructure or external services, as this can cause incorrect routing and connectivity failures.
Default: 10.80.0.0/12
Default Gateway
Default gateway IP address for the compute cluster network
VLAN
VLAN tag for the compute cluster network. Valid values: 1-4095
Click Finish.
The process of configuring the compute cluster begins. This can take a while. You can monitor the process from the Activities page.
Creating a Compute Cluster Using the VAST CLI
To create a compute cluster, use the computecluster create command.
Modifying a VAST Compute Cluster
You can modify the following properties of a VAST compute cluster:
Configuration properties of the compute cluster:
The description
Tenant attachments, including specific tenant attachments and automatic assignment to all tenants.
Tags associated with the VAST compute cluster's global configuration for all tenants.
The resources limitation, which determines how much resources are allocated from CNodes
CNode selection, which determines which CNodes are allocated
CNode IPs, although you cannot remove an IP from the range in use unless you remove the associated CNode from the compute cluster. You can add the CNode back to the compute cluster after removing the IP from the configured range.
For each tenant on which the compute cluster is linked to DataEngine, you can add (and remove) specific tags.
It is also possible to rotate the cluster's certificates.
Modifying a Compute Cluster Using the VAST Web UI
From the Data Engine menu, select Compute Clusters.
From the Tenant dropdown, select either All VAST Clusters or a specific tenant that is attached to the compute cluster you want to modify.
Under VAST Compute Clusters, right-click the compute cluster and select Edit.
Make changes to modifiable fields as needed. See Creating a Compute Cluster Using the VAST Web UI for details.
Note
IPs that are currently in use by CNodes cannot be removed. In order to remove an IP from the range in use, you need to remove the associated CNode from the compute cluster, modify the IP range and then add the CNode back to the compute cluster.
Click .
Modifying Tenant-Specific Description and Tags for DataEngine Linked Compute Clusters
From the Data Engine menu, select Compute Clusters.
From the Tenant dropdown, select the tenant.
Under DataEngine Linked Tenants, right-click the compute cluster and select Edit.
Enter a tenant-specific description in the Description field (optional).
Under Tags, click Add tags and add any tags that you want to add for the tenant (optional).
Click .
The tags you added appear under the Tags column of the DataEngine Linked Tenants list for the compute cluster.
Modifying a Compute Cluster Using the VAST CLI
To modify the configuration of a compute cluster, use the computecluster modify command.
Managing Attachment of VAST Compute Clusters to Tenants
VAST compute clusters can be managed only by cluster admin users, while DataEngine can be enabled on tenants by both cluster admin and tenant admin users. In order to link DataEngine to a VAST compute cluster, a cluster admin user must attach the compute cluster to the tenant. This can be done before or after DataEngine is enabled on the tenant.
When you attach a tenant to a VAST compute cluster and then DataEngine is subsequently enabled on that tenant, the compute cluster is automatically linked to the DataEngine during DataEngine enablement.
If you attach a tenant to a VAST compute cluster when DataEngine is already enabled on that tenant, the compute cluster is automatically linked to the DataEngine.
You can attach tenants to a VAST compute cluster the following ways:
By automatically assigning all tenants to the compute cluster.
By attaching the specific tenant to the compute cluster before or after enabling DataEngine on the tenant.
Attaching all Tenants to a VAST Compute Cluster Using the VAST Web UI
From the Data Engine menu, select Compute Clusters.
Right-click the compute cluster and select Edit.
Enable Automatically assign all tenants.
Click .
Attaching One or More Specific Tenants to a VAST Compute Cluster Using the VAST Web UI
Right-click the compute cluster, select Attach Tenant.
The Attach Tenants To Compute Cluster dialog appears.
From the Tenants dropdown, select each tenant you want to add and click Add Tenants.
Click Save.
Attaching or Removing One or More Specific Tenants to or from a VAST Compute Cluster Using the VAST CLI
Use the computecluster bulk-tenants command to attach one or more specific tenants to a VAST compute cluster or remove one or more specific tenants from a VAST compute cluster.
Managing Compute Cluster Certificates
A VAST compute cluster acts as a certificate authority (CA), issuing leaf certificates for internal services and client certificates. Three artifacts are required for the compute cluster to act as a CA: A root CA certificate, an intermediate CA certificate and an intermediate private key.
When you create the cluster, you can either provide all three artifacts or you can choose for VMS to autogenerate them.
The VAST Web UI enables you to rotate the intermediate CA certificate and intermediate private key, either by providing new ones or autogeneration, rotate leaf certificates, and to rotate the service key.
Rotating Certificates Using the VAST Web UI
In the Compute Cluster tab of the Data Engine page, right click the compute cluster and select Rotate Certificates.
In the Rotate Certificates dialog, select one rotation operation. Only one rotation operation can run at one time.
Rotate CA Certificates. Keep the existing root CA certificate, upload or autogenerate a new intermediate CA certificate and intermediate private key, and rotate leaf certificates. If you want to upload the intermediate CA certificate and intermediate private key, use the Add new buttons for the two certificates that you can rotate. If you want to autogenerate the new certificates, select ]Auto Generate Certificates.
Rotate Leaf Certificates. This option rotates leaf certificates without replacing the root CA or intermediate CA certificates.
Rotate Service Key. This option rotates the RSA service key used for token signing. This option does not rotate any of the certificates.
Click Rotate.
The process of rotating certificates starts. To monitor the process, you can navigate to the Activities page and watch the Rotate compute cluster base certificates activity. When the process is complete, the State of the activity changes to COMPLETED.
Rotating Certificates Using the VAST CLI
You can initiate the following rotation operations through the VAST CLI:
Retain the current root CA certificate, and either regenerate or upload a new intermediate CA certificate and intermediate private key, and renegerate leaf certificates, use the computecluster rotate-base-certificates command.
Note
Rotation of root CA certificate is not supported.
To rotate leaf certificates without replacing the root CA or intermediate CA certificates, use the computecluster rotate-leaf-certificates command.
To rotate the RSA service key used for token signing, use the computecluster rotate-service-key command. This command does not rotate any of the certificates.
Monitoring and Analytics
Compute Cluster Analytics in the VAST Web UI
The Compute Cluster Analytics page provides performance metrics broken down by tenants, namespaces, pods, deployments, CNodes, and services.
The Compute Cluster Analytics page is available only if a VAST compute cluster is deployed.
To view compute cluster analytics:
From the left navigation menu, select Analytics and then Compute Clusters.
Monitoring Using the VAST CLI
The following commands are available for monitoring VAST compute clusters:
Collecting Metrics from the Compute Cluster
The compute cluster API server accepts client requests for collecting metrics.
To get the API server IP address, do one of the following:
In the VAST Web UI, from the Data Engine menu, select Compute Clusters. Under VAST Compute Clusters, locate the compute cluster. The IP address is displayed under API Server URL.
In the VAST CLI, run computecluster list or computecluster show. The API server IP is displayed in the output as
Server-kube-vip-address.
For client authentication, you can generate client certificates.
Generating Client Certificates Using the VAST Web UI
In the VAST Web UI, from the Data Engine menu, select Compute Clusters.
Under VAST Compute Clusters, right-click the compute cluster and select Generate Metrics Client Credentials.
A set of client certificates is generated and displayed.
Use the download or copy to clipboard buttons provided to get each certificate.
Close the dialog. After closing the dialog, the certificates are no longer available.
Generating Client Certificates Using the VAST CLI
To generate client certifiicates for metric collection, use the computecluster metric-viewer-certificates command.
Deleting Compute Clusters
Caution
Deleting a VAST compute cluster deletes all DataEngine triggers, functions and pipelines deployed on the compute cluster.
Deleting a Compute Cluster Using the VAST Web UI
In the Compute Cluster tab of the Data Engine page, right click the compute cluster and select Delete.
Read the confirmation warning and check the confirmation box.
Click Yes, delete cluster.
The process of deleting the compute cluster starts. To monitor the process, you can navigate to the Activities page.
Deleting a Compute Cluster Using the VAST CLI
To delete a compute cluster, use the computecluster delete command.