Data Engine requires a container orchestration cluster, which can either be an externally connected Kubernetes cluster or a VAST compute cluster that is deployed internally to the VAST cluster on some of the cluster's CNodes. During the initial enablement of Data Engine, you can configure and assign an external compute (Kubernetes) cluster. This is done in deployments where no VAST compute cluster is assigned or, optionally, in addition to assigning a VAST compute cluster. After enablement, you can modify the configuration of an external Kubernetes cluster that you configured, and you can configure additional Kubernetes clusters.
Viewing Kubernetes Clusters
From the left navigation menu, select Data Engine and then Compute Clusters.
The following fields are displayed for each external Kubernetes cluster:
Name
The name of the Kubernetes cluster resource.
API Server
The API server endpoint URL of the Kubernetes cluster.
GUID
A globally unique VAST ID for the Kubernetes cluster resource.
Tenant
The tenant to which the Kubernetes cluster resource belongs.
Tags
Any metadata tags added to the Kubernetes cluster resource.
Adding a Kubernetes Cluster
From the left navigation menu, select Data Engine and then Compute Clusters.
Click Create Compute Cluster and then select Link Kubernetes Cluster.
Complete the following fields:
Name (required)
Enter a name for the Kubernetes cluster to which you want DataEngine to connect.
API Server URL (required)
Enter the Kubernetes API server endpoint URL.
For example: https://kube-server-57:6443
Description
Enter a description of the Kubernetes cluster. This is optional.
In the Certificates area, use the fields provided to upload the CA certificate, client certificate and private key used for authenticating over mTLS to the Kubernetes cluster. For each certificate, click +Add new , paste the certificate content, base64 encoded, and then click Save Client certificate /Save CA certificate/Save Private Key.
In the Tags area, enter any metadata tag that you want to tag the Kubernetes cluster resource with, and select Add Tag to add the tag. Repeat for additional tags. This is optional.
Click Next.
In the Select Namespaces screen, the namespaces on the Kubernetes cluster are listed on the left under All possible Properties. Use the selection boxes and arrow buttons to choose which of the namespaces to allow the tenant's DataEngine to access and list them on the right under Selected Properties. Pipelines will be deployed in the selected namespace(s). Choose a unique namespace for the tenant. Namespaces cannot be shared by multiple tenants.
Caution
Do not select Zarf-installed workspaces such as vast-dataengine, knative-eventing, and knative-servering. Using Zarf-owned namespaces workspaces for pipeline deployments will fail since pods will fail to pull function images from the correct location.
Click Done.
Modifying Kubernetes Clusters
To modify a Kubernetes cluster configuration, right-click the Kubernetes cluster and select Edit.
Deleting Kubernetes Clusters
To delete a Kubernetes cluster, right-click the Kubernetes cluster and select Remove.