Use the following method to deploy VAST COSI Driver:
VAST COSI Driver Requirements
Ensure that your environment meets the following requirements:
-
A Kubernetes cluster is up and running.
-
A VAST cluster is up and running.
-
All nodes of the Kubernetes cluster are networked with the VAST cluster.
-
At least one node can communicate with the VAST Cluster management virtual IP.
-
A host is available with a Helm client installed, preferably in the VMS network.
Supported Versions
| VAST COSI Driver | Kubernetes | VAST Cluster | Helm |
|---|---|---|---|
| 2.6.x | 1.25 - 1.31.1 | 4.7.0-SP6 or later with the Trash Folder Access feature enabled | 3.x.x |
Required Permissions
VAST COSI Driver requires root (sysadmin) privileges.
Deploy VAST COSI Driver with Helm Chart
Before you begin, ensure that your environment meets the requirements.
VAST COSI Driver is deployed using Helm charts. A Helm chart is an installation template that can be reused to install multiple instances of the software being deployed. Each instance is referred to as a release. Helm charts are available from Helm repositories.
Steps to deploy VAST COSI Driver include:
-
(Optional) Install CustomResourceDefinitions for COSI.
-
Install CustomResourceDefinitions for snapshots.
-
(Optional) Create a Kubernetes namespace for VAST COSI Driver.
Note: This step is required if you are going to deploy VAST COSI Driver in a Kubernetes namespace other than
default. Otherwise, skip to step 4. -
Configure the VAST cluster.
-
Create a Kubernetes secret with VMS user credentials or API token for VAST COSI Driver.
-
Add the Helm repository that contains the VAST COSI Driver chart.
-
Create a Helm chart configuration file for VAST COSI Driver.
-
Install the Helm chart for VAST COSI Driver.
-
(Optional) Verify the deployment by launching a test application.
Install CRDs for COSI
Run the following commands to install the Custom Resource Definitions (CRDs) for COSI:
kubectl create -k github.com/kubernetes-sigs/container-object-storage-interface-api
kubectl create -k github.com/kubernetes-sigs/container-object-storage-interface-controller
Install CRDs for VAST Snapshots
Complete this step if you are going to use VAST snapshots on your Kubernetes cluster. The Custom Resource Definitions (CRDs) installed in this step are only required for using VAST snapshots.
Run the following commands to install the CRDs for snapshots:
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml
Create Kubernetes Namespace for VAST COSI Driver
By default, VAST COSI Driver is deployed to the default namespace on the Kubernetes cluster.
If you want to use a different Kubernetes namespace for VAST COSI Driver, create it prior to deployment by running the following command:
kubectl create ns <namespace_name>
Configure VAST Cluster for VAST COSI Driver
Complete the following steps to make your VAST cluster ready for integration with Kubernetes through the use of VAST COSI Driver:
-
Set up virtual IP pools to be used by VAST COSI Driver.
-
Configure a view policy to be used for views created by VAST COSI Driver.
-
Set up a VMS Manager user to be used by VAST COSI Driver.
-
Enable trash folder access on the VAST cluster.
-
(Optional) Upload your CA-signed SSL certificate to the VAST cluster.
-
(Optional): Configure a QoS policy to be associated with views created by VAST COSI Driver.
Set Up Virtual IP Pools
VAST COSI Driver distributes the load among virtual IPs in one or more VAST virtual IP pools.
The virtual IP pool is specified in the Kubernetes bucket class defined in the VAST COSI Driver chart configuration file. The virtual IP pool specified for a bucket class is used when processing read and write operations requested by the application that is using that particular bucket class
To view and manage virtual IP pools in VAST Web UI, log in and choose Network Access -> Virtual IP Pools in the main navigation menu. For more information about VAST Cluster virtual IP pools, see VAST Cluster Administrator's Guide.
Configure View Policies
VAST COSI Driver automatically creates a VAST Cluster view for each storage claim being provisioned. These views are controlled using VAST Cluster view policies.
The view policy is specified in the Kubernetes bucket class defined in the VAST COSI Driver chart configuration file. The view policy specified for a bucket class is used when processing read and write operations requested by the application that is using that particular bucket class.
To view and manage existing view policies in VAST Web UI, log in and choose Element Store -> View Policies. For more information about VAST Cluster view policies, see VAST Cluster Administrator's Guide.
Set Up VMS User
Set up a VMS user for VAST COSI Driver to communicate with the VAST Management Service (VMS) via VAST REST API.
You'll need to supply the VMS user credentials or authentication token in a Kubernetes secret that is specified when creating the VAST COSI Driver's Helm chart configuration file.
VAST COSI Driver can use the default admin user that is created by VAST during the initial install, or you can create a new VMS user. The user does not need to have full VMS permissions. The required permissions are Create, View, Edit and Delete permissions in the Security and Logical realm.
Permissions are granted to a VMS user by assigning a role. To view and manage roles in VAST Web UI, log in and choose Administrators -> Roles in the main navigation menu. For more information about managing roles, see VAST Cluster Administrator's Guide.
To view and manage VMS users in VAST Web UI, log in and choose Administrators -> Managers in the main navigation menu. For more information about managing VMS Manager users, see VAST Cluster Administrator's Guide.
To view and manage VMS authentication tokens, use apitoken * commands of VAST CLI or VAST REST API.
Enable Trash Folder Access
Trash folder access is required to allow for COSI cleanup.
To enable trash folder access in VAST Web UI, log in and choose Settings -> Cluster. In the General tab of cluster settings, toggle the Trash folder access option on.
Upload CA-Signed SSL Certificate to VAST Cluster
If you want to use a Certificate Authority-signed SSL certificate to secure the connection to the VAST cluster, follow the SSL certificate upload procedure in the VAST Cluster Administrator's Guide to upload your SSL certificate to the VAST cluster.
For more information about configuring SSL encryption for VAST COSI Driver, see Configure SSL Encryption for VAST COSI Driver.
Configure QoS Policy
You can optionally set up a Quality of Service (QoS) policy to be associated with the views that VAST COSI Driver creates. A QoS policy is specified per Kubernetes storage or bucket class configured for the VAST driver.
Note: This capability requires VAST Cluster 4.6 or later.
To view and manage QoS policies via VAST Web UI, log in and choose Element Store -> QoS Policies. For more information about VAST Cluster QoS policies, see VAST Cluster Administrator's Guide.
Create Kubernetes Secret for VAST COSI Driver
Create a Kubernetes secret to keep VMS user credentials or API token that VAST COSI Driver will use to communicate with the VAST cluster. You need to supply the name of the secret when creating the VAST COSI Driver's Helm chart configuration file.
This Kubernetes secret will be used for all bucket classes defined for the VAST COSI Driver in its Helm chart configuration file. The VAST cluster to connect is specified on the endpoint parameter in the configuration file.
Do either of the following:
Using VMS API tokens is recommended over supplying VMS user credentials.
Create Secret with VMS API Token (Recommended)
Note: VMS API tokens are supported with VAST Cluster 5.3 or later.
To create a Kubernetes secret with the VMS user's API token:
-
Create a YAML file with the following content:
apiVersion: v1 kind: Secret metadata: name: <secret name> type: Opaque data: endpoint: <VAST cluster hostname> token: <VMS user's token> sslCert: <path to SSL certificate>Where:
-
(Required)
nameis the name of the Kubernetes secret. -
(Required)
endpointsets the hostname FQDN or IP address of the VMS URL of the VAST cluster where you want to provision block volumes. -
(Required)
tokenis the VMS API token of the VMS user to connect to the VAST cluster. The token must be Base64-encoded. -
(Optional)
sslCertis only required if you are using SSL encryption with a self-signed SSL certificate. It specifies the path to the SSL certificate.
-
-
Apply the YAML file:
kubectl apply -f <path to the YAML file>Note: If you are creating the secret in a namespace that is different from the namespace used to install the driver's Helm chart, specify the secret's namespace on the command:
-n <secret's namespace>.
Alternatively, you can create a secret with the following command:
kubectl create secret generic <secret name> \
--from-literal=endpoint='vms.example.com' \
--from-literal=token='<VMS user token>' \
--from-file=ssl_cert='<path to SSL certificate>' \
-n <secret's namespace>
For example:
kubectl create secret generic vast-mgmt \
--from-literal=endpoint='vms.example.com' \
--from-literal=token='xxxxxxxxxххххххххххххххххххххххххххххххх' \
--from-file=ssl_cert='VastCerts/RootCA.crt' \
-n secret_namespace
Create Secret with VMS User Credentials
To create a Kubernetes secret with VMS user credentials:
-
Create a YAML file with the following content:
apiVersion: v1 kind: Secret metadata: name: <secret name> type: Opaque data: endpoint: <VAST cluster hostname> username: <VMS user's username> password: <VMS user's password> sslCert: <path to SSL certificate>Where:
-
(Required)
nameis the name of the Kubernetes secret. -
(Required)
endpointsets the hostname FQDN or IP address of the VMS URL of the VAST cluster where you want to provision block volumes. -
(Required)
usernameis the username of the VMS user to connect to the VAST cluster. The username must be Base64-encoded. -
(Required)
passwordis the password for the VMS user connecting to the VAST cluster. The password must be Base64-encoded. -
(Optional)
sslCertis only required if you are using SSL encryption with a self-signed SSL certificate. It specifies the path to the SSL certificate.
-
-
Apply the YAML file:
kubectl apply -f <path to the YAML file>Note: If you are creating the secret in a namespace that is different from the namespace used to install the driver's Helm chart, specify the secret's namespace on the command:
-n <secret's namespace>.
Alternatively, you can create a secret with the following command:
kubectl create secret generic <secret name> \
--from-literal=endpoint='vms.example.com' \
--from-literal=username='<VMS user's username>' \
--from-literal=password='<VMS user's password>' \
--from-file=ssl_cert='<path to SSL certificate>' \
-n <secret's namespace>
For example:
kubectl create secret generic vast-mgmt \
--from-literal=endpoint='vms.example.com' \
--from-literal=username='user1' \
--from-literal=password='xxxxxxxxx' \
--from-file=ssl_cert='VastCerts/RootCA.crt' \
-n secret_namespace
Create Helm Chart Configuration File for VAST COSI Driver
The driver's Helm chart configuration file lets you override default installation settings provided in the chart with parameters that are specific to your environment.
The configuration file is a YAML file typically named values.yaml, although you can use any arbitrary name for it.
Create Configuration File
Create a YAML file as follows (see also the example below:
Note: For a detailed reference for parameters and values, see here.
secretName: "<secret>"
endpoint: "<endpoint>"
verifySsl: true|false
sslCertsSecretName: "<SSL secret>"
bucketClassDefaults:
<option 1>
<option 1>
...
<option n>
bucketClasses:
<bucket class name 1>:
<option 1>
<option 1>
...
<option n>
<bucket class name 2>:
<option 1>
<option 1>
...
<option n>
...
<bucket class name n>:
<option 1>
<option 1>
...
<option n>
In the YAML file:
-
Set session options:
-
secretName: "<secret>"(required): Specify the Kubernetes secret with VMS user credentials to be used by the VAST COSI Driver. For more information, see Create Kubernetes Secret with VMS User Credentials. -
endpoint: "<endpoint>"(required): Enter the VAST Cluster management hostname. -
verifySsl: true|false(optional): Specifytrueto enable SSL encryption for the connection to the VAST cluster. If set tofalseor not specified, SSL encryption is disabled.Tip: When enabling SSL encryption, either upload a CA-signed SSL certificate to the VAST cluster (see the procedure in the VAST Cluster Administrator's Guide), or supply a self-signed SSL certificate to the driver. The latter can be done either via the
sslCertsSecretNameoption, or using--set-file sslCerton the Helm chart installation command. -
sslCertsSecretName: "<SSL secret>"(optional): Specify the Kubernetes secret that contains the self-signed SSL certificate to be used to secure communications between VAST COSI Driver and the VAST cluster. For more information, see Configure SSL Encryption for VAST COSI Driver.
-
-
Set bucket class options:
-
<bucket class name>(required): Provide a name to identify the bucket class.Note: Define at least one bucket class.
-
<option 1>...<option n>: Specify parameters to be used when provisioning storage for buckets with this bucket class. For information on supported options, see Bucket Class Option Reference.For each bucket class, the required options are:
bucketClasses: <bucket class name>: storagePath: "<path>" vipPool: "<virtual IP pool name>" viewPolicy: "<view policy>" create_dir: true
-
-
(Optional) Configure registration of the VAST COSI Driver with
kubelet:kubeletPath: "<your kubelet root directory>"(optional): Add this option if you are going to run VAST COSI Driver on a Kubernetes cluster where the kubelet root directory is not/var/lib/kubelet.
The following snippet shows a sample configuration file for VAST COSI Driver:
secretName: "vast-mgmt"
endpoint: "my.endpoint"
bucketClasses:
vastdata-bucket:
storagePath: "/cosi/buckets"
vipPool: "vippool-1"
viewPolicy: "cosi"
Verify Configuration File
You can verify the newly created chart configuration file by running the following command:
helm template <release name> <repo>/<chart> -f <filename>.yaml -n <namespace>
Where:
-
<release name>identifies the release being deployed. -
<repo>is the name of the VAST driver's Helm repository. -
<chart>is the name of the VAST driver's Helm chart (vastcosi). -
<filename>.yamlis the VAST driver chart configuration file. -
<namespace>(optional) determines the Kubernetes namespace to which the release is deployed. If this parameter is not specified, thedefaultnamespace is used. Otherwise, create a custom namespace prior to installing the VAST driver chart.
For example:
helm template cosi-driver vastcosi/vastcosi -f values.yaml
Add Helm Repository for VAST COSI Driver
Add the Helm repository that contains the VAST COSI Driver Helm chart to the list of available repositories:
-
Add the Helm repository for VAST COSI Driver:
helm repo add <repo> https://vast-data.github.io/vast-csiSpecify any suitable name for
repo. This name will be used to refer to the VAST COS Driver repository when running Helm commands. For example: vastcosi -
Verify that the repository has been added:
helm repo listThe output is similar to the following:
NAME URL vastcosi https://vast-data.github.io/vast-csi
Install VAST COSI Driver Helm Chart
Installing a Helm chart results in deployment of a VAST driver's release in your Kubernetes environment. A release is identified with its release name, which you supply during the install.
To install the VAST COSI Driver chart:
-
Refresh Helm repository information:
helm repo update -
Run the following command to initiate the install:
helm install <release name> <repo>/<chart> -f <filename>.yaml -n <namespace> [--set-file sslCert=VastCerts/RootCA.crt]Where:
-
<release name>identifies the release being deployed. -
<repo>is the name of the VAST COSI Driver's Helm repository. -
<chart>is the name of the Helm chart to be installed (vastcosi). -
<filename>.yamlis the Helm chart configuration file for VAST COSI Driver. -
<namespace>(optional) determines the Kubernetes namespace to which the release is deployed. If this parameter is not specified, thedefaultnamespace is used. Otherwise, create a custom namespace prior to installing the Helm chart. -
--set-file sslCert=VastCerts/RootCA.crt(optional) specifies the path to a self-signed SSL certificate to secure the connection to the VAST cluster.
For example:
helm install cosi-driver vastcosi/vastcosi -f values.yamlThe output is similar to the following:
NAME: cosi-driver LAST DEPLOYED: Thu Dec 5 05:24:36 2024 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: Thank you for installing vastcosi. Your release is named cosi-driver. The release is installed in namespace default To learn more about the release, try: $ helm status -n default cosi-driver $ helm get all -n default cosi-driver <...> -
-
Verify that the Helm chart has been installed as follows:
-
Check the release status with the following command:
helm status -n <namespace> <release name>For example:
helm status -n default cosi-driver -
Ensure that the release appears in the list of releases:
helm list -n <namespace>For example:
helm list -n defaultThe output is similar to the following:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION cosi-driver default 1 2024-12-01 04:25:33.783236165 +0000 UTC deployed vastcosi-0.1.0 2.6.0
-
Verify VAST COSI Driver Deployment
To verify your VAST COSI Driver deployment, provision a bucket on the VAST cluster and prepare credentials for accessing the bucket:
-
Create a Kubernetes YAML configuration file that defines bucket access:
kind: BucketAccessClass apiVersion: objectstorage.k8s.io/v1alpha1 metadata: name: sample-bac driverName: csi.vastdata.com authenticationType: KEY --- kind: BucketAccess apiVersion: objectstorage.k8s.io/v1alpha1 metadata: name: sample-access spec: bucketClaimName: sample-bucket bucketAccessClassName: sample-bac credentialsSecretName: my-super-secret --- kind: BucketClaim apiVersion: objectstorage.k8s.io/v1alpha1 metadata: name: sample-bucket spec: bucketClassName: vastdata-bucket protocols: - s3 -
Apply the bucket access configuration file:
kubectl apply -f <filename>.yaml -
Verify that a newly created bucket
vastdata-bucketis displayed as a view in the VAST Cluster's Web UI. -
Verify that the newly created user named
vastdata-bucketis displayed in the VAST Cluster's Web UI. -
Verify that a newly created Kubernetes secret
my-super-secretis listed in the output of the following command:kubectl get secret -
Create a Kubernetes YAML configuration file for a pod that uses the newly created Kubernetes secret:
apiVersion: v1 kind: Pod metadata: name: awscli spec: containers: - name: awscli image: amazon/aws-cli command: ["sleep"] args: ["9999999999"] volumeMounts: - name: cosi-secrets mountPath: /data/cosi readOnly: true volumes: - name: cosi-secrets secret: secretName: my-super-secret -
Apply the pod configuration file:
kubectl apply -f <filename>.yaml -
Verify the content of the Kubernetes secret:
kubectl exec -it awscli cat /data/cosi/BucketInfoThe JSON output lists the VAST Cluster endpoint, the access key and the secret key, which can be used to access the newly created bucket.
Configure SSL Encryption for VAST COSI Driver
You can secure the connection between VAST COSI Driver and the VAST cluster with SSL encryption as follows:
-
Enable SSL encryption.
-
Do one of the following to install an SSL certificate:
-
If you want to use a Certified Authority-signed SSL certificate, upload it to the VAST cluster. Follow the SSL certificate upload procedure provided in the VAST Cluster Administrator's Guide.
-
If you want to use a self-signed SSL certificate, supply it to the VAST driver.
-
Enable SSL Encryption
By default, SSL encryption is disabled.
To enable SSL encryption:
-
Add the
verifySsl=trueoption to the VAST COSI Driver chart configuration file, for example:secretName: "vast-mgmt" endpoint: "my.endpoint" verifySsl: true <...>
Upload CA-Signed SSL Certificate to VAST Cluster
Follow the guidelines provided in the VAST Cluster Administrator's Guide to upload a CA-signed SSL certificate to the VAST cluster.
Supply Self-Signed SSL Certificate to VAST COSI Driver
You can either point to a file that contains a self-signed SSL certificate file, or specify an existing Kubernetes secret that contains the certificate. These two methods are mutually exclusive.
Do either of the following:
-
Install or upgrade the VAST COSI Driver Helm chart with the
--set-file sslCert=<path to certificate file>option specified, for example:helm install cosi-driver vastcosi/vastcosi -f values.yaml --set-file sslCert=<path to certificate file>
OR
-
Create a Kubernetes secret with the SSL certificate and specify the secret using the
sslCertsSecretNameoption in the VAST COSI Driver Helm chart configuration file:-
Create a Kubernetes secret that contains the SSL certificate, for example:
kubectl create secret generic vast-ca --from-file=ca-bundle.crt=<path to certificate file> -
Specify the newly created secret on the
sslCertsSecretNameoption in the configuration file, for example:secretName: "vast-mgmt" endpoint: "my.endpoint" verifySsl: true sslCertsSecretName: "vast-ca" <...> -
Install or upgrade the VAST driver Helm chart (without specifying
--set-file sslCert).
-
Replace Self-Signed SSL Certificate for VAST COSI Driver
Choose either of the following, depending on how you supplied the old self-signed SSL certificate:
-
If you supplied the old SSL certificate using the
--set-file sslCertoption on the Helm chart install or upgrade command:-
Upgrade the Helm chart with
--set-file sslCertpointing to the new SSL certificate file. For example:helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml --set-file sslCert=<path to new certificate>
-
OR
-
If the old SSL certificate was supplied via
sslCertsSecretNamein the VAST COSI Driver Helm chart configuration file:-
Create a new Kubernetes secret with the new SSL certificate:
kubectl create secret generic vast-ca-new --from-file=ca-bundle.crt=<path to new certificate file> -
Ensure that the new SSL certificate is specified on the
sslCertsSecretNameoption in the VAST COSI Driver Helm chart configuration file:secretName: "vast-mgmt" endpoint: "my.endpoint" verifySsl: true sslCertsSecretName: "vast-ca-new" <...> -
Upgrade the Helm chart, for example:
helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml
-
Remove Self-Signed SSL Certificate from VAST COSI Driver
Choose either of the following, depending on how you supplied the self-signed SSL certificate:
-
If you used
--set-file sslCertto supply the SSL certificate:-
Upgrade the Helm chart without the
--set-file sslCertoption specified. For example:helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml
-
OR
-
If the old SSL certificate was supplied via
sslCertsSecretName:-
Remove the
sslCertsSecretNameoption from the VAST COSI Driver Helm chart configuration file. -
Upgrade the Helm chart (without specifying
--set-file sslCert), for example:helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml
-