Documentation Index

Fetch the complete documentation index at: https://kb.vastdata.com/llms.txt

Use this file to discover all available pages before exploring further.

Deploy VAST COSI Driver

Prev Next

Use the following method to deploy VAST COSI Driver:

VAST COSI Driver Requirements

Ensure that your environment meets the following requirements:

  • A Kubernetes cluster is up and running.

  • A VAST cluster is up and running.

  • All nodes of the Kubernetes cluster are networked with the VAST cluster.

  • At least one node can communicate with the VAST Cluster management virtual IP.

  • A host is available with a Helm client installed, preferably in the VMS network.

Supported Versions

VAST COSI Driver Kubernetes VAST Cluster Helm
2.6.x 1.25 - 1.31.1 4.7.0-SP6 or later with the Trash Folder Access feature enabled 3.x.x

Required Permissions

VAST COSI Driver requires root (sysadmin) privileges.

Deploy VAST COSI Driver with Helm Chart

Before you begin, ensure that your environment meets the requirements.

VAST COSI Driver is deployed using Helm charts. A Helm chart is an installation template that can be reused to install multiple instances of the software being deployed. Each instance is referred to as a release. Helm charts are available from Helm repositories.

Steps to deploy VAST COSI Driver include:

  1. (Optional) Install CustomResourceDefinitions for COSI.

  2. Install CustomResourceDefinitions for snapshots.

  3. (Optional) Create a Kubernetes namespace for VAST COSI Driver.

    Note: This step is required if you are going to deploy VAST COSI Driver in a Kubernetes namespace other than default. Otherwise, skip to step 4.

  4. Configure the VAST cluster.

  5. Create a Kubernetes secret with VMS user credentials or API token for VAST COSI Driver.

  6. Add the Helm repository that contains the VAST COSI Driver chart.

  7. Create a Helm chart configuration file for VAST COSI Driver.

  8. Install the Helm chart for VAST COSI Driver.

  9. (Optional) Verify the deployment by launching a test application.

Install CRDs for COSI

Run the following commands to install the Custom Resource Definitions (CRDs) for COSI:

kubectl create -k github.com/kubernetes-sigs/container-object-storage-interface-api
kubectl create -k github.com/kubernetes-sigs/container-object-storage-interface-controller

Install CRDs for VAST Snapshots

Complete this step if you are going to use VAST snapshots on your Kubernetes cluster. The Custom Resource Definitions (CRDs) installed in this step are only required for using VAST snapshots.

Run the following commands to install the CRDs for snapshots:

kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v6.0.1/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml

Create Kubernetes Namespace for VAST COSI Driver

By default, VAST COSI Driver is deployed to the default namespace on the Kubernetes cluster.

If you want to use a different Kubernetes namespace for VAST COSI Driver, create it prior to deployment by running the following command:

kubectl create ns <namespace_name>

Configure VAST Cluster for VAST COSI Driver

Complete the following steps to make your VAST cluster ready for integration with Kubernetes through the use of VAST COSI Driver:

  • Set up virtual IP pools to be used by VAST COSI Driver.

  • Configure a view policy to be used for views created by VAST COSI Driver.

  • Set up a VMS Manager user to be used by VAST COSI Driver.

  • Enable trash folder access on the VAST cluster.

  • (Optional) Upload your CA-signed SSL certificate to the VAST cluster.

  • (Optional): Configure a QoS policy to be associated with views created by VAST COSI Driver.

Set Up Virtual IP Pools

VAST COSI Driver distributes the load among virtual IPs in one or more VAST virtual IP pools.

The virtual IP pool is specified in the Kubernetes bucket class defined in the VAST COSI Driver chart configuration file. The virtual IP pool specified for a bucket class is used when processing read and write operations requested by the application that is using that particular bucket class

To view and manage virtual IP pools in VAST Web UI, log in and choose Network Access -> Virtual IP Pools in the main navigation menu. For more information about VAST Cluster virtual IP pools, see VAST Cluster Administrator's Guide.

Configure View Policies

VAST COSI Driver automatically creates a VAST Cluster view for each storage claim being provisioned. These views are controlled using VAST Cluster view policies.

The view policy is specified in the Kubernetes bucket class defined in the VAST COSI Driver chart configuration file. The view policy specified for a bucket class is used when processing read and write operations requested by the application that is using that particular bucket class.

To view and manage existing view policies in VAST Web UI, log in and choose Element Store -> View Policies. For more information about VAST Cluster view policies, see VAST Cluster Administrator's Guide.

Set Up VMS User

Set up a VMS user for VAST COSI Driver to communicate with the VAST Management Service (VMS) via VAST REST API.

You'll need to supply the VMS user credentials or authentication token in a Kubernetes secret that is specified when creating the VAST COSI Driver's Helm chart configuration file.

VAST COSI Driver can use the default admin user that is created by VAST during the initial install, or you can create a new VMS user. The user does not need to have full VMS permissions. The required permissions are Create, View, Edit and Delete permissions in the Security and Logical realm.

Permissions are granted to a VMS user by assigning a role. To view and manage roles in VAST Web UI, log in and choose Administrators -> Roles in the main navigation menu. For more information about managing roles, see VAST Cluster Administrator's Guide.

To view and manage VMS users in VAST Web UI, log in and choose Administrators -> Managers in the main navigation menu. For more information about managing VMS Manager users, see VAST Cluster Administrator's Guide.

To view and manage VMS authentication tokens, use apitoken * commands of VAST CLI or VAST REST API.

Enable Trash Folder Access

Trash folder access is required to allow for COSI cleanup.

To enable trash folder access in VAST Web UI, log in and choose Settings -> Cluster. In the General tab of cluster settings, toggle the Trash folder access option on.

Upload CA-Signed SSL Certificate to VAST Cluster

If you want to use a Certificate Authority-signed SSL certificate to secure the connection to the VAST cluster, follow the SSL certificate upload procedure in the VAST Cluster Administrator's Guide to upload your SSL certificate to the VAST cluster.

For more information about configuring SSL encryption for VAST COSI Driver, see Configure SSL Encryption for VAST COSI Driver.

Configure QoS Policy

You can optionally set up a Quality of Service (QoS) policy to be associated with the views that VAST COSI Driver creates. A QoS policy is specified per Kubernetes storage or bucket class configured for the VAST driver.

Note: This capability requires VAST Cluster 4.6 or later.

To view and manage QoS policies via VAST Web UI, log in and choose Element Store -> QoS Policies. For more information about VAST Cluster QoS policies, see VAST Cluster Administrator's Guide.

Create Kubernetes Secret for VAST COSI Driver

Create a Kubernetes secret to keep VMS user credentials or API token that VAST COSI Driver will use to communicate with the VAST cluster. You need to supply the name of the secret when creating the VAST COSI Driver's Helm chart configuration file.

This Kubernetes secret will be used for all bucket classes defined for the VAST COSI Driver in its Helm chart configuration file. The VAST cluster to connect is specified on the endpoint parameter in the configuration file.

Do either of the following:

  • Create a secret with a VMS API token, or

  • create a secret with VMS user credentials.

Using VMS API tokens is recommended over supplying VMS user credentials.

Create Secret with VMS API Token (Recommended)

Note: VMS API tokens are supported with VAST Cluster 5.3 or later.

To create a Kubernetes secret with the VMS user's API token:

  1. Create a YAML file with the following content:

    apiVersion: v1
    kind: Secret
    metadata:
      name: <secret name>
    type: Opaque
    data:
      endpoint: <VAST cluster hostname>
      token: <VMS user's token>
      sslCert: <path to SSL certificate>
    

    Where:

    • (Required) name is the name of the Kubernetes secret.

    • (Required) endpoint sets the hostname FQDN or IP address of the VMS URL of the VAST cluster where you want to provision block volumes.

    • (Required) token is the VMS API token of the VMS user to connect to the VAST cluster. The token must be Base64-encoded.

    • (Optional) sslCert is only required if you are using SSL encryption with a self-signed SSL certificate. It specifies the path to the SSL certificate.

  2. Apply the YAML file:

    kubectl apply -f <path to the YAML file>
    

    Note: If you are creating the secret in a namespace that is different from the namespace used to install the driver's Helm chart, specify the secret's namespace on the command: -n <secret's namespace>.

Alternatively, you can create a secret with the following command:

kubectl create secret generic <secret name> \
    --from-literal=endpoint='vms.example.com' \
    --from-literal=token='<VMS user token>' \
    --from-file=ssl_cert='<path to SSL certificate>' \
    -n <secret's namespace>

For example:

kubectl create secret generic vast-mgmt \
    --from-literal=endpoint='vms.example.com' \
    --from-literal=token='xxxxxxxxxххххххххххххххххххххххххххххххх' \
    --from-file=ssl_cert='VastCerts/RootCA.crt' \
    -n secret_namespace

Create Secret with VMS User Credentials

To create a Kubernetes secret with VMS user credentials:

  1. Create a YAML file with the following content:

    apiVersion: v1
    kind: Secret
    metadata:
      name: <secret name>
    type: Opaque
    data:
      endpoint: <VAST cluster hostname>
      username: <VMS user's username>
      password: <VMS user's password>
      sslCert: <path to SSL certificate>
    

    Where:

    • (Required) name is the name of the Kubernetes secret.

    • (Required) endpoint sets the hostname FQDN or IP address of the VMS URL of the VAST cluster where you want to provision block volumes.

    • (Required) username is the username of the VMS user to connect to the VAST cluster. The username must be Base64-encoded.

    • (Required) password is the password for the VMS user connecting to the VAST cluster. The password must be Base64-encoded.

    • (Optional) sslCert is only required if you are using SSL encryption with a self-signed SSL certificate. It specifies the path to the SSL certificate.

  2. Apply the YAML file:

    kubectl apply -f <path to the YAML file>
    

    Note: If you are creating the secret in a namespace that is different from the namespace used to install the driver's Helm chart, specify the secret's namespace on the command: -n <secret's namespace>.

Alternatively, you can create a secret with the following command:

kubectl create secret generic <secret name> \
    --from-literal=endpoint='vms.example.com' \
    --from-literal=username='<VMS user's username>' \
    --from-literal=password='<VMS user's password>' \
    --from-file=ssl_cert='<path to SSL certificate>' \
    -n <secret's namespace>

For example:

kubectl create secret generic vast-mgmt \
    --from-literal=endpoint='vms.example.com' \
    --from-literal=username='user1' \
    --from-literal=password='xxxxxxxxx' \
    --from-file=ssl_cert='VastCerts/RootCA.crt' \
    -n secret_namespace

Create Helm Chart Configuration File for VAST COSI Driver

The driver's Helm chart configuration file lets you override default installation settings provided in the chart with parameters that are specific to your environment.

The configuration file is a YAML file typically named values.yaml, although you can use any arbitrary name for it.

Create Configuration File

Create a YAML file as follows (see also the example below:

Note: For a detailed reference for parameters and values, see here.

secretName: "<secret>"
endpoint: "<endpoint>"
verifySsl: true|false
sslCertsSecretName: "<SSL secret>"

bucketClassDefaults:
    <option 1>
    <option 1>
    ...
    <option n>

bucketClasses:
  <bucket class name 1>:
    <option 1>
    <option 1>
    ...
    <option n>
  <bucket class name 2>:
    <option 1>
    <option 1>
    ...
    <option n>
 ...
  <bucket class name n>:
    <option 1>
    <option 1>
    ...
    <option n>

In the YAML file:

  1. Set session options:

    • secretName: "<secret>" (required): Specify the Kubernetes secret with VMS user credentials to be used by the VAST COSI Driver. For more information, see Create Kubernetes Secret with VMS User Credentials.

    • endpoint: "<endpoint>" (required): Enter the VAST Cluster management hostname.

    • verifySsl: true|false (optional): Specify true to enable SSL encryption for the connection to the VAST cluster. If set to false or not specified, SSL encryption is disabled.

      Tip: When enabling SSL encryption, either upload a CA-signed SSL certificate to the VAST cluster (see the procedure in the VAST Cluster Administrator's Guide), or supply a self-signed SSL certificate to the driver. The latter can be done either via the sslCertsSecretName option, or using --set-file sslCert on the Helm chart installation command.

    • sslCertsSecretName: "<SSL secret>" (optional): Specify the Kubernetes secret that contains the self-signed SSL certificate to be used to secure communications between VAST COSI Driver and the VAST cluster. For more information, see Configure SSL Encryption for VAST COSI Driver.

  2. Set bucket class options:

    • <bucket class name> (required): Provide a name to identify the bucket class.

      Note: Define at least one bucket class.

    • <option 1>...<option n>: Specify parameters to be used when provisioning storage for buckets with this bucket class. For information on supported options, see Bucket Class Option Reference.

      For each bucket class, the required options are:

      bucketClasses:
        <bucket class name>:
          storagePath: "<path>"
          vipPool: "<virtual IP pool name>"
          viewPolicy: "<view policy>"
          create_dir: true
      
  3. (Optional) Configure registration of the VAST COSI Driver with kubelet:

    • kubeletPath: "<your kubelet root directory>" (optional): Add this option if you are going to run VAST COSI Driver on a Kubernetes cluster where the kubelet root directory is not /var/lib/kubelet.

The following snippet shows a sample configuration file for VAST COSI Driver:

secretName: "vast-mgmt"
endpoint: "my.endpoint"

bucketClasses:
  vastdata-bucket:
    storagePath: "/cosi/buckets"    
    vipPool: "vippool-1"
    viewPolicy: "cosi"

Verify Configuration File

You can verify the newly created chart configuration file by running the following command:

helm template <release name> <repo>/<chart> -f <filename>.yaml -n <namespace>

Where:

  • <release name> identifies the release being deployed.

  • <repo> is the name of the VAST driver's Helm repository.

  • <chart> is the name of the VAST driver's Helm chart (vastcosi).

  • <filename>.yaml is the VAST driver chart configuration file.

  • <namespace> (optional) determines the Kubernetes namespace to which the release is deployed. If this parameter is not specified, the default namespace is used. Otherwise, create a custom namespace prior to installing the VAST driver chart.

For example:

helm template cosi-driver vastcosi/vastcosi -f values.yaml

Add Helm Repository for VAST COSI Driver

Add the Helm repository that contains the VAST COSI Driver Helm chart to the list of available repositories:

  1. Add the Helm repository for VAST COSI Driver:

    helm repo add <repo> https://vast-data.github.io/vast-csi
    

    Specify any suitable name for repo. This name will be used to refer to the VAST COS Driver repository when running Helm commands. For example: vastcosi

  2. Verify that the repository has been added:

    helm repo list
    

    The output is similar to the following:

    NAME     URL 
    vastcosi https://vast-data.github.io/vast-csi
    

Install VAST COSI Driver Helm Chart

Installing a Helm chart results in deployment of a VAST driver's release in your Kubernetes environment. A release is identified with its release name, which you supply during the install.

To install the VAST COSI Driver chart:

  1. Refresh Helm repository information:

    helm repo update
    
  2. Run the following command to initiate the install:

    helm install <release name> <repo>/<chart> -f <filename>.yaml -n <namespace> [--set-file sslCert=VastCerts/RootCA.crt]
    

    Where:

    • <release name> identifies the release being deployed.

    • <repo> is the name of the VAST COSI Driver's Helm repository.

    • <chart> is the name of the Helm chart to be installed (vastcosi).

    • <filename>.yaml is the Helm chart configuration file for VAST COSI Driver.

    • <namespace> (optional) determines the Kubernetes namespace to which the release is deployed. If this parameter is not specified, the default namespace is used. Otherwise, create a custom namespace prior to installing the Helm chart.

    • --set-file sslCert=VastCerts/RootCA.crt (optional) specifies the path to a self-signed SSL certificate to secure the connection to the VAST cluster.

    For example:

    helm install cosi-driver vastcosi/vastcosi -f values.yaml
    

    The output is similar to the following:

    NAME: cosi-driver
    LAST DEPLOYED: Thu Dec  5 05:24:36 2024
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES:
    Thank you for installing vastcosi.
    
    Your release is named cosi-driver.
    The release is installed in namespace default
    
    To learn more about the release, try:
    
      $ helm status -n default cosi-driver
      $ helm get all -n default cosi-driver
    
    <...>
    
  3. Verify that the Helm chart has been installed as follows:

    • Check the release status with the following command:

      helm status -n <namespace> <release name>
      

      For example:

      helm status -n default cosi-driver
      
    • Ensure that the release appears in the list of releases:

      helm list -n <namespace>
      

      For example:

      helm list -n default
      

      The output is similar to the following:

      NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
      cosi-driver     default         1               2024-12-01 04:25:33.783236165 +0000 UTC deployed        vastcosi-0.1.0  2.6.0    
      

Verify VAST COSI Driver Deployment

To verify your VAST COSI Driver deployment, provision a bucket on the VAST cluster and prepare credentials for accessing the bucket:

  1. Create a Kubernetes YAML configuration file that defines bucket access:

    kind: BucketAccessClass
    apiVersion: objectstorage.k8s.io/v1alpha1
    metadata:
     name: sample-bac
    driverName: csi.vastdata.com
    authenticationType: KEY
    ---
    kind: BucketAccess
    apiVersion: objectstorage.k8s.io/v1alpha1
    metadata:
     name: sample-access
    spec:
     bucketClaimName: sample-bucket
     bucketAccessClassName: sample-bac
     credentialsSecretName: my-super-secret
    ---
    kind: BucketClaim
    apiVersion: objectstorage.k8s.io/v1alpha1
    metadata:
     name: sample-bucket
    spec:
     bucketClassName: vastdata-bucket
     protocols:
     - s3
    
  2. Apply the bucket access configuration file:

    kubectl apply -f <filename>.yaml
    
  3. Verify that a newly created bucket vastdata-bucket is displayed as a view in the VAST Cluster's Web UI.

  4. Verify that the newly created user named vastdata-bucket is displayed in the VAST Cluster's Web UI.

  5. Verify that a newly created Kubernetes secret my-super-secret is listed in the output of the following command:

    kubectl get secret
    
  6. Create a Kubernetes YAML configuration file for a pod that uses the newly created Kubernetes secret:

    apiVersion: v1
    kind: Pod
    metadata:
     name: awscli
    spec:
     containers:
       - name: awscli
         image: amazon/aws-cli
         command: ["sleep"]
         args: ["9999999999"]
         volumeMounts:
           - name: cosi-secrets
             mountPath: /data/cosi
             readOnly: true
     volumes:
     - name: cosi-secrets
       secret:
         secretName: my-super-secret
    
  7. Apply the pod configuration file:

    kubectl apply -f <filename>.yaml
    
  8. Verify the content of the Kubernetes secret:

    kubectl exec -it awscli cat /data/cosi/BucketInfo
    

    The JSON output lists the VAST Cluster endpoint, the access key and the secret key, which can be used to access the newly created bucket.

Configure SSL Encryption for VAST COSI Driver

You can secure the connection between VAST COSI Driver and the VAST cluster with SSL encryption as follows:

  1. Enable SSL encryption.

  2. Do one of the following to install an SSL certificate:

    • If you want to use a Certified Authority-signed SSL certificate, upload it to the VAST cluster. Follow the SSL certificate upload procedure provided in the VAST Cluster Administrator's Guide.

    • If you want to use a self-signed SSL certificate, supply it to the VAST driver.

Enable SSL Encryption

By default, SSL encryption is disabled.

To enable SSL encryption:

  1. Add the verifySsl=true option to the VAST COSI Driver chart configuration file, for example:

    secretName: "vast-mgmt"
    endpoint: "my.endpoint"
    verifySsl: true
    <...>
    
  2. Install or upgrade the VAST COSI Driver Helm chart.

Upload CA-Signed SSL Certificate to VAST Cluster

Follow the guidelines provided in the VAST Cluster Administrator's Guide to upload a CA-signed SSL certificate to the VAST cluster.

Supply Self-Signed SSL Certificate to VAST COSI Driver

You can either point to a file that contains a self-signed SSL certificate file, or specify an existing Kubernetes secret that contains the certificate. These two methods are mutually exclusive.

Do either of the following:

  • Install or upgrade the VAST COSI Driver Helm chart with the --set-file sslCert=<path to certificate file> option specified, for example:

    helm install cosi-driver vastcosi/vastcosi -f values.yaml --set-file sslCert=<path to certificate file>
    

OR

  • Create a Kubernetes secret with the SSL certificate and specify the secret using the sslCertsSecretName option in the VAST COSI Driver Helm chart configuration file:

    1. Create a Kubernetes secret that contains the SSL certificate, for example:

      kubectl create secret generic vast-ca --from-file=ca-bundle.crt=<path to certificate file>
      
    2. Specify the newly created secret on the sslCertsSecretName option in the configuration file, for example:

      secretName: "vast-mgmt"
      endpoint: "my.endpoint"
      verifySsl: true
      sslCertsSecretName: "vast-ca"
      <...>
      
    3. Install or upgrade the VAST driver Helm chart (without specifying --set-file sslCert).

Replace Self-Signed SSL Certificate for VAST COSI Driver

Choose either of the following, depending on how you supplied the old self-signed SSL certificate:

  • If you supplied the old SSL certificate using the --set-file sslCert option on the Helm chart install or upgrade command:

    • Upgrade the Helm chart with --set-file sslCert pointing to the new SSL certificate file. For example:

      helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml --set-file sslCert=<path to new certificate>
      

OR

  • If the old SSL certificate was supplied via sslCertsSecretName in the VAST COSI Driver Helm chart configuration file:

    1. Create a new Kubernetes secret with the new SSL certificate:

      kubectl create secret generic vast-ca-new --from-file=ca-bundle.crt=<path to new certificate file>
      
    2. Ensure that the new SSL certificate is specified on the sslCertsSecretName option in the VAST COSI Driver Helm chart configuration file:

      secretName: "vast-mgmt"
      endpoint: "my.endpoint"
      verifySsl: true
      sslCertsSecretName: "vast-ca-new"
      <...>
      
    3. Upgrade the Helm chart, for example:

      helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml
      

Remove Self-Signed SSL Certificate from VAST COSI Driver

Choose either of the following, depending on how you supplied the self-signed SSL certificate:

  • If you used --set-file sslCert to supply the SSL certificate:

    • Upgrade the Helm chart without the --set-file sslCert option specified. For example:

      helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml
      

OR

  • If the old SSL certificate was supplied via sslCertsSecretName:

    1. Remove the sslCertsSecretName option from the VAST COSI Driver Helm chart configuration file.

    2. Upgrade the Helm chart (without specifying --set-file sslCert), for example:

      helm upgrade cosi-driver vastcosi/vastcosi -f values.yaml