VAST with Kubernetes

Overview

Kubernetes is a popular and successful orchestration framework. By using Kubernetes, one can deploy and manage large numbers of containers within an enterprise. These containers, of course, contain applications that execute whatever is needed. Kubernetes does a great job of managing those containers, but it does not provide for the persistence of data within those containers.

When a container runs in Kubernetes, it can request volumes or persistent volumes from the Kubernetes framework. In the simplest case, those volumes are stored on the local host. The problem with that is that if the container is moved to another host, the stored data is lost. Additionally, two containers running on different hosts cannot see data that is stored locally on one host. To enable shared and reliable persistent storage, Kubernetes allows for customizable persistent storage. Kubernetes applications need a persistent store that can scale well, is very reliable, is very fast, and can be accessed remotely by any container. VAST is one such store.

In this article, we'll explain how to use VAST as a persistent store for applications running in Kubernetes. We show two techniques. In the first part, we show using VAST simply as an NFS file server, which Kubernetes natively supports. Then, in the second part, we show how to leverage the VAST CSI plugin for deeper Kubernetes integration.

For complete information about the VAST CSI plugin, see the CSI Driver Administrator's Guide.

You may be interested in our webinar that covers our Kubernetes work.

Volumes

Kubernetes Volumes allow a container (via its YAML specification) to state what persistent storage it wishes to use. Kubernetes supports a variety of volume types, including an NFS volume. Conveniently, VAST supports NFS, and thus one can easily reference an existing VAST cluster using standard Kubernetes constructs.

Here's an example of a trivial pod that uses an existing VAST cluster as persistent storage. In advance, we did the following:

Created a DNS round robin entry (not in the Kubernetes DNS server as the NFS Kubernetes clients cannot see the Kubernetes DNS, only the host DNS).
Created a VAST export for use with Kubernetes - named /k8s.

The application prints a date to a file within VAST, demonstrating proper connectivity. This is the complete YAML file for that application.

# Create a pod that reads and writes to the
# VAST NFS server via an NFS volume.

kind: Pod
apiVersion: v1
metadata:
  name: pod-using-nfs
spec:
  # Add the server as an NFS volume for the pod
  volumes:
    - name: nfs-volume
      nfs:
        # URL for the NFS server. The DNS name must be defined at the OS level, not within K8S DNS.
        server: vastvips
        path: /k8s

  # In this container, we'll mount the NFS volume
  # and write the date to a file inside it.
  containers:
    - name: app
      image: centos

      # Mount the NFS volume in the container
      volumeMounts:
        - name: nfs-volume
          mountPath: /var/nfs

      # Write to a file inside our NFS
      command: ["/bin/sh"]
      args: ["-c", "while true; do date >> /var/nfs/nfsdirect-dates.txt; sleep 5; done"]

To deploy this using kubectl, create the file above and then run:

kubectl apply -f pod-using-nfs.yaml

That's more than sufficient for many applications, but in some Kubernetes environments, the administrators prefer to better abstract storage from the applications by using Persistent Volumes and Persistent Volume Claims. As you likely noticed, the YAML file above for the pod included very specific environmental details - the DNS name and mount point for VAST. It is advantageous for applications to simply state "I need storage," and Kubernetes can then provide it as needed. That is done via the higher-level abstraction of Persistent Volumes and Persistent Volume Claims. Essentially, an administrator creates a Persistent Volume in advance (the information looks a lot like the information above) and then applications express their need by using a Persistent Volume Claim. We aren't going to cover that as it's a rather trivial extension of the previous example.

A much more interesting enhancement is to move from statically creating Persistent Volumes to Dynamic Provisioning. This frees the administrator from having to create all those container-specific Persistent Volumes. Keep in mind that if your goal is to expose existing storage on VAST to both Kubernetes containerized applications and non-Kubernetes applications, dynamic provisioning is not a good approach. Dynamic provisioning is optimized for easily creating storage dynamically that is accessible only from the Kubernetes containerized environment.

Dynamic Provisioning

With dynamic provisioning, applications still specify Persistent Volume Claims, but instead of the Kubernetes administrator creating the volumes, the administrator deploys in advance the elements needed for Dynamic Provisioning, specifically these key components:

StorageClass - a YAML class definition that specifies the dynamic provisioner as well as some volume-related options
Dynamic provisioner - in this case, a Container Storage Interface (CSI) aware provisioner

The linkage between the StorageClass and the provisioner is what enables dynamic provisioning. When a Persistent Storage Claim is made, the Kubernetes framework will use the provisioner to dynamically create a Persistent Volume. Later, when a container (via its YAML) references a Persistent Storage Claim, Kubernetes will provide the volume to the container. This is a bit of a simplification, and Persistent Volumes have lifecycle rules - Persistent Volume lifecycle behaviors are controlled by the Kubernetes framework and outside the scope of this article.

Kubernetes has a few built-in provisioners, but also provides for CSI-based provisioning, which is the focus of the remainder of this article.

Dynamically Provisioning Volumes Using VAST's CSI

With the introduction of CSI, containers can obtain dynamically provisioned storage exposed by VAST systems. VAST's CSI allows CO (container orchestrators) such as Kubernetes to provision storage volumes from a VAST system. The VAST CSI Driver Administrator's Guide explains how to install the plugin on a Kubernetes cluster.

Closing

You've seen how VAST can be used easily with Kubernetes to provide dynamically provisioned Persistent Volumes. Since VAST provides a highly scalable, fast, and remotely accessible data store, it is a great choice for use with Kubernetes.

We are very interested in feedback on our Kubernetes support. Please contact your VAST representative to let us know how well this works for you and what enhancements would be useful for a productized version.