PVC Timeout due to Missing NFS Client on Kubernetes Worker Node
If a PVC does not mount with a timeout, a likely outcome is the Kubernetes worker node is missing the NFS client.
Run the kubectl logs csi-vast-node-<NODE_ID> --namespace vast-csi -c csi-vast-plugin command to check the node logs.
The output can be similar to the following:
Warning FailedMount 50s kubelet, k8s-a-node03 Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[mariadb-credentials default-token-zpkp5 data config]: timed out waiting for the condition
Warning FailedMount 37s kubelet, k8s-a-node03 MountVolume.SetUp failed for volume "pvc-4ebe1936-3613-4c37-957c-4d21e623aa98" : kubernetes.io/csi: mounter.SetupAt failed: rpc error: code = DeadlineExceeded desc = context deadline exceededTo resolve the problem, install the NFS client and test mounts locally on the client, then retry via Kubernetes.
To install the NFS client:
sudo yum -y install nfs-utilsPVC Timeout due to No Communication with VMS
Kubernetes nodes can be on the VAST data network and not on the VAST management network. In this case, the return path from VMS to the Kubernetes node will fail.
Ensure that you have valid routes on the correct network.
On the Kubernetes node, run the following command and verify that the correct interface is used:
ip route get <vms_ip> Run this command on the VMS node:
ip route get <k8s_node_ip>