Installing VAST Data software in Base Command Manager

  • Published on Jan 24, 2026
  • 5 minute(s) read
Prev Next

Summary:

This document describes the procedure for leveraging the VAST Data software stack, specifically the VAST-NFS client and relevant mount options, with the NVIDIA Base Command Manager (formerly Bright Cluster Manager, BCM for short). The instructions in this document use a DGX A100 BCM software image, but the same procedure applies to any BCM image.

 Procedure:

Step 1: Prepare a copy of the software image

This example uses the dgxa100-image software image as the base image. Any other software image with NVIDIA MOFED installed can be used for this purpose.

Start by logging into the BCM Head node.

Clone the software image to an image called dgxa100-vast-image.

# Create a clone of the Bright software image you want to update
$ cmsh
  softwareimage
  clone dgxa100-image dgxa100-vast-image
  commit

  exit

Step 2: Assign a new image to the node(s) or category

Define a new node. say dgx-a100 and assign the prepared dgx-image from step 1 to it. After the initial ramdisk has been generated, provision the node.

In the example below, we will set the software image for an individual node, but if you have many nodes it makes more sense to create a node category, make your nodes part of this category, and set the software image for the category (which will then be inherited by all nodes in the category).

$ cmsh
  device use dgx-a100;
  set softwareimage dgxa100-vast-image;
  commit

  # Wait for ramdisk to be generated
  # It can take several minutes for the Initial ramdisk creation to start
  # Wait for the message "... image ... was generated successfully"
  # if you dont see it, run this command :
  # createramdisk dgxa100-vast-image -w
  
  # Please note that its important that a ramdisk is created for each individual node as well.
  # Something like 'Initial ramdisk for node dgx-a100 based on image dgxa100-vast-image was generated successfully'
  # We will do this again in Step 8.

Step 3: Provision the node

Boot up the node dgx-a100 and confirm that the correct image (from step 2) is installed.

Step 4: Install the VAST software client

The steps in this section must be performed on the node provisioned in Step 3. The URLs, names of the repositories, and driver versions in this section are subject to change. Please refer to the official VAST NFS install guide at https://vastnfs.vastdata.com/  for the latest information when required.

The latest software version can be identified in the change log.

As of October 2024, please use VAST NFS version 4.0.30
https://vastnfs.vastdata.com/version/4.0.30/source/vastnfs-4.0.30.tar.xz

Log on to the node provisioned in step 3 with super-user permission.

Install the VAST-NFS client

# Lets create a directory for VAST
mkdir -p /vast
cd /vast

# Download the VAST NFS Client source
curl -sSf https://vast-nfs.s3.amazonaws.com/download.sh | bash -s -- --version 4.0.XX


tar xf vastnfs-4.0.XX.tar.xz
cd vastnfs-4.0.XX/

# Install a dependency that might be missing
apt install debhelper

# Compile
./build.sh bin 

# Output pkg will be in dist/
cd dist/

# Recommended: Disable unattended upgrades
# Example install command on Ubuntu
apt install ./vastnfs-dkms_4.0.XX-vastdata-MLNX.OFED.LINUX-5.4-3.4.0.0_all.deb

NOTE:  your generated .deb will be named differently so please update the file name.

Step 5: Set readahead on nfs mounts in /etc/nfs.conf

This step is meant to optimize performance when using the VAST-NFS client.

sudo vi /etc/nfs.conf
#add the following snippet to the end and save the file
# No need to add any of these comment lines.

#start
[nfsrahead]
default=4096
#end

OPTIONAL Step 6: Update / Create “/etc/nfsmount.conf.d/vast.conf”

This step 6 is only necessary if you want to make use of multiple local network interfaces (ports) for NFS storage access.

If you are interested in this and other nfs mount configuration options, you can refer here: nfsmount.conf .

# create the /etc/nfsmount.conf.d directory
# as root, otherwise use sudo
mkdir -p /etc/nfsmount.conf.d/
vi /etc/nfsmount.conf.d/vast.conf

#---START---
[ NFSMount_Global_Options ]

# This line and section are optional
# Everything else can be added in fstab.  Your choice.
# Replace $IFACEx with actual interface names used to access VAST storage
# localports=$IFACE1,$IFACE2,...,$IFACEn
localports=ibp97s0f0~ibp225s0f0
#---END---

 

Step 7: Update fstab to mount the VAST NFS exports

The steps in this section must be performed on the node you provisioned in Step 3, and after completing Steps 5 and optionally Step 6.

NOTE: You can leave the options section blank if you have those options defined (uncommented) in Step 6.

NOTES:

  1. Replace vastcluster01 one of the following:

    1. A DNS name resolving to the VAST VIP Pool (if using VAST-DNS)

    2. A DNS A-record pointing to ANY one of the VAST VIPs in the appropriate VIP Pool

    3. Any IP address in the appropriate VAST VIP Pool.

  2. You will need to replace $VIP_START-$VIP_END with the appropriate VIP Pool range from your VAST Cluster.

  3. Set nconnect to the # of VIP’s or 16, whichever is smaller. This will ensure optimal balance across VAST VIPs and CNodes.

vi /etc/fstab
# Add as many entries (one per line) as you have need
# VIP:/export mount_point fs_type options
# vastcluster01 below is just a customer dns entry resolving to the VAST VIP Pool
# vastcluster01:/  /mnt/vast  nfs vers=3,proto=rdma,port=20049,nconnect=16,spread_reads,spread_writes,localports_failover,remoteports=192.168.100.1-192.168.100.16
vastcluster01:/  /mnt/vast  nfs vers=3,proto=rdma,port=20049,nconnect=16,spread_reads,spread_writes,localports_failover,remoteports=$VIP_START-$VIP_END

Step 8: Sync the image from the node and update the RAM disk.

Sync the disk image from the running node onto the software image dgxa100-image on the head node.

$ cmsh
  device use dgx-a100
  grabimage -w
  createramdisk
  commit
  

OPTIONAL Step 9: Set kernel version and create initial ramdisk

Installation of OS updates in Step 4 can result in a new kernel getting installed (as part of installing the linux-generic package), and the required kernel modules are built against the new kernel. In such a scenario, the kernelversion setting of the software image must be updated as follows:

$ cmsh
  softwareimage use dgxa100-vast-image
  set kernelversion <TAB> (choose the correct kernel version)
  commit

  # Wait for ramdisk to be generated

Step 10: Create initial ram disks. Important

This step is important

Note: Make sure to run commit twice.

$ cmsh
softwareimage createramdisk dgxa100-vast-image -w
device use dgx-a100
createramdisk -w

commit
commit
exit

Step 11: Provision node(s) with new image

The image (dgxa100-vast-image in this example) is now ready to boot any number of nodes. If you have created a BCM node category, you can configure any node to be part of that category by setting its category property. This will make the nodes inherit the softwareimage setting that you have defined for the category. Alternatively, you can configure the softwareimage property for individual nodes.

When nodes are powered on, they will be imaged using the BCM software image to which we have added the VAST Data software. The following command can be used to verify that the Nvidia drivers and services are working as expected:

cmsh
category use dgx-a100-vast
set softwareimage dgxa100-vast-image
commit
commit

Validation Steps

Run the following steps on the node(s) that were deployed using the BCM image that you built.

  1. Run vastnfs-ctl status and verify that the version matches the version which was downloaded in Step 4

$ vastnfs-ctl status
version: 4.0.22
kernel modules: sunrpc rpcrdma compat_nfs_ssc lockd nfs_acl auth_rpcgss nfsd rpcsec_gss_krb5 nfs nfsv3 nfsv4
services: rpcbind.socket rpcbind rpc-gssd
rpc_pipefs: /var/lib/nfs/rpc_pipefs

  1. Run the mount command and verify that the vast-nfs server is mounted based on what was configured in /etc/fstab in Step 7. Note that the specific mount-options you specified there will not necessarily be visible in the mount command output. (eg: remoteports)

 

If, for some reason, the validation steps fail or you don’t understand the output, please open a ticket with VAST-Support.

Related articles