vperfcheck: Automated Performance Testing

Intro

vperfcheck is a profile-driven elbencho wrapper designed for automated performance testing across multiple clients. The tool simplifies distributed storage benchmarking by automatically coordinating workloads and sizing datasets.

It supports testing of shared filesystems and S3 object storage.

Prerequisites

Ensure your environment meets the following requirements before running the tool:

• Connectivity: Passwordless SSH must be configured on the given hosts.

• Placement: If elbencho is not preinstalled on the hosts, the script needs to run from a shared file system to make the executable available across all testing hosts.

• Elbencho: If elbencho is not already pre-installed on the hosts, the script will automatically attempt to download it from GitHub. You can download an installable rpm/deb package for x86_64 and ARM64 from the “Releases” section on GitHub here.

• S3 Mode: 

  • For S3 object storage tests, you must export the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_ENDPOINT_URL_S3 environment variables.

  • The official aws CLI tool must be preinstalled on the host where you run the script. It is available from the standard repo of major distros under the package name awscli or aws-cli or also directly from AWS here.

Download

Info

The latest stable release of vperfcheck is always available here:
http://vast-vperfsanity.s3.amazonaws.com/download/vperfcheck-latest-stable.tar.gz

You can download the vperfcheck package directly to an existing shared filesystem mount point on a client:

# change to a directory on your shared filesystem
cd /mnt/vast

# download the vperfcheck script tar file
wget http://vast-vperfsanity.s3.amazonaws.com/download/vperfcheck-latest-stable.tar.gz

# extract tar file
tar -xf vperfcheck-latest-stable.tar.gz

# change into the extracted directory
cd vperfcheck

Automatic Dataset Sizing

To ensure tests run for a meaningful duration without waiting for too long for results, vperfcheck handles dataset sizing automatically:

• Default Auto-Sizing: If no --prep parameter is provided on the first run, the tool dynamically increases the dataset size until the write phase runs for at least 60 seconds.

  • Re-Sizing: Trigger the automatic sizing phase again by using --prep auto.

  • Host Scaling: It is highly recommended to run --prep auto after significantly increasing the number of hosts.

• Manual Sizing: You can explicitly specify a dataset size, such as --prep 1t.

  • Validation: On startup, vperfcheck validates the existing dataset and only triggers a preparation phase if the dataset does not meet the specified size.

Quick Start Examples

The vperfcheck.sh script accepts hosts as direct arguments (e.g., node[01,03-05]) or alternatively by giving a path to a newline-separated hosts file (e.g. myhosts.txt) instead of hostnames.

See built-in --help for basic options and --help-adv for advanced options.

Shared Filesystem

# Automatically size a dataset and run basic performance tests from current host
./vperfcheck.sh --dir /mnt/vast/benchdir

# Run basic tests from multiple hosts and auto-resize the dataset
./vperfcheck.sh --dir /mnt/vast/benchdir --prep auto myclient001 "myclient[010-100]" 

# Run a specific read bandwidth test explicitly
./vperfcheck.sh --dir /mnt/vast/benchdir --test file-read-bw myclient001 myclient005 "myclient[010-100]"

Info

The above examples assume you created a target directory at /mnt/vast/benchdir.

S3 Object Storage

# Export necessary S3 credentials and endpoints
export AWS_ACCESS_KEY_ID="mys3user"
export AWS_SECRET_ACCESS_KEY="mys3secret"
export AWS_ENDPOINT_URL_S3="http://172.200.201.[1-10]"

# Automatically size a dataset into an S3 bucket under "benchtest/" prefix
./vperfcheck.sh --dir s3://mybucket/benchtest/

# Resize dataset and run basic tests from multiple hosts against the S3 target
./vperfcheck.sh --dir s3://mybucket/benchtest/ --prep auto myclient001 "myclient[010-100]" 

Info

Multiple S3 server endpoint IPs can be specified by using square bracket ranges in the environment variables.

Info

If the given S3 bucket doesn’t exist, it will automatically be created if the user has appropriate permissions.

Result Output

Results are generated in a summary table directly on the console. The summary, full result details (including .csv and .json files) and the executed elbencho command lines are also automatically logged to the results/ subdirectory.

Result Summary Example

========================================================================================
 SUMMARY (2026-06-19 12:05:15 -0700) | Dataset: 4 TiB, 4096 files
========================================================================================
 Profile          Hosts  Threads  Block  Throughput      IOPS   Avg_IO_Lat       Elapsed
 ---------------  -----  -------  -----  ----------  --------  -----------  ------------
 prepare              8       64     8M   57.8 GB/s      6.9K            -       1m18.1s
 file-read-bw         8       64     1M  102.3 GB/s     97.6K            -         43.7s
 file-read-iops       8       64     4K    7.3 GB/s      1.8M            -        1m0.0s
========================================================================================

Packaging and Sharing Results

After running your benchmarks, you can easily bundle your test results into a .tar.gz archive to share with others or for archival purposes.

Use the bundled packaging helper:

helpers/vperfcheck_package.sh

This script packages your local results/ folder (along with the base tool configuration) and places the final archive in the builds/ subdirectory. The downloaded elbencho binary is automatically excluded to keep the archive file size small.

List of Configuration Options

The tables below show the main configuration options for reference. For the complete list of options, use ./vperfcheck.sh --help to see the basic options and ./vperfcheck.sh --help-adv to see the advanced options.

Basic Options

Advanced Options

List of Available Benchmark Profiles

The table below shows the main available benchmark profiles that can be selected via the --test PROFILE_NAME parameter. The --test parameter can be given multiple times and also accepts wildcards (e.g. --test file-read-*).

For the full list of available profile names, see the profiles/ subdir of the vperfcheck package.