Use AWS CLI and S3cmd with VAST

Prev Next

Summary

Here are a few simple examples of using the AWS CLI and S3cmd command-line tools with VAST. The tools are very similar in behavior, so we describe both together.

If you need assistance using these tools or would like to use a different tool, please let us know, and we'll be happy to help.

Create Credentials

To obtain the needed S3 credentials, use VMS to create them for the target user. The steps are in the official documentation.

Create Configuration File

S3cmd

s3cmd needs a configuration file that specifies the credentials as well as the endpoint information. To make a config file for s3cmd, name it something like vast.s3cfg. Put the following in, replacing IP_ADDRESS with one of the VAST VIPs (don't specify a port!).  And of course specify the access key and secret key you created earlier.

 

[default] 
access_key = REPLACE
secret_key = REPLACE
host_base = IP_ADDRESS
host_bucket = IP_ADDRESS
use_https = False
server_side_encryption = False
signature_v2 = False
signurl_use_https = False
encrypt = False

 

AWS CLI

The AWS CLI also works with VAST. First of course, you'll need a credentials file. Normall,y this is in ~/.aws/credentials, but you can override the location by specifying the environment variable AWS_CONFIG_FILE. If you already have a configuration file, you can add a new stanza just for VAST. If you do not have a configuration file, let the AWS CLI create it for you by running this command and answering the questions. You can use the defaults for everything if you wish, except, of course, provide the VAST access key and secret key when asked.

aws configure

If you already have a credentials file, you can easily add a new stanza for VAST. The resulting file will look something like this:

cat ~/.aws/credentials 
[default]
aws_access_key_id = xxxxxxx
aws_secret_access_key =xxxxxx
[vast]
aws_access_key_id = VASTACCESSKEY
aws_secret_access_key = VASTSECRETKEY

 If you've chosen to create a VAST-specific profile for the AWS CLI, then you'll need to specify --profile vast when using the command. Our examples will assume that.

You'll notice that, unlike the s3cmd case, we have not specified the IP address for VAST above, nor anything about SSL certificate verification. We will instead specify that on the command line when running the AWS CLI.

Sample Commands

Now that we've set up our environment, let's try some simple operations.

List the root directory

s3cmd -c vast.s3cfg ls s3://

or 

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 ls s3://

 

Make a bucket to work in

s3cmd -c vast.s3cfg mb s3://mybucket

or

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 mb s3://mybucket

 

Put a file

s3cmd -c vast.s3cfg put vast.s3cfg s3://mybucket/

or

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 cp vast.s3cfg s3://mybucket

 

List the new bucket

s3cmd -c vast.s3cfg ls s3://mybucket/

or

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 ls s3://mybucket/