Boto3 Client

To connect to the low-level client interface, use Boto3’s client() method. You must pass your VAST Cluster S3 credentials and other configurations as parameters with hardcoded values. This is the only way to specify a VAST Cluster VIP as the S3 endpoint.

The following example imports the boto module and instantiates a client with the minimum configuration needed for connecting the client to your VAST Cluster S3 account over an HTTP connection:

import boto3
s3_client = boto3.client(
    's3',
     use_ssl=False,
     endpoint_url=<ENDPOINT-URL>    
     aws_access_key_id=<ACCESS-KEY>,
     aws_secret_access_key=<SECRET-KEY>
     region_name=<REGION>
     config=boto3.session.Config(
          signature_version='s3v4'
          s3={'addressing_style': 'path'}
    
     )
)

in which:

For HTTPS Connection

For an HTTPS connection, pass parameters as follows in the client() call:

Once you have an instance of the S3 service client, you can call the create_bucket() method on the client instance to create a bucket.

Note

The LocationConstraint(string) parameter can be provided within the CreateBucketConfiguration (dict) parameter to specify a string representing a region where the bucket is created. If you don't specify a region, the region is set to 'vast-1'.

In this example, we create a bucket called mybucket.

response = s3_client.create_bucket(
    Bucket='mybucket'
)

The list_buckets() method returns a list of all buckets owned by the authenticated sender of the request.

response = s3_client.list_buckets()

The list_objects_v2() method returns some or all (up to 1000) of the objects in a bucket. You can use the request parameters as selection criteria to return a subset of the objects in a bucket.

The list_objects() method is a prior version of the same method, supported for backward compatibility.

This example retrieves the list of objects in the bucket "mybucket".

response = s3_client.list_objects_v2(
    Bucket='mybucket',
)

The head_bucket() method is used to determine if a bucket exists and if the user has permission to access it.

The delete_bucket() method deletes a bucket. All objects in the bucket must be deleted before the bucket can be deleted.

response = s3_client.delete_bucket(
    Bucket='mybucket'
)

Before setting ACL permissions, we recommend you read Managing S3 Access Control Lists (ACLs).

The put_bucket_acl () method sets the permissions on a bucket using access control lists (ACL).

To grant permission to a user, specify the grantee with the following parameters:

To grant permission to a group, specify the grantee as follows:

To grant permission to a predefined group, specify Group as the 'Type' and pass the group's URI as the 'URI':

In this example, a user with VID 3 is granted full control permission to the bucket my_bucket owned by JDoe whose VID is 2.

response = s3_client.put_bucket_acl(
      AccessControlPolicy={
        'Grants': [
            {
                'Grantee': {
                    'ID': '54',
                    'Type': 'CanonicalUser',
                },
                'Permission': 'FULL_CONTROL'
            },
        ],
        'Owner': {
            'DisplayName': 'BSmith',
            'ID': '4'
        }
    },
    Bucket='BobsBucket',
)

In the following example, a group identified with an email address of mygroup@domain.com is granted WRITE permissions on the bucket BobsBucket.

response = s3_client.put_bucket_acl(
      AccessControlPolicy={
        'Grants': [
            {
                'Grantee': {

                    'Type': 'GroupLoginName',
                    'EmailAddress': 'mygroup@domain.com'
                },
                'Permission': 'WRITE'
            },
        ],
        'Owner': {
            'DisplayName': 'BSmith',
            'ID': '4'
        }
    },
    Bucket='BobsBucket',
)

In the following example, the Authenticated_Users group is granted READ permission on the bucket BobsBucket.

response = s3_client.put_bucket_acl(
      AccessControlPolicy={
        'Grants': [
            {
                'Grantee': {

                    'Type': 'Group',
                    'URI': 'http://acs.amazonaws.com/groups/global/AuthenticatedUsers'
                },
                'Permission': 'READ'
            },
        ],
        'Owner': {
            'DisplayName': 'BSmith',
            'ID': '4'
        }
    },
    Bucket='BobsBucket',
)

The get_bucket_acl() method retrieves the ACL of a bucket.

To learn about VAST Cluster's support for S3 ACLs, read Managing S3 Access Control Lists (ACLs).

The get_bucket_location() method is used to return the region in which the bucket resides.

If a region was specified in the CreateBucket() request, it is returned by this method. If no region was specified in the CreateBucket() request, the bucket region was set to 'vast-1', which is returned by this method.

response = s3_client.get_bucket_location(
    Bucket='mybucket',
)

The put_object() method adds an object to a bucket.

The copy_object() method creates a copy of an object already stored on the server.

In this example, we copy the object MyObject from the bucket MyBucket to the bucket MyOtherBucket and name the copy MyObjectCopy.

response = S3_client.copy_object(
    Bucket='MyOtherBucket',
    CopySource='MyBucket/MyObject'
    Key='MyObjectCopy',
)

The get_object() method retrieves an object.

To download a specified range of bytes of an object, use the Range parameter. For more information about the HTTP Range header, go to http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.

In this example, we download only bytes 32-64 of the object "MyObject" from the bucket "MyBucket".

response = S3_client.get_object(
    Bucket='MyBucket',
    Key='MyObject',
    Range='bytes=32-64',
)

The head_object() method retrieves metadata from an object without returning the object itself.

The delete_object() method deletes an object.

The delete_objects() method deletes multiple objects in a bucket.

response = s3_client.delete_objects(
    Bucket='mybucket',
    Delete={
        'Objects': [
            {'Key': 'file1'},
            {'Key': 'file2'},
            {'Key': 'file3'},
        ],
    },
)

Before setting ACL permissions, we recommend you read Managing S3 Access Control Lists (ACLs).

The put_object_acl() method sets the permissions on an object using access control lists (ACL).

To grant permission to a user, specify the grantee with the following parameters:

To grant permission to a group, specify the grantee as follows:

To grant permission to a predefined group, specify Group as the 'Type' and pass the group's URI as the 'URI':

In this example, a user with VID 3 is granted full control permission to the object my_object in the bucket my_bucket owned by JDoe whose VID is 2.

response = client.put_object_acl(
      AccessControlPolicy={
        'Grants': [
            {
                'Grantee': {
                    'ID': '3',
                    'Type': 'CanonicalUser',
                },
                'Permission': 'FULL_CONTROL'
            },
        ],
        'Owner': {
            'DisplayName': 'JDoe',
            'ID': '2'
        }
    },
    Bucket='my_bucket',
    Key='my_object',
)

In this example, the predefined AUTHENTICATED_USERS group is granted WRITE permission to the object my_object in the bucket my_bucket owned by JDoe whose VID is 2..

response = client.put_object_acl(
    AccessControlPolicy={
        'Grants': [
            {
                'Grantee': {
                    'Type': 'Group',
                    'URI': 'http://acs.amazonaws.com/groups/global/AuthenticatedUsers'
                },
                'Permission': 'WRITE'
            },
        ],
        'Owner': {
            'DisplayName': 'JDoe',
            'ID': '2'
        }
    },
    Bucket='my_bucket',
    Key='my_object',
)

The get_object_acl() method returns an object's ACL.

To learn about VAST Cluster's support for S3 ACLs, read Managing S3 Access Control Lists (ACLs).

The create_multipart_upload() method initiates a multipart upload and returns an upload ID.

After initiating the multipart upload, you then need to upload all parts and then complete the upload.

The abort_multipart_upload() method aborts a multipart upload after it was initiated.

After a multipart upload is aborted, no additional parts can be uploaded using the upload ID of that multipart upload. The storage consumed by any previously uploaded parts will be freed. However, if any part uploads are currently in progress, those part uploads might or might not succeed. As a result, it might be necessary to abort a given multipart upload multiple times in order to completely free all storage consumed by all parts.

The complete_multipart_upload() method completes a multipart upload by assembling previously uploaded parts.

The upload_part() method uploads a part in a multipart upload that was already initiated.

After uploading all parts, the upload needs to be completed.

The upload_part_copy() method uploads a part of a multipart upload by copying data from an existing object as data source.

The list_parts() method lists the parts that have been uploaded for a specific multipart upload.

The list_multipart_uploads() method lists all multipart uploads in progress.

Use the put_bucket_logging() method to configure bucket logging.

The following example enables logging to a destination bucket named c-2024-10-10-112417-595579, where the log object keys will have a prefix of -V/E-Test-. Users with emails myuser@example.com and onemoreuser@example.com will have full control permissions to the log objects.

s3.client.put_bucket_logging(
    Bucket=bucket_name,
    BucketLoggingStatus={
        'LoggingEnabled': {
            'TargetBucket': 'c-2024-10-10-112417-595579', 'TargetPrefix': '-V/E-Test-', 'TargetGrants':
                 [{'Grantee': {'Type':'AmazonCustomerByEmail', 'EmailAddress': 'myuser@example.com'}, 'Permission': 'FULL_CONTROL'},
                 {'Grantee': {'Type': 'AmazonCustomerByEmail', 'EmailAddress':'onemoreuser@example.com'}, 'Permission': 'FULL_CONTROL'}]
        }
    }
)

Use the get_bucket_logging() method to configure bucket logging, for example:

response = client.get_bucket_logging(
    Bucket='mybucket',
)

The put_bucket_notification_configuration() method can be used to configure bucket event notifications, for example:

s3.client.put_bucket_notification_configuration(
    Bucket=bucket_name, NotificationConfiguration={
        'TopicConfigurations': [
            {'Id': 'from_S3',
             'TopicArn': 'long-analysis:authority-hand',
             'Events': ['s3:ObjectCreated:*'],
             'Filter': {'Key': {'FilterRules': [{'Name': 'suffix', 'Value':'foo'}]}}}
        ]
    }
)

The get_bucket_notification_configuration() method can be used to return event notification configuration for a bucket, for example:

s3.client.get_bucket_notification_configuration(Bucket=bucket.name)