VAST Cluster performs verification of payload checksums in S3 requests. When receiving an S3 upload request with a checksum header in it, the cluster verifies data integrity of the upload by calculating the checksum of the uploaded content and comparing it to the checksum attached to the content. If the checksums do not match, an error is returned.
S3 checksum verification is enabled by default. To disable it on your cluster, contact VAST Support.
Checksum verification is supported for PutObject, CopyObject, UploadPart and POST uploads. Note that in case of a multipart upload, checksum verification is performed for each of UploadPart requests, but the final checksum is not verified.
Supported checksum algorithms are CRC32, CRC32C, SHA1, SHA256, CRC64NVME.
NOTE: The CRC32, SHA1 and CRC64NVME algorithms are supported starting with VAST Cluster 5.4.6.
For PutObject requests, the checksum is expected to be found in the client request's x-amz-checksum-<algorithm> header where <algorithm> is one of the supported algorithms. The cluster also accepts the optional x-amz-sdk-checksum-algorithm header. If this header is present, its value must match the <algorithm> in the x-amz-checksum-<algorithm> header of the request.
CopyObject requests can optionally specify the checksum algorithm in the x-amz-checksum-algorithm header. If the header is included in the request, the copied object is created with the checksum calculated using the algorithm specified. If this header is not present, the cluster keeps the checksum from the source object.
NOTE: Prior to VAST Cluster 5.4.6, CopyObject requests with a checksum algorithm that differs from the one used to upload the original object, are rejected.
The checksum calculated by the cluster is returned in response to the PutObject, CopyObject or POST upload request that had a checksum header in it, and also in response to GetObject and HeadObject requests that contain the x-amz-checksum-mode: ENABLED header.
Trailing Checksums
Handling of S3 trailing checksums included in the x-amz-trailer header depends on the VAST Cluster version.
In VAST Cluster 5.4.6 and later, trailing checksums are recognized and verified in the same way as regular checksums.
Prior to version 5.4.6, the checksum (CRC32C only) is recognized as such and handled separately from the uploaded content. VAST Cluster does not perform checksum verification.
Limitations
For multipart uploads, checksum verification is performed for each of UploadPart requests, but the final checksum is not verified.