Handling Object ETags and the Content-MD5 Header

For regular S3 uploads (non-multipart):

The ETag calculated by the VAST cluster is the MD5 checksum of the data in the upload request.

The ETag that the cluster calculates is compared with the value of the (optional) Content-MD5 header sent by the client. If the comparison fails, the request fails.

If the request succeeds, the ETag is stored and is also returned to the client.

For multipart uploads:

For each part being uploaded, a separate ETag is calculated for each part, compared with the MD5 checksum sent by the client in the Content-MD5 header, as for non-multipart uploads, and returned as a header in the response.

As part of multipart upload completion, the VAST cluster generates a single ETag for the whole multipart upload. This ETag is the MD5 checksum of the concatenation of the parts’ ETags, followed by a dash and then the number of parts. For example, for a 10-part upload: 9b2cf535f27731c974343645a3985328-10.

As part of the complete multipart upload request, the client must send a list of parts and an ETag for each part during completion. The VAST cluster validates that list.

When a non-S3 protocol such as NFS or SMB creates or modifies an S3 object or changes its size:

A unique S3 ETag is calculated based on metadata. Such ETags include a fixed prefix to identify them (48ed760a74). In case of modification, where the object previously had an S3 ETag, the ETag is replaced.