In an S3 bucket, the objects are stored in a flat hierarchy where every object has a unique key. The key is the name of the object and can contain slashes, unlike in other access protocols where a slash denotes a directory delineator in a file system path. In S3, directories do not exist but slashes can be used to organize the namespace and simulate the illusion of directories. Some S3 applications use objects with trailing slashes in their keys to simulate directory nodes. For example, an application could use an object called 'dev/' as a directory by creating new objects with keys that have 'dev/' as a common prefix. This use of objects as directory nodes may be referred to as 'S3 directories'.
The VAST Cluster S3 implementation supports 'S3 directories' with limitations in terms of multiprotocol access. S3 directories are supported as follows:
Object keys specified in S3 requests can contain slashes:
Object keys can start or end with a slash, such as '/a' or ‘a/’.
Object keys can contain multiple slashes in series, such as '//a', 'a//', 'a//b', or 'a///'.
You can allow or prohibit use of S3 object names containing character combinations that are not compatible with other access protocols:
Names starting with
./or../Names containing
//,/./or/../Names ending with
/.or/..
To do so in VAST Web UI, use the Enable S3 special chars support slider in the S3 tab of a view policy (Element Store -> View Policies -> choose to create or edit a view policy).
When using VAST CLI, specify the
--s3-special-charsor--s3-special-chars-not-supportedoption on theviewpolicy createorviewpolicy modifycommand.An object created via S3 RPC cannot be accessed via NFS or SMB, neither as a file nor as a directory, if the object key starts with a slash, or ends with a slash, or includes multiple slashes.
An object with a key that has a trailing slash is a valid S3 object and is usable as an S3 directory by virtue of the trailing slash. It is otherwise the same as any other S3 object, supporting S3 operations and able to contain data as well as metadata, ACL, and versions.
In non-versioned buckets, two objects can have keys that are identical except that one of them has a single trailing slash. This is effectively like having a file and a directory with the same name. For example, you can have an object with key 'a/b' and a 'directory' object with key 'a/b/'. These are valid as two distinct objects. However, there is a limitation that you cannot have a file with the same name as a directory if you also have a file 'inside' that directory. For example: although you can have "key1" and 'key1/" as object keys, you cannot have "key1" and "key1/key2".
In versioned buckets, object keys must be distinguished by more than a single trailing slash, since a trailing slash is reserved for the object versioning directory. So, for example, the following are valid in a versioned buckets as keys for two distinct objects: 'a/b' and a/b//'.