VAST S3 with LDAP - Driven Group Access via Keycloak STS (KB.md)

KB.md

A reproducible demo showing how to give S3 users on a VAST cluster temporary credentials scoped by LDAP group membership, where the LDAP group is the load-bearing piece for access control. Moving a user between LDAP groups causes their effective S3 access to change immediately on the next token request.

What this demo proves

A customer running OpenLDAP (or any LDAP) wants to:

1. Authenticate S3 users against their existing LDAP directory.

2. Have S3 access automatically scoped by LDAP group membership — no per-user policy bookkeeping in VAST.

3. Use temporary S3 credentials (STS), not long-lived access keys, so credentials expire, and revocation is just a group membership change.

This demo wires up four worked examples:

It also adds a fifth optional example: a hidden prefix inside demo-bucket that only one specific user (rwuser1) can read or write, even though other rw users have otherwise identical access.

End-to-end test result: 17 of 17 access assertions pass, plus a negative test that proves Keycloak refuses to issue a token to a user who is not a member of the matching LDAP group.

Vocabulary

A few terms before we go further. If you already know what OIDC, JWT claims, and IAM trust policies are, skip ahead.

OpenID Connect (OIDC) is an internet standard for "log in with…" flows, built on top of OAuth 2.0. An OIDC identity provider (IdP) like Keycloak holds the user database, runs the login screen, and hands back a signed token after a successful login.

That token is a JSON Web Token (JWT) — a short, signed JSON document. A JWT carries claims: small key-value facts about the holder, signed by the IdP's private key. A relying party can verify a JWT by checking its signature against the IdP's public key, which the IdP publishes over HTTPS at a JWKS endpoint.

Two claims matter for this demo. The iss claim names the IdP that minted the token (always the realm's issuer URL — VAST uses it to fetch the JWKS). The aud claim says which audience the token was minted for — usually the client_id of the OIDC client that requested it. We gate VAST role assumption on aud, which is why every per-group Keycloak client in this demo issues tokens with a distinct aud value.

VAST exposes a Security Token Service (STS) that mirrors the AWS STS API. The relevant call is AssumeRoleWithWebIdentity: hand it a JWT and the ARN of an IAM role, and if the role's trust policy allows it, you get back temporary S3 credentials (AccessKeyId, SecretAccessKey, SessionToken) that expire after about an hour.

A VAST IAM role has two policy attachments. The trust policy says who is allowed to assume this role — its conditions are evaluated against the incoming JWT's claims. The identity policy says what S3 actions the resulting session is allowed to perform. Both are JSON documents in the AWS IAM policy dialect; both support Allow and Deny statements; Deny always wins. A bucket policy is a third JSON document attached directly to a bucket. It applies to every caller regardless of which role they assumed, and is the right place for cross-cutting rules — the hidden-prefix bonus uses one.

With those terms in hand, here's how the pieces fit together.

Architecture

                     +-------------------+
                     |     OpenLDAP      |       <-- single source of truth
                     |  (users, groups)  |           for users and groups
                     +---------+---------+
                               |
                               |  LDAP federation
                               |  (group sync)
                               v
                     +-------------------+
                     |     Keycloak      |       <-- OIDC IdP
                     | realm: vast       |
                     |                   |
                     | per-group clients:|
                     |   vast-s3-rw      |
                     |   vast-s3-ro      |       <-- each client refuses
                     |   vast-s3-wo      |           to issue a token unless
                     |   vast-s3-na      |           the user has the
                     +---------+---------+           matching realm role
                               |                     (mapped from LDAP group)
                               |  JWT
                               |  aud=vast-s3-XX
                               v
                     +-------------------+
                     |    AWS CLI /      |       <-- client passes JWT to
                     |    boto3 / app    |           VAST STS endpoint
                     +---------+---------+
                               |
                               |  AssumeRoleWithWebIdentity
                               v
                     +-------------------+
                     |   VAST Cluster    |
                     | (your tenant)     |
                     |                   |
                     | IAM roles:        |
                     |   rw-role <-- aud=vast-s3-rw
                     |   ro-role <-- aud=vast-s3-ro
                     |   wo-role <-- aud=vast-s3-wo
                     |   na-role <-- aud=vast-s3-na
                     |                   |       <-- trust policy gates on
                     | identity policies:|           JWT aud claim. STS issues
                     |   rw-policy       |           temporary creds tied to
                     |   ro-policy       |           the matched role.
                     |   wo-policy       |
                     |   na-policy       |
                     +---------+---------+
                               |
                               | temporary access key + session token
                               v
                     +-------------------+
                     |    S3 endpoint    |
                     |   demo-bucket     |       <-- identity policy on the
                     |                   |           role decides what S3
                     | (optional)        |           operations succeed
                     |   bucket policy   |
                     |   for hidden/     |       <-- optional bucket-policy
                     |   prefix          |           overlay for cross-cutting
                     +-------------------+           rules (e.g. hidden prefix)

The cleanest way to think about responsibility:

What's in each component

OpenLDAP

A vanilla osixia/openldap container exposes port 389. The directory has two organizational units:

• ou=users,dc=vast,dc=local — eight inetOrgPerson users (rwuser1, rwuser2, rouser1, rouser2, wouser1, wouser2, nauser1, nauser2). Each has uid, cn, sn, mail, and userPassword.

• ou=groups,dc=vast,dc=local — four groupOfNames groups (rw-group, ro-group, wo-group, no-access-group). Each holds two member DN values pointing back into ou=users.

Bind DN for the demo: cn=admin,dc=vast,dc=local. The bind credential is whatever you set as LDAP_PASS (and pass into the OpenLDAP container as its LDAP_ADMIN_PASSWORD env var when you start it).

The three LDIF files in ldap-files define this structure. They are loaded into the container with ldapadd at bootstrap time.

There is no special schema required — only stock inetOrgPerson and groupOfNames from the standard OpenLDAP cosine and inetorgperson schemas.

Keycloak

Keycloak runs in its own container exposing 8080. The realm is named vast. There are two configuration phases:

Phase 1 — LDAP federation and group sync (script: configure-keycloak-ldap.sh)

• Creates the vast realm.

• Creates a User Federation provider of type ldap pointing at the OpenLDAP container with the demo bind credentials. editMode is READ_ONLY and importEnabled is true, so Keycloak imports users on first login and keeps them in sync with LDAP.

• Adds a group-ldap-mapper so the LDAP groupOfNames groups are imported as Keycloak groups under /rw-group, /ro-group, etc., and group membership is recomputed on each login.

• Triggers an initial full sync.

• Creates one base client (vast-s3, public, direct grants enabled) used by tooling that doesn't care about per-group restriction. The per-group clients are added in Phase 2.

After this phase, an LDAP user can authenticate against Keycloak with username + password (using the password grant against any client) and gets back a JWT containing a groups claim with their LDAP group names.

Phase 2 — Per-group clients with conditional flow (script: configure-keycloak-clients.sh)

For each of the four LDAP groups, the script creates a complete chain of objects so that token issuance is gated server-side by group membership. The chain for rw-group looks like this; the other three are identical substitutions:

1. Realm role vast-s3-rw is created in the vast realm.

2. Group role mapping: the Keycloak group /rw-group (which is already kept in sync from the LDAP rw-group) is given the vast-s3-rw realm role. Any LDAP user who joins rw-group therefore inherits the vast-s3-rw realm role on their next login.

3. Client vast-s3-rw is created — public, direct grants enabled, OpenID Connect protocol. A groups protocol mapper is attached so the issued JWT carries the user's group memberships in a groups claim (this is informational only; the actual gate is the trust policy on the matching VAST role, which checks aud, not groups).

4. Authentication flow named vast-s3-rw direct grant is created by copying Keycloak's built-in direct grant flow.

5. A conditional sub-flow named vast-s3-rw group check is added at the end of that flow with requirement: CONDITIONAL.

6. Inside the sub-flow, two executions are added in order:

   • conditional-user-role (REQUIRED), configured with condUserRole=vast-s3-rw and negate=true.

   • deny-access-authenticator (REQUIRED).

7. The new flow is bound to the client as the direct_grant override via clients.<id>.authenticationFlowBindingOverrides.

How the conditional flow gates token issuance:

• A user requests a token against the vast-s3-rw client via the password grant.

• Keycloak runs the bound vast-s3-rw direct grant flow.

• It validates username and password as normal.

• It enters the vast-s3-rw group check conditional sub-flow.

• The conditional condition is conditional-user-role with negate=true and condUserRole=vast-s3-rw. This evaluates to true when the user does not have the role.

• If the user has vast-s3-rw (because they're in rw-group in LDAP): the negated check returns false, the conditional sub-flow is skipped, the parent flow completes successfully, and a JWT is issued.

• If the user lacks vast-s3-rw (they're in some other group): the negated check returns true, the conditional sub-flow runs, the deny-access-authenticator execution fires, the password grant returns an error, and no token is issued.

The same chain exists for vast-s3-ro, vast-s3-wo, vast-s3-na.

The JWTs that do get issued by these clients have aud set to the clientId (e.g. aud=vast-s3-rw). That aud is what VAST will gate role assumption on.

VAST (VMS configuration)

All VAST objects live inside the tenant named by TENANT_NAME. The tenant must already have an OIDC provider configured against your Keycloak realm's issuer URL (e.g. http://<keycloak-host>:8080/realms/vast). Any pre-existing IAM roles, AD / local providers, or other tenant-level config are left untouched by this demo.

The demo creates four identity policies and four IAM roles in this tenant via the VMS REST API (script: create-iam.sh).

Identity policies define what S3 operations are permitted. They are JSON documents stored at iam-roles/{rw,ro,wo,na}-identity-policy.json in this repo and uploaded to the cluster as s3policies via the API.

IAM roles wrap an identity policy with a trust policy that says which JWTs are allowed to assume the role. The four trust policies in this demo are structurally identical except for one field — the expected aud value — so create-iam.sh doesn't ship them as separate JSON files. Instead, its build_trust_policy() function renders the trust policy at runtime from the OIDC issuer URL it derives from KC_FRONTEND_URL (or KC_URL) and the matching aud value. The shape it stamps out is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "oidc-provider/<issuer>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "<issuer>:aud": "<aud>"
                }
            }
        }
    ]
}

<issuer> is the realm's authority + path, no scheme — e.g. <keycloak-host>:8080/realms/vast. <aud> is one of vast-s3-rw, vast-s3-ro, vast-s3-wo, vast-s3-na.

The condition key <issuer>:aud reads the JWT's aud claim. Each role's trust policy hardcodes a different expected aud value, so a JWT issued by the vast-s3-rw client can only be used to assume rw-role, and so on. Because <issuer> is derived from a single env var the user already sets in Step 0, the iss claim Keycloak embeds in JWTs and the issuer in the trust policy stay in lockstep automatically — no editing four files when you move to a different Keycloak host.

Because Keycloak refuses to issue a vast-s3-rw token unless the user is in rw-group, the chain is complete: only members of rw-group can ever end up with credentials backed by rw-role, and rw-role is the only role whose identity policy grants full access to demo-bucket.

When the create-iam.sh script POSTs an IAM role to the VMS API, it includes a write-only s3_policies: [<policy_id>] field; VMS persists the relationship by adding the role name to the users field of the referenced policy. Both views are consistent — you can ask "which roles use this policy?" by inspecting the policy, and the verification block at the end of create-iam.sh does exactly that.

Bucket policy (optional)

The base demo does not require a bucket policy. The four roles, plus their identity policies, are sufficient to satisfy the per-group access matrix.

A bucket policy applies only to the hidden-prefix bonus described later. Bucket policies are useful when you want a rule that applies regardless of which role the caller assumed — for example, "these specific objects can only be touched by this specific user, no matter what role they hold". They are layered on top of identity-policy grants, with Deny statements winning over Allows.

How a request flows end-to-end

Concrete example: rwuser1 runs aws s3 cp file.txt s3://demo-bucket/file.txt.

1. rwuser1 (or rather, the test script on their behalf) hits Keycloak with the password grant against the vast-s3-rw client:

   POST /realms/vast/protocol/openid-connect/token
   grant_type=password
   client_id=vast-s3-rw
   username=rwuser1
   password=<the LDAP demo-user password, $PASSWORD>
   scope=openid

2. Keycloak runs the vast-s3-rw direct grant flow:

   • Validates username + password against the LDAP-federated user store.

   • Enters the conditional vast-s3-rw group check sub-flow.

   • conditional-user-role(vast-s3-rw, negate=true) evaluates: does the user have the realm role vast-s3-rw? Yes (because the /rw-group Keycloak group has the vast-s3-rw role mapped to it, and rwuser1 is in /rw-group). So the negated condition is false, and the entire sub-flow is skipped.

   • Flow exits success.

3. Keycloak issues a JWT. The aud claim is vast-s3-rw. The groups claim contains ["rw-group"]. The email claim is rwuser1@example.com.

4. The test script calls aws sts assume-role-with-web-identity against the VAST cluster STS endpoint, passing this JWT and the role ARN arn:vast::<tenant>:role/rw-role.

5. VAST validates the JWT's signature against the OIDC provider's JWKS (cached from the issuer URL in the JWT's iss claim).

6. VAST evaluates rw-role's trust policy. The condition StringEquals on <issuer>:aud == vast-s3-rw matches. STS allows the role to be assumed.

7. VAST issues temporary credentials. The session is associated with rw-role for permission purposes; the aws:username condition key (when used in bucket policies) resolves to the underlying JWT user's email, rwuser1@example.com.

8. The test script exports AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN from those credentials and runs aws s3 cp.

9. The S3 endpoint authenticates the SigV4 request, looks up the session, and evaluates rw-role's identity policy: s3:* on demo-bucket/* is Allow. There is no bucket policy denying it. The PUT succeeds.

For an rouser1 request to the same vast-s3-rw client:

1. Same Keycloak password grant, but client_id=vast-s3-rw.

2. Flow runs username + password validation: success.

3. Enters the conditional sub-flow. conditional-user-role(vast-s3-rw, negate=true) evaluates: does rouser1 have role vast-s3-rw? No (the /ro-group group is mapped to vast-s3-ro, not vast-s3-rw). So the negated condition is true, and the sub-flow runs.

4. deny-access-authenticator executes inside the sub-flow.

5. The password grant returns HTTP 401 with an error response. No JWT is issued. rouser1 simply cannot get a token from the vast-s3-rw client, full stop. There is no client-side workaround — the gate is enforced at Keycloak before the JWT is signed.

For an nauser1 request to the vast-s3-na client (their matching client):

1. Same flow. nauser1 is in no-access-group, which is mapped to vast-s3-na. They successfully obtain a JWT with aud=vast-s3-na.

2. They assume na-role (its trust policy gates on aud == vast-s3-na).

3. They run an S3 PUT.

4. na-role's identity policy is an explicit Deny on s3:* on *. The request is rejected.

The negative case is what proves the gate is real: a user can get a token, can assume a role, but the role they end up with is one whose identity policy explicitly denies everything. Membership in no-access-group is therefore equivalent to "has an account, has no S3 access".

Set this up yourself

Prerequisites

1. A VAST cluster with:

   • A tenant configured to your liking. Set TENANT_NAME to its name.

   • An OIDC provider on that tenant pointing at your Keycloak realm's issuer URL.

   • At least one S3-enabled view exposing a bucket named whatever you set BUCKET to (the walkthrough uses demo-bucket).

   • Admin credentials for the VMS REST API.

   • At least one S3 access key for the bucket owner so you can apply bucket policies later.

2. A Linux host that can reach both Keycloak (http://<host>:8080) and the VAST cluster's S3 endpoints. The host needs:

   • bash, curl, jq, aws CLI v2, python3.

   • podman (or docker) if you want to run OpenLDAP and Keycloak on the same host as the demo (recommended).

The demo runs OpenLDAP and Keycloak as Podman containers on the same host as the test runner.

Step 0 — Set environment variables

Every script in this walkthrough reads its config from the environment. Set these once, and the rest of the steps just work; if you skip this step, the scripts will refuse to run with placeholder values.

# VMS — used by create-iam.sh (TENANT_NAME also used by tests)
export VMS_HOST=vms.example.com              # <-- your VMS host
export VMS_USER=admin
export VMS_PASS='your-vms-admin-password'
export TENANT_NAME=your-tenant-name          # <-- your tenant name

# Keycloak + LDAP — used by both configure-keycloak-*.sh scripts
# (KC_URL also used by tests)
export KC_URL=http://localhost:8080
export KC_USER=admin
export KC_PASS='your-keycloak-admin-password'
export KC_REALM=vast
export LDAP_PASS='your-openldap-admin-password'

# Tests — used by test-*.sh (also reuse KC_URL and TENANT_NAME above)
export VAST_VIP=172.200.201.8
export BUCKET=demo-bucket
export PASSWORD='your-demo-user-password'   # the LDAP demo-user password

If Keycloak runs in a container without --network=host, also set LDAP_URL so it can reach LDAP from inside its container:

export LDAP_URL='ldap://192.168.1.10:389'   # your host's reachable IP

If VAST sees Keycloak at a different URL than this script does (reverse proxy, NAT, or a separate management network), set the issuer URL that VAST will embed in JWTs. Both configure-keycloak-ldap.sh and create-iam.sh read this same variable, so the issuer in the JWT and the issuer in the trust policies stay in lockstep:

export KC_FRONTEND_URL='http://keycloak.lab.example.com:8080'

Step 1 — Run OpenLDAP and Keycloak

The container env-var names below (LDAP_ADMIN_PASSWORD, KEYCLOAK_ADMIN_PASSWORD) are what the upstream images expect; we just feed our ${LDAP_PASS} and ${KC_PASS} into them.

sudo podman run -d --name openldap \
  -p 389:389 -p 636:636 \
  -e LDAP_ORGANISATION="VAST Demo" \
  -e LDAP_DOMAIN="vast.local" \
  -e LDAP_ADMIN_PASSWORD="${LDAP_PASS}" \
  docker.io/osixia/openldap:1.5.0

sudo podman run -d --name keycloak \
  -p 8080:8080 -p 8443:8443 -p 9000:9000 \
  -e KEYCLOAK_ADMIN="${KC_USER}" \
  -e KEYCLOAK_ADMIN_PASSWORD="${KC_PASS}" \
  quay.io/keycloak/keycloak:26.5.6 \
  start-dev --health-enabled=true

Wait until both containers are healthy (sudo podman ps shows Up X seconds).

Step 2 — Edit users.ldif, then load LDAP

users.ldif ships with each user's password as the literal string CHANGEME_BEFORE_LOAD. You must substitute it before loading, and the value must equal whatever you exported as ${PASSWORD} in Step 0 (otherwise the test scripts won't be able to log in as the demo users):

sed -i "s/CHANGEME_BEFORE_LOAD/${PASSWORD}/g" ldap-files/users.ldif

Then load the three LDIFs in order — OUs first (the parents), then users, then groups:

ldapadd -x -H ldap://localhost:389 \
  -D "cn=admin,dc=vast,dc=local" -w "${LDAP_PASS}" \
  -f ldap-files/ous.ldif

ldapadd -x -H ldap://localhost:389 \
  -D "cn=admin,dc=vast,dc=local" -w "${LDAP_PASS}" \
  -f ldap-files/users.ldif

ldapadd -x -H ldap://localhost:389 \
  -D "cn=admin,dc=vast,dc=local" -w "${LDAP_PASS}" \
  -f ldap-files/groups.ldif

Verify with ldapsearch:

ldapsearch -x -H ldap://localhost:389 \
  -D "cn=admin,dc=vast,dc=local" -w "${LDAP_PASS}" \
  -b "dc=vast,dc=local" "(uid=rwuser1)" dn uid mail

Step 3 — Configure Keycloak realm + LDAP federation + base client

./configure-keycloak-ldap.sh

configure-keycloak-ldap.sh

This creates the vast realm, the vast-s3 base client, the LDAP user federation, the group LDAP mapper, triggers a full LDAP sync, and adds a groups protocol mapper to the base client. The script reads KC_PASS, LDAP_PASS, and the optional KC_* / LDAP_URL overrides from the environment you set in Step 0.

It is idempotent: each step looks up existing state and either skips or PATCHes. Safe to re-run.

After this, you can sanity-check the OIDC chain end-to-end:

ID_TOKEN=$(curl -s -X POST "${KC_URL}/realms/${KC_REALM}/protocol/openid-connect/token" \
  -d "grant_type=password" -d "client_id=vast-s3" \
  -d "username=rwuser1" -d "password=${PASSWORD}" \
  -d "scope=openid" | jq -r '.id_token')
echo "$ID_TOKEN" | cut -d. -f2 | base64 -d 2>/dev/null | jq '{aud, email, groups}'

You should see groups: ["rw-group"] in the output.

Step 4 — Configure per-group Keycloak clients

./configure-keycloak-clients.sh

configure-keycloak-clients.sh

This is the Phase 2 work described above: 4 realm roles, 4 group→role mappings, 4 clients, 4 conditional flows, 4 flow bindings. The script prints a summary of each client at the end, including its authenticationFlowBindingOverrides. It reads the same KC_* env vars from Step 0 and is also idempotent.

Step 5 — Configure VAST IAM via VMS REST

./create-iam.sh

create-iam.sh

What the script does, call by call:

1. Auth — POST /api/token/ with VMS_USER / VMS_PASS as a JSON body. Returns a bearer token used for every subsequent call.

2. Tenant lookup — GET /api/tenants/?name=${TENANT_NAME} resolves the tenant's numeric id.

3. Four identity policies (rw-policy, ro-policy, wo-policy, na-policy) — for each: a GET /api/s3policies/?name=<n>&tenant_id=<tid> lookup, then either POST /api/s3policies/ (if missing) or PATCH /api/s3policies/<id>/ (if present). The policy body is read verbatim from iam-roles/<name>-identity-policy.json.

4. Four IAM roles (rw-role, ro-role, wo-role, na-role) — for each: a GET /api/iamroles/?name=<n> lookup followed by POST /api/iamroles/ or PATCH /api/iamroles/<id>/. The role body carries the in-script-generated trust policy, the matching identity policy's id, and max_session_duration (from MAX_SESSION_DURATION, default 900).

5. Verification — GET /api/iamroles/ and GET /api/s3policies/?tenant_id=<tid> filtered to the four demo names, printed as a final summary.

The trust policies are generated in-script by build_trust_policy(), which picks the OIDC issuer to bake in in this order:

1. OIDC_ISSUER if you set it directly (e.g. <keycloak-host>:8080/realms/vast)

2. Otherwise, derived from KC_FRONTEND_URL + KC_REALM

3. Otherwise, fall back to KC_URL + KC_REALM

The first line of output is the issuer the script ultimately used, so a misconfiguration is immediately obvious.

The script is idempotent: re-running it PATCHes existing policies and roles in place. If you re-run after changing KC_FRONTEND_URL or MAX_SESSION_DURATION, the affected field on every role is rewritten.

Step 6 — Verify the access matrix

unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN
./test-ldap-sts.sh

test-ldap-sts.sh

The script tests every (user, role) pair against demo-bucket, expecting the matrix in the table at the top of this article. It also runs one negative test: rouser1 attempting to get a token from the vast-s3-rw client. That request must be refused at the Keycloak layer.

Expected output:

Summary: 17 passed, 0 failed
(17 expected — 16 access-matrix + 1 negative)

If anything fails, check the troubleshooting section below.

Bonus: hidden prefix for one specific user

The base demo proves group-based access. The hidden-prefix scenario proves you can layer per-user scoping on top of that, without changing the underlying role architecture.

Goal: rwuser1 should be the only user who can read, write, or list inside s3://demo-bucket/hidden/. rwuser2 (in the same rw-group, assuming the same rw-role, with the same identity-policy grants) should be denied.

This is done with a bucket policy that adds two Deny statements. Bucket-policy Deny overrides identity-policy Allow, so even though rw-role grants s3:* on demo-bucket, the targeted Denys carve out the hidden/ prefix.

bucket-policies/hidden-prefix-policy.json:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyListInsideHiddenPrefixForOthers",
            "Effect": "Deny",
            "Principal": "*",
            "Action": ["s3:ListBucket", "s3:ListBucketVersions"],
            "Resource": "arn:aws:s3:::demo-bucket",
            "Condition": {
                "StringLike":     {"s3:prefix":   "hidden/*"},
                "StringNotEquals": {"aws:username": "rwuser1@example.com"}
            }
        },
        {
            "Sid": "DenyAccessToHiddenObjectsForOthers",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::demo-bucket/hidden",
                "arn:aws:s3:::demo-bucket/hidden/*"
            ],
            "Condition": {
                "StringNotEquals": {"aws:username": "rwuser1@example.com"}
            }
        }
    ]
}

Why this works:

• Principal: "*" matches every caller, including STS sessions.

• aws:username for an STS session resolves to the underlying JWT user's email — rwuser1@example.com for rwuser1, rwuser2@example.com for rwuser2, etc. (This is empirical VAST behavior; it works the same way for all OIDC STS callers.)

• StringNotEquals on aws:username flips the test: the Deny applies to everyone except rwuser1.

• The first statement also requires s3:prefix to start with hidden/, so it only fires on listings that target the hidden prefix.

• The second statement applies to any s3:* action whose resource matches demo-bucket/hidden or anything under it — covering get, put, delete, head, multipart, etc.

Apply it as the bucket owner (you'll need an S3 access key with policy write permission on the bucket):

export AWS_ACCESS_KEY_ID=<bucket-owner-key>
export AWS_SECRET_ACCESS_KEY=<bucket-owner-secret>
unset AWS_SESSION_TOKEN

aws s3api put-bucket-policy \
  --bucket demo-bucket \
  --policy file://bucket-policies/hidden-prefix-policy.json \
  --endpoint-url "https://${VAST_VIP}" \
  --no-verify-ssl

Then run the focused test:

unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN
./test-hidden-prefix.sh

test-hidden-prefix.sh

Expected:

Summary: 10 passed, 0 failed
(10 expected)

Importantly, after applying the hidden-prefix bucket policy, the base 17-cell test (test-ldap-sts.sh) still passes 17/17. The Deny statements are scoped narrowly enough that they don't disturb normal group-based access.

Two design patterns for prefix isolation

The bucket policy above is a Deny overlay applied on top of an identity policy that already grants broad access. It's the simplest pattern: the role's identity policy says "you can do anything in demo-bucket", and the bucket policy says "...except inside hidden/, and only for that one user".

There is a second, more rigorous pattern worth knowing about, which the VAST documentation page on bucket policy variables hints at. Instead of overlaying Denys, you make the bucket policy itself the sole grant for ListBucket, and condition it tightly on s3:prefix:

{
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:ListBucket",
    "Resource": "arn:aws:s3:::demo-bucket",
    "Condition": {
        "StringLike": {"s3:prefix": "hidden/*"},
        "StringEquals": {"aws:username": "rwuser1@example.com"}
    }
}

With only this Allow (and no broader ListBucket grant from any identity policy), a request like aws s3 ls s3://demo-bucket/ arrives at the policy engine with s3:prefix="". The empty prefix doesn't match hidden/*, so the StringLike fails, the condition fails, the Allow doesn't grant, and there's no other Allow either — so the entire ListBucket request is denied. The caller doesn't even receive an empty response with CommonPrefixes; they get an AccessDenied error and see nothing about the existence of the hidden/ prefix.

Likewise, VAST documents support for the variable ${username} in Resource and Condition entries, which lets you write a single policy that auto-scopes to whoever is calling:

{
    "Effect": "Allow",
    "Principal": "*",
    "Action": ["s3:ListBucket", "s3:GetObject", "s3:PutObject"],
    "Resource": [
        "arn:aws:s3:::demo-bucket",
        "arn:aws:s3:::demo-bucket/${username}/*"
    ],
    "Condition": {
        "StringLike": {"s3:prefix": "${username}/*"}
    }
}

Each user automatically sees only their own prefix — no per-user policy statements needed.

Trade-off: the Allow-scoping patterns are stricter, but they break clients that do an empty-prefix ListBucket as part of normal browsing (some AWS CLI flows, some boto3 patterns). If you control the clients, or the use case is "API access only", the strict pattern is the right default. If you have humans browsing with off-the-shelf S3 GUIs, the Deny-overlay pattern is more forgiving — at the cost of leaking the existence of the hidden prefix as a CommonPrefix entry in root listings.

One open caveat

The base hidden-prefix policy in this demo (the Deny overlay) does not hide the existence of hidden/ as a CommonPrefix in root listings. A non-rwuser doing aws s3 ls s3://demo-bucket/ will see hidden/ appear, even though they cannot list inside it. Switch to the Allow-scoping pattern above to also hide the prefix's existence.

Reference

 Files and Scripts index

VAST IAM trust policy condition keys

The JWT claim names that work as trust-policy condition keys (using the <issuer>:<claim> form, e.g. <keycloak-host>:8080/realms/vast:aud) are:

• aud — used by this demo for role gating

• sub — JWT subject (Keycloak user UUID)

• email — user email

• azp — authorized party (typically the same as the client_id)

The demo uses aud because it's the cleanest way to project group identity onto one of these four: one Keycloak client per group, each issuing tokens with a distinct aud, one trust policy per role gating on that value.

A parser quirk worth noting: you cannot mix operator variants from the same family in one Condition block (e.g. both StringEquals and ForAnyValue:StringEquals). Use one operator family per Condition.

Bucket policy condition keys used in this demo

• s3:prefix — for ListBucket / ListBucketVersions. Compared with StringLike / StringEquals. Used in the hidden-prefix demo.

• aws:username — resolves for STS sessions to the underlying JWT user's email (e.g. rwuser1@example.com). Used in the hidden-prefix demo to scope a Deny to a specific user.

The bucket policy's Principal: "*" is the form that works for all callers, including STS sessions. (VAST also accepts native {"User": "name@domain"} and {"Group": "name@domain"} Principal forms, but those resolve against an LDAP/AD provider attached to the tenant — they do not resolve OIDC STS callers via the tenant's OIDC provider. For the OIDC STS use case, scope by aws:username instead of by Principal.)

Credential lifetimes

Two independent clocks run on every STS call:

• JWT lifetime — set by Keycloak at the realm level (Realm Settings → Tokens → Access Token Lifespan; default 5 minutes). The JWT only needs to live long enough to complete the AssumeRoleWithWebIdentity call — once VAST has issued STS credentials it stops caring about the original JWT.

• STS credential lifetime — set by VAST at the role level via the max_session_duration field on each IAM role. This governs how long the returned AccessKeyId / SecretAccessKey / SessionToken remain valid. VAST enforces a floor of 900 seconds (15 minutes) and defaults to 129600 seconds (36 hours) when the field is omitted.

The client can optionally request a shorter-than-max lifetime via --duration-seconds on aws sts assume-role-with-web-identity (or DurationSeconds on the SDK call). Requests above the role's max_session_duration are rejected; requests at or below it return creds with that lifetime. If the client doesn't pass the flag, AWS/VAST default to 3600 s (1 hour) — subject to the role cap.

This demo pins max_session_duration to 900 seconds (the floor — 15 minutes), so a leaked session has the smallest possible blast radius. create-iam.sh reads this from the MAX_SESSION_DURATION env var, defaulting to 900. Raise it for workflows that can't re-auth mid-run:

export MAX_SESSION_DURATION=3600   # 1 hour
./create-iam.sh

create-iam.sh

Check the actual expiry on any issued credential by reading the Expiration field on the STS response:

aws sts assume-role-with-web-identity \
  --role-arn "arn:vast::${TENANT_NAME}:role/rw-role" \
  --role-session-name rwuser1-session \
  --web-identity-token "$ID_TOKEN" \
  --endpoint-url "https://${VAST_VIP}" \
  --no-verify-ssl \
  | jq -r '.Credentials.Expiration'
# → "2026-04-20T18:15:27Z"

To change max_session_duration on an existing role without re-running create-iam.sh, PATCH it directly via the VMS REST API:

# Admin token
TOKEN=$(curl -sk -X POST "https://${VMS_HOST}/api/token/" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg u "$VMS_USER" --arg p "$VMS_PASS" '{username:$u,password:$p}')" \
  | jq -r '.access')

# Role id
ROLE_ID=$(curl -sk "https://${VMS_HOST}/api/iamroles/?name=rw-role" \
  -H "Authorization: Bearer ${TOKEN}" | jq -r '.[0].id')

# Patch
curl -sk -X PATCH "https://${VMS_HOST}/api/iamroles/${ROLE_ID}/" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"max_session_duration": "900"}'

Or via vcli if you prefer a CLI:

vcli: admin> iamrole modify --id <ROLE_ID> --max-session-duration 900

For reference, creating a role from scratch with vcli (the equivalent of what create-iam.sh does via REST) looks like:

vcli: admin> iamrole create \
  --name rw-role \
  --tenant-id <TENANT_ID> \
  --identity-policies-ids <POLICY_ID> \
  --max-session-duration 900 \
  --trust-policy '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Federated":"oidc-provider/<ISSUER>"},"Action":"sts:AssumeRoleWithWebIdentity","Condition":{"StringEquals":{"<ISSUER>:aud":"vast-s3-rw"}}}]}'

The demo uses the REST API directly so everything is idempotent and scriptable; the vcli form is shown here only as a quick human-CLI equivalent.

VMS REST API endpoints used by create-iam.sh

• POST /api/token/ — auth, returns {"access": "...", "refresh": "..."}

• GET /api/tenants/?name=<n> — tenant lookup by name

• GET POST PATCH /api/s3policies/ — both identity policies and bucket-attached policies live here. The tenant_id field on a record determines which tenant the policy is in. The users field on a policy holds the names of any IAM roles attached to it.

• GET POST PATCH /api/iamroles/ — IAM roles. trust_policy is a stringified JSON. s3_policies is a write-only field accepting an array of policy IDs; VMS persists the relationship by adding the role name to each referenced policy's users field. max_session_duration is a string-typed field (seconds) capping the lifetime of any STS session assumed via this role — floor 900, default 129600 when omitted. See Credential lifetimes above.

PATCH quirk: VMS re-runs the name-uniqueness check on PATCH without excluding the row being updated, so PATCH bodies must omit name and tenant_id to avoid spurious 400 unique per Tenant errors. create-iam.sh handles this.

Keycloak admin API endpoints used by configure-keycloak-clients.sh

• POST /realms/master/protocol/openid-connect/token — admin token via admin-cli client.

• GET /admin/realms/<realm>/groups?search=<n>&exact=true — group lookup.

• GET POST /admin/realms/<realm>/roles and /admin/realms/<realm>/roles/<name> — realm role create/get.

• GET POST /admin/realms/<realm>/groups/<id>/role-mappings/realm — group → realm role assignment.

• GET POST PUT /admin/realms/<realm>/clients — client create/update. The authenticationFlowBindingOverrides.direct_grant field on the client object is what binds a custom flow.

• GET POST /admin/realms/<realm>/clients/<id>/protocol-mappers/models — protocol mapper create.

• GET POST /admin/realms/<realm>/authentication/flows and .../flows/<alias>/copy — flow create/copy from built-in.

• POST /admin/realms/<realm>/authentication/flows/<alias>/executions/flow — add a sub-flow to a flow.

• POST /admin/realms/<realm>/authentication/flows/<alias>/executions/execution — add an execution to a flow or sub-flow.

• PUT /admin/realms/<realm>/authentication/flows/<alias>/executions — update an execution's requirement (REQUIRED, ALTERNATIVE, CONDITIONAL, DISABLED).

• POST /admin/realms/<realm>/authentication/executions/<id>/config — set an execution's authenticator config (e.g. the condUserRole and negate keys for conditional-user-role).

Troubleshooting

aws sts assume-role-with-web-identity returns InvalidArgument: Unknown role. The IAM role doesn't exist in the target tenant. Re-run create-iam.sh and verify the role appears in the script's verification block. Make sure TENANT_NAME is set to the right tenant before running.

The token request returns unauthorized_client or invalid_grant. The user is not in the LDAP group whose realm role gates that client's flow. Either the user is genuinely not in the group, or the LDAP sync hasn't picked up a recent change. Trigger a manual sync via Keycloak admin UI or via POST /admin/realms/vast/user-storage/<ldap-component-id>/sync?action=triggerFullSync.

Couldn't parse condition_key when applying a trust policy. The condition key isn't on the supported list. Only aud, sub, email, and azp work in trust-policy conditions on VAST. Use one of those, or project your real claim into one of those four (the per-group client architecture in this demo does that for groups → aud).

put-bucket-policy hangs forever. Two known causes:

1. The specific VIP you're hitting is unresponsive. Switch the --endpoint-url to a different VIP in the same pool.

2. LDAP is attached directly to the VAST tenant (see next item).

Do not attach an LDAP provider directly to a VAST tenant used for OIDC STS. On some VAST builds, attaching an LDAP provider to a tenant causes put-bucket-policy and delete-bucket-policy to hang for any policy that references a Group principal — and the symptom can spread to other policy operations. This demo keeps Keycloak as the sole LDAP consumer; VAST sees only OIDC STS sessions, never the LDAP directly. If you need LDAP attached to a VAST tenant for another reason, keep it separate from the tenant you use for this demo.

The negative test prints a parse error line then PASS. Cosmetic only. Keycloak's refusal response trips up jq when the script tries to extract .id_token. The empty result is correctly interpreted as a refusal, and the test records PASS.