Unified Architectural Overview
Managing a secure VAST cluster deployment across on-premises data centers and public cloud environments requires a layered security approach to ensure operational resilience and simplify long-term maintenance. Because modern firewalls operate on an implicit "deny all" logic once an explicit rule is created, managing cluster security requires segmenting traffic into distinct operational profiles rather than treating all cluster traffic uniformly.
To maintain the Internal Cluster Fabric layer without administrative fatigue, this architectural design pattern uses identity-based filtering rather than opening entire subnet CIDRs. This prevents unauthorized instances (such as a compromised jumpbox in the same subnet) from intercepting internal cluster traffic.
Core Architectural Principle: Do not open the subnet; open the cluster to itself.
The Three Traffic Segregation Layers
Protocol & Access Ports (Perimeter Ingress)
Scope: Inbound traffic from end-users, administrators, and client applications (e.g., Spark, Kafka, Trino).
Strategy: Strict, explicit whitelisting of well-known ports (NFS, SMB, REST API, SSH). Access must be tightly controlled and restricted only to authorized client subnets.
Internal Cluster Fabric (Intracluster Mesh)
Scope: High-volume, low-latency node-to-node communication and orchestration (C/Dnodes and Eboxes) strictly within the internal logical identity (vast-internal).
Strategy: Identity-Based Trust (Port/Protocol-Agnostic). Future-proof the installation against new cluster services by opening all TCP, UDP, and ICMP traffic exclusively to the cluster's identity, completely bypassing port-by-port micromanagement.
Replication Ports (Edge Ingress & Egress)
Scope: Global data replication across data centers or hybrid-cloud environments.
Strategy: Bi-directional explicit routing. These ports require stateful tracking and configuration across both host firewalls and all intermediate border routers bridging the sites.
[Add Link to the ports document I created that matches this design pattern]
Mapping Internal Fabric Identity-Based Trust across platforms
Platform / Framework | Native Mechanism | BKM Architectural Implementation |
|---|---|---|
AWS | Self-Referencing Security Groups | A single ingress and egress rule is created where the Source is the Security Group ID (sg-xxxx) of the group itself. This acts as a cryptographic "badge," allowing any node assigned that SG to talk to any other node with the same SG on all ports. |
GCP & Azure | Network Tags / Network Security Groups (NSGs) | Implements identical self-referencing logic where the source and target filters are defined by the exact same identity (Network Tag in GCP or NSG in Azure). |
Linux firewalld | Trusted Zone | Binds all VAST cluster node IPs as sources to the native trusted zone. This automatically permits all protocols and ports between nodes, while leaving the default zone to restrict outside traffic. |
Linux iptables | IPset Integration | Aggregates cluster IPs into a single ipset named list. An INPUT rule accepts the set as a source while intentionally omitting protocol (-p) and destination port (--dport) matchers to allow all traffic. |
Linux nftables | Native IP Sets | Defines a native IP address set (e.g., @vast_cluster_nodes) within the configuration table. Writes an input chain rule allowing the set as a source without declaring specific tcp or udp destination ports. |