Overview
As required by many regulated industries, VAST Cluster features the ability to encrypt the data that is saved on the cluster's storage media (data 'at rest') to protect data from unauthorized usage.
When encryption is enabled, all data on each of the cluster's tenants is encrypted and decrypted transparently using 256-bit AES-XTS encryption. VAST Cluster generates a random unique 256-bit key at cluster initialization. Keys can be managed internally or they can be managed by external key manager (EKM). The key is unique to the cluster with the internal key management option.
With the EKM option, the key is by default unique per encryption group, which can be per cluster, per tenant or per group of tenants.
You can encrypt any new path with its own dedicated, individually manageable, encryption key. This is done by creating the path as an encrypted path before creating a view that makes the path accessible to client access. A default key per tenant encrypts all other paths on the tenant.
This feature supports the following EKM solutions:
Thales Group CipherTrust Data Security Platform, versions 2.11, 2.14 and 2.4
Fortanix DSM (supported from VAST Cluster 5.2.1)
HashiCorp Vault Enterprise (supported from VAST Cluster 5.2.1)
Encryption is disabled by default. It can be enabled at cluster creation when installing a new cluster. Encryption with internal management of encryption keys can also be enabled on a running cluster. If encryption is enabled on a running cluster, after installation, a rewrite is automatically triggered. The rewrite process rewrites all data on the cluster with encryption, scrubs the drives from any old unencrypted data and restripes the data across the drives.
FIPS 140-3 Encryption
VAST Cluster encryption of data at rest is FIPS 140-3 capable.
Limitations
Note
External generation of keys is not supported.
External management of keys is supported only if enabled at cluster installation.
Enabling Encryption at Cluster Installation
Enabling Encryption During Cluster Creation via VAST Web UI
Encryption can be enabled with the VAST Web UI Easy Install utility provided you are configuring internal key management or Thales CipherTrust as the EKM provider. EKM Encryption with other providers can be enabled only with the VAST CLI.
Enabling Encryption During Cluster Creation via VAST CLI
Note
Cluster creation is part of the cluster installation procedure and must be done in conjunction with VAST Data engineers. It is usually done using the VAST Data Easy Install utility, which supports enabling internal encryption or encryption with external key management through Thales Group CipherTrust Data Security Platform. Depending on the specifics of the deployment, a CLI command line may be used instead, with guidance. The details below relate only to the encryption parameters provided in such a command line. Cluster creation with CLI also supports encryption with other EKMs.
When creating a new cluster using the cluster create CLI command, include the following command line options in the command line.
--enable-encryption. Enables encryption.--encryption-type INTERNAL|CIPHER_TRUST_KMIP|FORTANIX_KMIP|HASHICORP_KMIP. Specifies the type of key management:INTERNAL= internally managed keys.CIPHER_TRUST_KMIP=Keys stored on a Thales Group CipherTrust Data Security Platform.FORTANIX_KMIP=Keys stored on Fortanix DSM. (supported from VAST Cluster 5.2.1)HASHICORP_KMIP=Keys stored on HashiCorp Vault Enterprise. (supported from VAST Cluster 5.2.1)
For all EKM encryption types (not for internally managed keys):
--ekm-servers EKM_ADDRESS1[:PORT1][,EKM_ADDRESS2[:PORT2][,EKM_ADDRESS3[:PORT3][,EKM_ADDRESS4[:PORT4]]]]. Specifies the IP addresses or DNS names and port numbers for up to four EKM servers. Valid port range: 1024 - 65535. Default: 5696.Either of the following:
--ekm-certificate CERTIFICATE. Specifies the SSL certificate for the connection to the EKM servers. Enter the certificate content encapsulated in quotation marks (""). Include the "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" lines from the certificate file content.--ekm-certificate-file CERTIFICATE_FILE. Specifies the SSL certificate file for the connection to the EKM servers. Place the file on the CNode host from which you are running the VAST CLI under/vast/bundles. Specify the file path in quotation marks asCERTIFICATE_FILErelative to/vast/bundles. For example:--ekm-certificate_file "/vast/bundles/cert.pem".
Either of the following:
--ekm-private_key PRIVATE_KEY. Specifies the private key of the SSL certificate for connecting to the EKM servers. Enter the private key content encapsulated in quotation marks (""). Include the "-----BEGIN EC PRIVATE KEY-----" and "-----END EC PRIVATE KEY-----" lines from the private key file content.--ekm-private_key-file PRIVATE_KEY_FILE. Specifies the private key file of the SSL certificate for connecting to the EKM servers. Place the private key file on the CNode host from which you are running the VAST CLIunder/vast/bundles. Specify the file path relative to/vast/bundlesin quotation marks asPRIVATE_KEY_FILE. For example:--ekm-private_key_file "/vast/bundles/tmp/cert.key".
Either of the following:
--ekm-ca-certificate CA_CERTIFICATE. Specifies the CA certificate file content for the connection to the EKM servers. Enter the CA certificate file content encapsulated in quotation marks (""). Include the ""-----BEGIN CA CERTIFICATE-----" and "-----END CA CERTIFICATE-----" lines from the CA certificate file content.--ekm-ca-certificate_file CA_CERTIFICATE_FILE. Specifies the CA certificate file for the connection to the EKM servers. Place the file on the CNode host from which you are running the VAST CLIunder/vast/bundles. Specify the file path relative to/vast/bundlesin quotation marks asCA_CERTIFICATE_FILE. For example:--ekm-ca-certificate_file "/vast/bundles/ca-cert.pem"
If you need to bypass certificate validation:
--ekm-bypass-validationFor Thales CipherTrust only, the following are supported and optional:
Notice
These options are introduced in VAST Cluster 5.2.1.
To connect the cluster to a child domain of the Thales CipherTrust Manager, instead of the default root domain, you can specify the domain name using the
--ekm-auth-domainoption. The subdomain needs to be created on the Thales CipherTrust manager. For example, if you created the domain "vastdomain" on the Thales CipherTrust manager, you should include--ekm-auth-domain vastdomainin the command line.When the deployment is complete, encryption groups created on the cluster will have their encryption-at-rest keys generated within the EKM Domain that was specified during deployment.
To connect to the Thales CipherTrust Manager via a proxy, specify the
--ekm-proxy-addressoption with the proxy address in the format https://proxy-address:port. For example:--ekm-proxy-address https://squid:squid@10.27.103.73:3128
This example enables encryption with internal key management:
vcli: admin> cluster create --cnode-ips 192.0.2.0,192.0.2.1,192.0.2.2,192.0.2.3 --dnode-ips 192.0.2.4,192.0.2.5 --name mycluster --psnt mycluster --enable-encryption --encryption-type INTERNAL [...]
Enabling Encryption After Installation
Limitations
Enabling encryption on a running cluster (after installation) is supported with the following limitation:
It is only possible to enable encryption with internally managed keys. Encryption with externally managed keys can only be enabled at installation.
Impact of Enablement on a Running Cluster
Enabling encryption during cluster operation triggers a rewrite of all the data and name blocks to ensure that all pre-existing data and name blocks on the cluster are encrypted.
The following are important points to note about the rewrite:
All data is typically rewritten during this rewrite and therefore the impact on storage media endurance is approximately similar to that of deleting all data on the cluster and writing it.
The rewrite proceeds as a background task that cannot be paused or stopped. In case of severe performance degradation, it may be possible for VAST Support to throttle the process and reduce the performance impact.
The rewrite may take a while, and may impact performance for workloads.
If expansions are planned, they should be done prior to enabling encryption so that the rewrite will utilize as many DBoxes as possible and minimize RAID overhead.
A combined option is available for enabling DBox High Availability and encryption simultaneously (detailed in the procedures below). If DBox HA is not yet enabled on the cluster and you intend to enable DBox HA, you should choose the combined option to avoid triggering a rewrite twice, when you enable each feature.
DBox expansion is not available while the rewrite is in progress.
Enabling Encryption from the VAST Web UI
In the VAST Web UI, open the Cluster tab of the Settings page. You can reach this by searching at the top left or from navigation menu on the left of the page.
In the Data Management section, click Enable only Encryption or Enable Encryption and DBox HA if you also plan to enable DBox High Availability.
A confirmation prompt is displayed:
These changes require rewrite and cannot be undone. Rewrite may impact workloads while it is in progress. Stopping rewrite requires support intervention. DBox expansion will not be available during rewrite. Are you sure you want to proceed?
Click Yes if you are sure you would like to proceed.
The rewrite begins and a progress bar appears at the top right of the page, reporting the current phase of the rewrite as it progresses and the percentage progress.
When the rewrite is complete, the now inactive Enable only Encryption and Enable Encryption and DBox HA buttons, as well as the Enable only DBox HA button if you chose to enable DBox HA as well as encryption, are all disabled. The tooltip for the info icon next to the buttons changes to report that DBox HA and/or encryption is/are enabled.
Enabling Encryption from the VAST CLI
Run the cluster modify command with the
--enable-encryptionoption, or, if you wish to enable DBox High Availability at the same time, run the commandcluster modify --enable-encryption --enable-dbox-ha:Note
Enabling both options at the same time reduces impact on drives and can reduce impact on workloads.
For encryption without DBox HA:
vcli: admin> cluster modify --enable-encryption
For encryption with DBox HA:
vcli: admin> cluster modify --enable-encryption --enable-dbox-ha
You are warned:
Enabling Encryption/DBox HA support triggers a required rewrite of current data. Are you sure you want to proceed? [y/N]
Enter 'y' to confirm that you want to proceed.
The rewrite begins.
You can now monitor the progress of the rewrite. Enter the command
cluster show. The command output includes the following fields:Rewrite-phase. During the rewrite, one of the main phases appears here. The order of the phases is:INTERNAL_PRE_REWRITEDATA_REWRITE_PREDATA_REWRITE_SCRUBDATA_REWRITELAYOUT_REWRITE_PRELAYOUT_REWRITEFINALIZE
Rewrite-progress. This shows the percentage progress of the current phase of the rewrite. When it reaches 100% of the final phase, the rewrite is complete.Encryption (and DBox HA capability if applicable) is now fully enabled.
Viewing Current Encryption Configuration
The VAST Web UI displays the current encryption configuration. To view this configuration on a running cluster:
From the left navigation menu, select Settings, Cluster and then KMIP.
The following fields, which are not editable, display the EKM configuration.
Encryption Type
Shows the type of encryption enabled on the cluster:
CIPHER_TRUST_KMIP. Encryption with keys managed externally on Thales Group CipherTrust Data Security Platform.
FORTANIX_KMIP. Encryption with keys managed externally on Fortanix DSM (supported from VAST Cluster 5.2.1).
HASHICORP_KMIP. Encryption with keys managed externally on HashiCorp Vault Enterprise (supported from VAST Cluster 5.2.1).
INTERNAL. Encryption with keys managed internally.
No encryption.
EKM Servers
The IP addresses and port numbers of the EKM servers, except if Encryption Type is INTERNAL.
Managing Encrypted Paths
When encryption is enabled on a cluster, all paths on the cluster are encrypted. However, encrypted path refers to a path that is encrypted with a dedicated encryption key.
Creating Encrypted Paths
An encrypted path can only be created on a new path. A view can only be created on the path after it is created as an encrypted path.
Creating an encryption path creates a new encryption group and key for the path.
Creating an Encrypted Path from the VAST Web UI
From the left navigation menu, select Element Store and then Encrypted Paths.
Note
The Encrypted Paths tab only appears in the VAST Web UI if EKM encryption is enabled.
Click Create Encrypted Path.
Complete the fields:
Name
Provide a name for the encrypted path.
Tenant
Select the tenant on which want to create the path.
Path
Enter a path under the specified tenant that you want to create and encrypt. This must be a path that does not already exist.
Click Create.
The path is created and an encryption key is created for the path. You can now create a view on the path.
Creating an Encrypted Path from the VAST CLI
Use the encryptedpath create command.
Modifying Encrypted Paths
You cannot change the tenant or the actual path of an existing encrypted path resource. You can only change its name.
Modifying Encrypted Paths from the VAST Web UI
From the left navigation menu, select Element Store and then Encrypted Paths.
Right-click the encrypted path and select Edit.
In the Name field, modify the name.
Click Update.
Modifying Encrypted Paths from the VAST CLI
To modify an encrypted path from the VAST CLI, use the encryptedpath modify command.
Deleting Encrypted Paths
Removing an encrypted path deletes the directory and the encryption group associated with the encrypted path.
Deleting Encrypted Paths from the VAST Web UI
Right-click the encrypted path and select Remove.
Click Yes to confirm the removal.
Deleting Encrypted Paths from the VAST CLI
Use the encryptedpath delete command.
Managing Encryption Groups and Keys
If encryption is enabled with EKM, an encryption group is required at tenant creation. Multiple tenants can optionally share the same encryption group. The group cannot be changed per tenant after tenant creation.
The default tenant is encrypted by a special encryption group that is created internally. It is named INTERNAL_ENCRYPTION_GROUP_CRN and is not user-provided. This encryption group does not use the EKM. Data stored in the default tenant is not protected with encryption keys provided by the EKM. Be sure to create at least one tenant to make use of EKM-managed keys.
With EKM encryption, data is encrypted using a data encryption key (DEK). There is one default DEK per encryption group which is used to encrypt all paths on the tenant(s) associated with the encryption group. Additional dedicated encryption groups can be created and used to encrypt specific paths, called encrypted paths.
Each encrypted path's encryption group can be managed independently to control the encryption status of the specific path. It is also affected by the encryption group of the tenant. For example, if the tenant's encryption group is revoked, access is blocked to all paths on the tenant, including so-called encrypted paths.
DEKs are retrieved when needed from the EKM. VAST Cluster uses a different key, called the key encryption key, to retrieve the DEKs for a given encryption group. VAST Cluster generates a master key per cluster. The cluster uses the master key to encrypt the data encryption keys when they are distributed from the cluster node that hosts the encryption service client to other nodes in the cluster.
Listing Encryption Groups
The encrytiongroup list VAST CLI command lists all encryption groups on the cluster and their state. For example:
vcli: admin> encryptiongroup list +----+------+--------------------------------------------------------------------------------+--------+ | Id | Name | Crn | State | +----+------+--------------------------------------------------------------------------------+--------+ | 4 | N/A | T_04757adb-85c0-48eb-9cd9-ef8d23cdae6d_EP_602eb029-61c1-4570-a9b5-3a3161dad798 | ACTIVE | | 1 | N/A | INTERNAL_ENCRYPTION_GROUP_CRN | ACTIVE | +----+------+--------------------------------------------------------------------------------+--------+
To map the ID of an encryption path or tenant to an encryption group, check the encryption group ID in the details of the encryption path or tenant. For example:
vcli: admin> encryptedpath show --id 1 +------------------+-------------+ | ID | 1 | | Name | EPath | | Path | /epath | | Tenant-id | 1 | | Tenant-name | default | | Encryption-group | 4 | +------------------+-------------+
Managing Key Encryption Keys
You can manage key encryption keys and the master key in the following ways:
Revoking and Reinstating Key Encryption Keys
Encryption keys can be revoked and reinstated per encryption group. When keys are revoked, the keys can no longer be used and data that was written with the revoked key can no longer be accessed (unless reinstated).
Revoking and Reinstating Key Encryption Keys from the VAST Web UI
To revoke or reinstate the key encryption key for a tenant-level encryption group:
From the left navigation menu, select Element Store and then Tenants.
Right click on a tenant that belongs to the encryption group and select either Revoke Encryption Group or Reinstate Encryption Group.
Click Yes to confirm the action.
To revoke or reinstate the key encryption key for an encrypted path's encryption group:
Right-click the encrypted path and select Revoke Key or Reinstate Key.
Click Yes to confirm the action.
It takes some time for a key to be revoked or reinstated. Once the key is revoked, the Reinstate option becomes available. The path is no longer accessible to clients and remains inaccessible until and unless the key is reinstated.
Revoking and Reinstating Key Encryption Keys from the VAST CLI
To revoke any encryption group's key, use the encrytiongroup revoke-encryption-group VAST CLIcommand.
To reinstate any encryption group's key, use the encrytiongroup reinstate-encryption-group VAST CLI command.
To revoke or reinstate encryption keys for a tenant-level encryption group, use the tenant alter-encryption-group-state command.
Rotating Key Encryption Keys
VAST Cluster supports the rotation of key encryption keys. Rotating a key encryption key generates a new version of the key encryption key for a given encryption group. You can rotate a key encryption key from the VAST Web UI or the VAST CLI. It is preferable to do this on the cluster rather than on the EKM. If a key is rotated on the EKM, there will be a small delay until the cluster is aware of the change.
Rotating Key Encryption Keys from the VAST Web UI
To rotate a key encryption key for an encryption group:
From the left navigation menu, select Element Store and then Tenants.
Right click on a tenant that belongs to the encryption group and select Rotate Encryption Group.
Click Yes to confirm the action.
To rotate an encryption path key:
Right-click the encrypted path and select Rotate Key.
Click Yes to confirm the action.
Rotating Key Encryption Keys from the VAST CLI
To rotate a key encryption key for a tenant-level encryption group from the VAST CLI, use the tenant rotate-encryption-group-key command.
To rotate any encryption group's key, use the encryptedpath rotate-encryption-group-key VAST CLI command.
Rotating the Master Key
The master key should only be rotated from the cluster and not directly on the EKM. If a key is rotated on the EKM, there will be a small delay until the cluster is aware of the change.
Rotating the Master Key from the VAST Web UI
From the left navigation menu, select Settings, then Cluster and then KMIP.
Click the Rotate button and then click Yes to confirm the action.
Rotating the Master Key from the VAST CLI
You can rotate the master key using the cluster rotate-master-encryption-group-key VAST CLI command.
Managing Encryption Key Expiration
Important
VAST Cluster monitors encryption key expiration and issues a critical alarm one week, two days and one day before an encryption key expires on your EKM. In order to retain access to your encrypted data, it is essential to generate a new key version for the encryption group on your EKM before the existing key expires.