VAST Data Platform with Commvault Backup

Introduction

The ever-accelerating pace of IT operations and the shift to all-flash for primary storage have left many organizations dissatisfied with the performance of their backup targets, especially on restores. While many users still think of backups and restores as sequential, as they were in the days of tape, modern backup techniques, from incremental forever to data deduplication, have made backups and restores much more random than they used to be.

Users too often discover this when they must perform their first large restore; they find that restores are only 1/5 as fast as backups to it. A VAST cluster provides a single storage pool, namespace, and data reduction realm (See VAST Data Reduction Whitepaper) capable of spanning 100’s of petabytes and is actually faster reading than writing.

VAST Data’s multiprotocol S3 and NFS protocol support also allows for pluggable integration as a primary copy of data or a replication target without compromising on storage scale capabilities, redundant appliances, or isolated silos of backup targets.

All-Flash performance also unlocks the secondary copies of data for uses beyond just backup. Commvault’s data management suite can index and search secondary copies for eDiscovery, ransomware detection, data exfiltration analysis, etc., but only if the repository is fast enough to support these functions.

Commvault has an architecture that employs a balance of scale-out and locality options for being able to back up and recover data quickly and efficiently. With its distributed nature of employing a metadata server (CommServe) and loosely coupled data mover installations referred to as MediaAgents (MA), the Commvault product can control operations against storage devices, scaling up to petabytes organically as business needs grow.

With Commvault’s lineage of operating system support for MediaAgents and with new advancements in the S3 protocol and immutable backups (WORM), plus innovation from VAST, the solution is well positioned to drive resilience from cyber-attack, coupled with exceptional backup and restore performance with minimal effort, protecting a customer’s most important asset, their data.

Overview

With cybersecurity at the forefront of today’s data world, immutable backups with S3 Object Lock and WORM are at the center of minimizing breaches and downtime.

Commvault brings extensive application support and data management functionality, and the VAST Data Platform delivers all-flash performance that users require, with unique data reduction technology that brings TCO below that of many spinning disk-based backup solutions. Together, it is a best-in-class cyber resilient backup solution.

This is the third revision of this guide, and it covers some new content around VAST’s S3 Object Lock and Commvault’s WORM, along with many updates. The first half of the document will discuss technical concepts to ensure that the combined solution can achieve the highest performance and maximum capacity usage needed for a user’s specifications. The second half of the document highlights specific configuration steps to ensure the solution is setup properly.

For a quick high-level synopsis, there is a key takeaway section next, followed by a more In-depth examination of the aforementioned topics.

Key Takeaways

This paper will describe how Commvault users should configure their Commvault software to best operate with VAST. It outlines technical details, architectural considerations, and recommended best practices when combining VAST Data’s Universal Storage with the Commvault data protection suite. For a quick summary of the important takeaways, here is the fast list:

Encryption

• To maximize data reduction rates, allow VAST to handle encryption with encryption over the wire and at rest.

Compression and deduplication

• Enable Commvault client-side deduplication to maximize data reduction, which minimizes network traffic and accelerates both backup and restores.

• VAST recommends the Commvault default LZO compression scheme where possible.

• VAST Deduplication and Similarity

  • Provides additional capacity savings and complements Commvault’s compression and deduplication.

  • Significantly reduces storage usage across similar datasets – when a DDB is sealed, or WORM mode is enabled.

• VAST can provide significant data reduction between native formats and the backup copy format stored on the same VAST cluster, enabling additional use cases such as disaster recovery and high availability.

• Deduplication Database - When configuring VAST as a cloud library, Commvault defaults to using a 512KB block size for deduplication. This can be used if VAST is the only copy or if there are secondary copies to the public cloud (such as Commvault Air Gap Protect). To maximize Commvault’s deduplication efficiency, a 128KB block size is recommended, especially if there are secondary copies on other storage devices.

Immutability

• Leveraging VAST S3 object lock and Commvault WORM mode is supported for backup copies which can be used as part of a cyber recovery solution. Compliance mode immutability is fully supported between Commvault & VAST.

• VAST can deduplicate and similarity reduce across Commvault backup cycles when using WORM mode (DDB seal) for storage immutability. This unique capability removes the WORM lock storage penalty and allows larger data sets to be locked for longer periods of time, saving on requiring 2.5x to 3x extra back-end storage.

MediaAgents

• VAST recommends using nConnect with Linux MediaAgents to maximize throughput to/from the VAST cluster if the NFS protocol is leveraged.

Overall Solution

• Cost-effective: The combination of data reduction and reduced infrastructure leads to lower solution costs and increased overall TCO.

• Scalability: The Storage Accelerator model distributes the data movement and processing across the clients to provide horizontal scaling, minimizing the quantity of MediaAgent servers, while the VAST architecture can scale up capacity to the largest enterprise needs.

• Complementary data reduction technologies: Commvault’s client-side deduplication and compression accelerate data operations and minimize data movement across the network, while VAST can further reduce the backup footprint and copies of data on the storage, synergistically minimizing space consumption.

Best Practices

This section expands on some of the key takeaways by investigating and examining several areas of performance characteristics and how best to configure them. The first area is compression and deduplication, which is of the utmost importance for both network bandwidth and storage capacity considerations.

Before discussing those, it is important to understand encryption on the VAST cluster, Commvault encryption, and how it can impact deduplication.

Encryption

When considering a solution with Commvault and VAST, one needs to look at balancing security, performance, and capacity savings. There are many settings within Commvault and VAST that can help optimize for these.

VAST cluster provides encryption-at-rest and encryption on the wire. VAST encryption-at-rest uses FIPS 140-2 Level 1 ciphers and encrypts both the data and the metadata. Encryption at rest currently needs to be enabled at the time of cluster creation; however, a later release provides the ability to enable this feature on existing clusters that did not initially enable it.

Commvault also has an option to encrypt the data it is protecting. The decision to enable Commvault encryption can have performance and capacity downsides. Enabling the application-side encryption on Commvault will result in zero data reduction on the VAST side, which is reflected in a Data Reduction Rate (DRR) of one-to-one (1:1). Also, client-side encryption may affect the MediaAgent’s performance as it requires additional CPU to encrypt each block. The performance penalty can be overcome by beefier hardware for the MediaAgent servers, but the capacity savings that would be lost cannot be overcome.

With Commvault’s quality data reduction technologies, by way of deduplication and compression, traffic across the wire is minimized, and for DASH copy replication, which is why VAST encourages our customers to implement these. However, when the encrypted data is sent to VAST, the overall space savings will be much lower than when VAST is chosen to manage the encryption. By choosing to encrypt at the VAST layer, you will benefit from both Commvault’s data reduction to create a smaller data set, reduce overall network traffic and benefit from VAST’s advanced data reduction techniques without compromising the overall security of your data.

There are scenarios in which Commvault may need to duplicate data even when deduplication is desired. Some of these scenarios include recovering from a missing or corrupted DDB, operations such as sealing the DDB, or adjusting the block size of the Storage Pool or Storage Policy. In these cases, a full copy of data needs to be completely rewritten upon the next backup. The result is that 100% of the base backup set size is sent to VAST, but with Commvault’s encryption disabled, the net impact to usable capacity is essentially zero with VAST’s advanced data reduction capabilities.

However, if Commvault encrypts the data, VAST cannot reduce it, and the result is a doubling of used capacity until previous restore points age out and are deleted. More on DDB best practices is discussed in the Compression and Deduplication section.

You may still choose to perform the encryption within Commvault, and VAST is still an excellent storage platform to store Commvault-encrypted data. Even if VAST’s deduplication, compression, and similarity savings cannot further reduce the data, the overall solution is still very compelling and is denser, faster for backup, recovery & object maintenance tasks, and requires less power and cooling than any alternatives available today. Below are several ways to approach encrypting data based on the desired level of security.

Encryption - Data In-Flight

To secure data on the network between Commvault MediaAgents and a VAST cluster, it’s recommended to configure the VAST cluster as a cloud library, which will use the SSL protocol for communication. SSL will use TLS encryption on all data in-flight to the VAST cluster. This also applies to clients that utilize the Storage Accelerator, which will move data directly to and from the VAST cloud library via an SSL (TLS) encrypted tunnel. (Note: You may need to add the VAST SSL security certificate to the Commvault CA bundle to enable this configuration – See SSL Certificates for more information)

For the Commvault clients that will back up through a MediaAgent, you can enable encryption on network traffic in the Commvault software to secure the data in transit between these devices. The MediaAgent will then transmit the data via the encrypted SSL tunnel to the VAST cloud library.

Encryption - Data at Rest

If you need to have data encrypted at rest, both Commvault and VAST offer this capability, and you can choose the option that best meets your requirements.

Using VAST encryption: To get the advanced benefits of VAST’s data reduction (Adaptive Chunking, Similarity, etc.), turn off Commvault encryption on media, and ensure encryption is enabled on VAST (All new systems have encryption at rest turned on by default). Combined with the in-flight TLS encryption from the previous section, this will achieve end-to-end security and still allow VAST to perform its advanced data reduction techniques on the non-encrypted data and write it in a secure format.

If additional copies are needed to different non-encrypting storage, enable encryption in Commvault for the auxiliary copy. Commvault will write the data in an encrypted format to the secondary device so that it’s also protected at rest on that copy.

Using Commvault encryption on media: If there are specific security needs, Commvault can encrypt the data and write it to VAST in a secure format. This will require additional compute resources on the Commvault MediaAgents, and VAST will not be able to provide the additional storage reduction since encrypted data will look like random data to VAST.

Allowing VAST to do the encryption at rest removes any performance hit on the Media Agents and allows for storage-side data reduction. The SSL connections to VAST leverage TLS encryption to secure the data in flight, and the Commvault client to Media Agent using a secure connection completes the end-to-end security.

Allowing VAST to do the encryption removes any performance hit on the MediaAgents and maximizes capacity savings.

Compression and Deduplication

Commvault and VAST data reduction techniques are complementary to each other, and using them both simultaneously provides an overall best practice for both network efficiency and storage capacity considerations. By utilizing both technologies, it maximizes space savings while optimizing network efficiency. This section examines the various components of Commvault’s built-in data compression and deduplication configuration options with VAST’s unique data reduction techniques.

Compression

By default, Commvault uses LZO compression and is very effective at getting the most out of compressing data. Tests performed previously looked at this, and an older compression type, and LZO was by far the best at maximizing space savings and minimizing MediaAgent compute usage, which equates to faster backups.

Since LZO is the default setting, this is an ‘easy button,’ and nothing needs to be configured to achieve the best results.

Deduplication

The VAST Data Platform’s unique deduplication techniques, including ada¬ptive chunking and similarity, reduce misaligned data well and can give additional reduction savings on top of Commvault’s already deduplicated data. There might be some desire to turn off Commvault deduplication and compression to minimize compute resources, but the trade-off with that is increased bandwidth usage on the local network, as well as the increased data needing to be sent to VAST. All of this will lead to longer backup times. The perfect balance between compute and network usage and backup intervals is to allow both Commvault and VAST to perform deduplication and compression.

Another advantage to the VAST and Commvault solution is readily apparent when using WORM or when the deduplication database (See - DDB Block Size) needs to be sealed independently of WORM. When a DDB is sealed, a new baseline full backup is run with a brand new DDB, so all the data is now considered new and therefore written again to storage. This typically costs 2.5x or more of total storage capacity. However, due to VAST’s superior deduplication, it has essentially no capacity cost and, in tests, shows only 1-3% additional storage consumed.

VAST’s byte level adaptive chunking and similarity on data written to VAST maximizes storage efficiency in addition to Commvault deduplication alone.

Additionally, if WORM mode is enabled, or for some reason the DDB needs to be sealed and a new set of baseline full backups is run, the VAST Data Platform can deduplicate the new set of data against the older set, leading to reduced storage consumption.

Client-side Deduplication and Compression

Enabling deduplication and compression on the client-side minimizes the data sent over the local network and to the VAST cluster, like MediaAgent-based processing, but with the potential of reducing backup and restore times as the process is now distributed across each individual client.

The same deduplication and compression technology that is on MediaAgents now runs on the client, and the VAST cluster will still further deduplicate, compress, and reduce with similarity the data written to storage, maximizing storage efficiency.

DDB Block Size

Another aspect of Commvault’s deduplication is the Deduplication Database (DDB) that it uses to do hash lookups of file and object data to determine if they have already been written to storage. If the hash of an object or file is in the DDB, then the data is not sent again to VAST. This not only minimizes the data sent from the MediaAgents to VAST but also frees up local network bandwidth from clients to the MediaAgents.

When configuring VAST as a cloud library, Commvault defaults to using a 512KB block size for deduplication. This can be used if VAST is the only copy or if there are secondary copies to the public cloud (such as Commvault Air Gap Protect).

To maximize Commvault’s deduplication efficiency, a 128KB block size is recommended for high-speed local disk storage. 128KB will give better deduplication ratios for Commvault (less data moved through the network) and more granular data aging to minimize storage bloat. 128KB should be used if there are copies on other storage devices, so that the storage efficiency is maximized across all copies.

Solution Design

With VAST’s unique all-flash DASE (Disaggregated Shared Everything) platform, the architecture of the backup environment plays a crucial role and must be considered in the design. Identifying the lowest common denominator in the data flow is essential as it determines the limiting factor.

When considering restore scenarios, the VAST platform will usually not be the bottleneck as its current restore speeds are almost 8x faster than writes, so the target system’s write speeds and network links should be considered first.

Considerations:

• VAST currently has 8x read speeds at almost 40GB/s, compared to write speeds (write speeds will increase 2x or more in subsequent releases).

• The network architecture may become a bottleneck if it’s not designed for the high-performance restore speeds of the VAST platform. VAST provides multiple 100 Gbit/s NIC’s for connectivity to the customer network.

• The disks on the clients will always be a major consideration for restore performance when considering Recovery Time Objectives (RTO).

Solution Sizing

MediaAgent sizing

The network architecture is important to consider in the overall solution design to make sure that data can be moved fast enough to VAST. MediaAgents can be sized using the Commvault specifications for deduplication mode with the back-end sized for cloud storage. Since VAST is a high-speed solution, the MediaAgent should include fast interconnects so that it can get the full benefits of the cluster, as this connection will typically be the bottleneck of the solution.

Sizing for WORM storage

On typical object storage that lacks duplication but supports the S3 Object Lock API on a per-object level (or equivalent), enabling WORM will result in approximately a 2.2x to 2.5x increase in storage utilization. Two baselines of data must be retained on the device, along with some overhead and potential cycle overlap. Based on testing results, VAST can achieve 96% or more of deduplication across two baselines, reducing the cost of using WORM with VAST to just a few percentage points, lowering storage utilization to around 1x.

The result is that WORM mode becomes cost-effective. This not only reduces the cost of utilizing this feature but also enables customers to retain larger datasets for longer periods, which may not be economically feasible with other storage systems.

Commvault Storage Accelerator

The Commvault Storage Accelerator can be used on clients to directly send the backup data to the VAST cloud storage library (sometimes referred to as Client-Direct mode). Without the Storage Accelerator, backups send the client data to the MediaAgent first, and then the MediaAgent stores the data in the VAST cloud storage library. When used on many clients, this can achieve high aggregate throughput for both backup and restores while lowering MediaAgent requirements. Storage Accelerator does consume more resources on the client, so resource allocation should be considered on systems that are heavily loaded. The Storage Accelerator configuration has the following advantages:

• Speeds up the backup and restore process by directly backing up and restoring from the cloud storage library and avoiding additional network transmission.

• Reduces the load on the cloud MediaAgent by eliminating the need for processing backup data. (Pruning, Data Verification (DDB Lookups), and defrag operations would continue to be processed by the destination MediaAgent.)

Any client that has the Storage Accelerator installed and has a backup policy configured to a cloud storage library will automatically try and use the direct mode, and if the connection to the storage fails, it will fall back to sending data through the MediaAgent instead. The same SSL certificate requirements and additional settings that are required by the MediaAgent would also be needed on each client (See Appendix - General Configuration for more information).

The Storage Accelerator model distributes the data movement and processing across the clients without sending data through the MediaAgent to provide horizontal scaling.

Space Reclamation

With VAST configured as cloud storage (in your DC or in a private cloud DC), Commvault space reclamation can be run regularly to compact the backup files and increase storage efficiency. And due to VAST’s performance and IO capabilities, these tasks run swiftly without impacting regular backup schedules.

Conclusion

The combination of Commvault software and the VAST all-flash platform creates a secure, cyber-resilient data protection solution that harnesses the technology of both vendors synergistically.

The data reduction capabilities of VAST and Commvault are complementary. Commvault will compress and deduplicate data, minimizing local network traffic and speeding up backups, and then VAST will further reduce Commvault’s post-deduplicated and post-compressed data by up to an additional 2x. Almost cost-free data reduction is also achieved between backup and native formats and opens additional use cases in disaster recovery, high availability, and test/dev operations.

Extraordinary capacity savings are also seen between backup cycles when sealing a DDB in conjunction with using Commvault’s WORM functionality, where VAST’s duplication nullifies a typical 2.2x - 2.5x capacity requirement down to 4% or less and allows for larger data sets to be securely stored for extended periods of time.

The solution is best-in-class, offering flash-speed backups and unrivaled restore performance. Additionally, it offers economic advantages by reducing infrastructure requirements, thereby improving the total cost of ownership. It is particularly well-suited for large-scale environments, thanks to Commvault's scalable infrastructure and VAST's DisAggregated Shared Everything (DASE) architecture. This joint solution is fully integrated and operational without the need for complex configurations, providing robust cyber-resilient data management and protection capabilities.

Appendix - General Configuration

This section covers the integration between VAST and Commvault. The goal is to cover as many critical configuration steps as possible to ensure the implementation can be up and running quickly.

VAST Configuration

During the process of adding an S3 backup repository, S3 credentials will be needed, so creating an S3 user is discussed first in the next section.

Creating an S3 User

To create an S3 user with the VAST UI, go to the Dashboard and then User Management and select Users (Figure 1).

Figure 1 - Selecting Users Category

The Add User window appears (Figure 2), and a username needs to be entered along with any desired UID. This user was not granted full S3 credentials because they are not required during the Commvault process of adding an S3 repository.

Figure 2 - Creating a New User for S3

After clicking Create, the user will show up in the User Management window (Figure 3).

Figure 3 - User Created

Now that the user has been created, it needs to be edited to generate and capture the active and secret keys. On the far right of the user under the action column, click on the three dots and select Edit (Figure 3).

In the Update User window (Figure 4), click on the Create new key button to create a new Access Key. Now copy the active key and the secret key somewhere for later. This is the only time when the secret key will be obtainable, so be sure to copy it down in some safe location. Click Update to close the window.

Figure 4 - Capturing Active and Secret Keys

The user can now be used in the creation of a bucket or view within the VAST UI. For S3 implementations with Commvault and Identity policy is also needed and discussed next.

S3 Identity Policy

To ensure that VAST and Commvault S3 interoperate correctly, it’s necessary to apply all appropriate S3 permissions to the bucket owner.

Commvault’s required S3 permissions are:

• "s3:CreateBucket",

• "s3:GetBucketLocation",

• "s3:GetObject",

• "s3:PutObject",

• "s3:PutObjectRetention",

• "s3:PutObjectTagging",

• "s3:ListBucket",

• "s3:ListAllMyBuckets",

• "s3:DeleteObject",

• "s3:DeleteObjectVersion",

• "s3:ListBucketVersions",

• "s3:PutBucketObjectLockConfiguration"

To apply these to the bucket owner, first create an identity policy by going to Identity Policy under User Management (Figure 5).

Figure 5 - User Management IdentityPolicies

Then click Create Policy and ensure the following JSON code is inserted:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:CreateBucket",
                "s3:GetBucketLocation",
                "s3:GetBucketVersioning",
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectRetention",
                "s3:GetObjectRetention",
                "s3:PutObjectTagging",
                "s3:ListBucket",
                "s3:ListAllMyBuckets",
                "s3:PutObjectLegalHold",
                "s3:GetObjectLegalHold",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
                "s3:PutBucketObjectLockConfiguration",
                "s3:GetBucketObjectLockConfiguration",
                "s3:AbortMultipartUpload",
                "s3:ListBucketMultipartUploads"
                ],
            "Resource": "*"
        }
    }
   ]
}

Note: The above code has some additional S3 permissions beyond Commvault’s requirements to ensure complete coverage.

Once done, the Policy should look like the one in Figure 6.

Figure 6 - Identity Policy

Note that the User List shows the users who have this policy applied. To add a user to the policy, that user needs to be modified. To do that, click on the Users tab, which is right next to Identity Policies (Figure 5). Find the user that will be assigned as the owner of the Endpoint (Each new bucket in the Endpoint will inherit the Endpoint properties – See Creating an S3 Endpoint on VAST) and edit it by clicking on the dots to the right. In Figure 7, user cvlt-TMPHX10 is being edited. In the pull-down menu for Identity Policies, the CVLT-ObjectLock policy has been selected, which contains all the proper S3 permissions. Simply click on Update to complete the edit.

Figure 7 - Update User with Identity Policy

After completing these tasks, an appropriate Cloud Storage pool and policy can be created within Commvault (See - Configure Commvault with VAST S3).

Creating an S3 Endpoint on VAST

A good best practice for organizational purposes and ease of use with creating buckets within the Commvault workflow is to create an endpoint at the root of where buckets would be placed. Once an endpoint is established, buckets created with the same user and secret key will be placed in an orderly manner beneath that endpoint.

Buckets can be created within the VAST UI, the Commvault workflow, or through an S3 utility like S3 Browser®.

To create an endpoint, go to the Element Store within the VAST UI, select the Views tab, and then click Create View as highlighted with the upper right box in Figure 8.

Figure 8 - Creating an S3 Endpoint on VAST

Figure 9 - S3 Endpoint General Details

This will open an Add View wizard as shown in Figure 9. Here, the process is similar to bucket creation, where a path is entered, a policy is given, but only the Endpoint protocol is selected. (Don’t forget to slide the Create Directory toggle over). On the S3 tab (Figure 10) of this wizard, ensure the appropriate user is given access to create buckets. Click Create when finished. This will show the endpoint as in Figure 8.

Figure 10 - S3 Endpoint S3 Details

Again, a handy feature of having an endpoint is that when using any S3 utility, along with the same user, it will propagate all parameters according to the endpoint and place buckets cleanly under the endpoint path.

For example, the endpoint that was just created here has a path of /commvault. A new bucket created with the name – bucket1 will show up like this in the VAST UI: /commvault/bucket1. This becomes especially useful when creating the bucket within the Commvault Cloud Library workflow or other S3 utilities that allow bucket creation.

Creating an S3 Bucket

During the Commvault library creation workflow, a VAST bucket can be configured without having to use the VAST UI. If a user wants to create one ahead of time, this section will describe those steps. To create a bucket, go to the dashboard and go to Element Store as shown in Figure 11, and then click Views.

Figure 11 - Selecting Views to Create a Bucket

This brings up the Element Store window with the Views tab selected. In the upper right corner, select Create View (Figure 12).

Figure 12 - Select Create View

This view is being configured as a bucket only using the S3 Bucket protocol. Multiple protocols (NFS, S3) can be used on a view. Figure 13 shows all the settings needed to create the new bucket. Since this is a new view, the create directory toggle is selected.

Figure 13 - S3 Bucket Configuration

The user previously created needs to be given access or ownership of this view. This is done on the S3 tab as shown in Figure 14. When finished, click Create from either tab.

Figure 14 - Adding Bucket Owner (User)

The view (bucket) now shows up in the Element Store as shown in Figure 15.

Figure 15 - S3 Bucket Created

With the bucket now created, a cloud storage pool using this bucket can be created within the Commvault UI.

Enabling S3 Object Lock

When the intention is to use Commvault WORM, object lock will need to be enabled on the bucket within the VAST UI. During the creation of a Commvault cloud storage library (See - Configure Commvault with VAST S3), a VAST bucket is either created within that workflow or manually, as shown in the previous section. Once the bucket has been chosen or created, then that is the time to enable object lock.

To do that, start by selecting the Element Store and finding the appropriate bucket, and click on the three dots on the right to edit (Figure 15).

Within the bucket on the General tab, ensure that this is a bucket and not an endpoint (Figure 16).

Figure 16 - Editing Bucket General Tab

Click on the neighboring S3 tab and toggle S3 object lock on (this will also turn on S3 Versioning). The key part here is the Retention Mode setting.

Figure 17 - Editing Bucket S3 Tab

Since Commvault will be managing all aspects of the locked objects and their corresponding retention, ensure that the Retention Mode (Figure 17) is set to None. Enabling object lock at the time of cloud creation ensures that it is not overlooked, as backups will fail without it.

SSL Certificates

One of the advantages of S3 is that it can use TLS encryption for the data in-flight from either the MediaAgent or a client with Storage Accelerator. Combine that with VAST’s encryption at rest and Commvault’s network encryption, and you have an end-to-end solution without encrypting the data (See - Encryption). This allows for VAST to give additional deduplication and compression savings on top of Commvault.

To be able to properly use an encrypted connection to a VAST cloud library over S3, an SSL Certificate will need to be generated and added to any client or MediaAgent connecting directly to VAST.

The first step is to create an SSL certificate for the VAST cluster. There are several different certificates that are allowed on the system (Figure 19), but this one will be needed for S3. The same certificate could be used for all the categories if desired.

The following sections do not cover the creation of the SSL certificate – Please consult the appropriate team within your organization to acquire an SSL certificate. Ensure that you are provided with two files: a certificate file and a key file, and that they are in an X.509 output file format containing ASCII (Base64) encoded data.

To make uploading the document easy, open the 2 files and ensure there is only content from:

[----Begin Certificate ]

… through …

[----End Certificate ]

Once an SSL certificate is acquired and the files have been inspected, then the following steps can be used to install them into VAST and within Commvault.

Installing SSL Certificate into VAST

From the VAST UI and from the left menu, select Settings and Certificates as shown in Figure 18.

Figure 18 - Select VMS From Menu

Within the Certificates window, ensure that S3 has been selected as shown in Figure 19.

Figure 19 - Selecting S3 Certificate Definitions

Now upload or paste the certificate contents into the left section – appropriately labeled Certificate (Figure 20).

Figure 20 - Paste Certificate and Private Key

Next, upload or paste the .pem file (private key) into the Key section within VMS (Figure 20). To save the settings, click Update.

Installing SSL Certificate in Commvault

The easiest approach is if the Commvault server (Windows) is a member of the domain from which the VAST S3 certificate was issued, because it will already have the root CA included as a Trusted Root Certification Authority for User, Local Computer, and Services certificate stores. So, if the Commvault server (Computer) is already part of the same domain as the VAST cluster, then by default, it’s trusted, and configuring secure cloud storage will work without issues.

This functionality is not currently available for Linux with Commvault releases through v11.34. Starting in Commvault v11.36, it will introduce domain-level certificates. Previous versions will need to have each MediaAgent and client manually modified.

If not using the domain method, then the curl-ca-bundle.crt file must be modified manually for both Windows or Linux MediaAgents and clients for SSL communication to function properly. This file is in the Commvault Base folder. (Before you modify the file, it’s a good idea to back up the file and verify the host isn’t currently running any backups.)

In Windows, the file is located under

*\Commvault\ContentStore\Base

and in Linux under

*/Commvault/Base or

*/Commvault/Base64

Simply copy the content of the certificate contents (not the private key) into the file and save it. If there are still issues, this knowledge base article can help resolve them - Fixing SSL Certificate Problems

Once the VAST cluster has a proper certificate installed and the client/MediaAgents have had their certificate filed updated SSL communication should function seamlessly.

Configure Commvault with VAST S3

Configuring a cloud-based storage pool with VAST is an easy task. The methodology differs slightly beginning in Commvault version 11.36. In that release, VAST has been integrated into the workflow and is available as a pull-down menu option. To begin, from Commvault Command Center, in the left column, click Storage and then select Cloud as shown in Figure 21.

Figure 21 - Creating a Cloud Storage Pool

Now, on the Cloud page, click the Add button in the upper right (Figure 22).

Figure 22 - Add Cloud Storage

This will bring up the Add Storage Cloud window (Figure 23). The first entry to select is the object storage type – use the pull-down menu to select VAST Data Platform (v11.36+) or S3 Compatible Storage (Figure 24) if the choice for VAST is not listed . Select a MediaAgent that will be used as the data mover. In the Service Host field, ensure the entry includes http:// or https:// for it to effectively communicate with the VAST cluster.

Figure 23 - Add VAST Data Platform (FR 11.36+)

The Credentials setting is the user who was set up and assigned as the bucket owner on the VAST cluster. Enter in the name of the bucket to be used – this will either use a preconfigured bucket or create it on the fly. This cloud storage example is also using a DDB, which can be configured by clicking Add. This brings up the configuration window as shown in Figure 24. Enter a MediaAgent and location for the DDB database and click Add.

Figure 24 - Add VAST as S3 Compatible Storage (Pre FR11.36)

The Credentials setting is the user who was set up and assigned as the bucket owner on the VAST cluster. Enter in the name of the bucket to be used – this will either use a preconfigured bucket or create it on the fly. This cloud storage example is also using a DDB, which can be configured by clicking Add. This brings up the configuration window as shown in Figure 25. Enter a MediaAgent and location for the DDB database and click Add.

Figure 25 - Deduplication DB Location

Once everything has been properly entered for the Cloud Storage, it should be similar to Figure 26. Simply click Save.

Figure 26 - VAST Cloud Storage

With the cloud storage configured, it should now show in the Cloud window as shown in Figure 27.

Figure 27 - Cloud Storage Configured

Commvault WORM

Within Commvault, you can enable WORM storage mode on the VAST cloud library to provide immutable storage for backups. This feature will leverage S3 object lock to ensure that all backup data is immutable for the duration of the Commvault retention period.

In some combinations of Commvault and VAST, an additional setting is required to allow for proper WORM functionality.

The additional setting needed is - bDisableNativeFileForMacroPruning and has the following parameters:

Description

In V11.30+, when WORM mode is enabled, we write in “native cloud file” format, which defaults to using multi-part uploads.

This may create problems with WORM storage, where multi-part upload is not supported. To revert to the older methods that do not use multi-part upload, set this key to true: bDisableNativeFileForMacroPruning

• Name: bDisableNativeFileForMacroPruning

• Entity: MediaAgent

• Created On: MediaAgent

• Type: Boolean (true or false)

• Value: true

This key must be set for all MediaAgents and Storage Accelerator clients that will access the VAST library directly.

This is done under the Additional Settings tab under the client or MediaAgent properties setting.

Figure 28 - Additional setting necessary for proper WORM functionality

Enabling WORM mode

Under the configuration tab in the cloud library properties, you can click the slider named WORM Storage Lock to enable immutability on the storage (Figure 29).

WORM Storage Lock will leverage the S3 object lock API to set retention on the files in the storage. Compliance lock will also be enabled automatically and will prevent data and job deletion within the Commvault software.

Figure 29 - Enabling Commvault WORM

Configure Commvault with VAST NFS

Create Commvault NFS Storage Pool

Creating an NFS Storage Pool for Commvault is just as simple as creating a cloud storage pool. From Commvault Command Center, in the left column, click Storage and then select Disk, which is just above Cloud, as shown in Figure 21.

Now, on the Disk page, click the Add button in the upper right, similar to what is shown in Figure 22 for Cloud. This will bring up the Add Disk Storage window (Figure 30).

Figure 30 - Adding NFS Disk Storage

Now, enter a name for the disk storage and pick a MediaAgent that has access to the NFS share. In this example, the share is mounted to the MediaAgent and is selected from a browse window (Figure 31).

Figure 31 - Selecting Backup Location

Entering the DDB information is the same configuration method as is done for cloud storage. When finished, it should be like Figure 30. Simply click Save to finish, and the new NFS Disk storage will be as shown in Figure 32.

Figure 32 - NFS Disk Storage Created

The disk can now be used in a Plan or Storage Policy.

For more information on the VAST Data Platform and how it can help you solve your application problems, reach out to us at hello@vastdata.com.