VAST SyncEngine 3.0.2 Release Notes
VAST SyncEngine 3.0.2 is the first official release of VAST SyncEngine.
VAST SyncEngine is a powerful distributed data migration and synchronization platform designed for enterprise-scale data operations. It provides a robust solution for seamless data transfer between various data sources and destinations, while maintaining comprehensive metadata indexing and cataloging capabilities.
VAST SyncEngine consists of a control plane that handles scheduling, coordination, and monitoring, and multiple workers that are deployed across multiple hosts and perform data transfer operations between migration source and destination.
Supported migration sources include POSIX file systems, AWS S3 or VAST S3, Azure blob storage, Confluence, and Google Drive.
VAST SyncEngine data migration process involves the following steps, which can be accomplished using VAST SyncEngine REST API:
Create a connector for the migration source.
A connector represent a storage endpoint. When you create a connector, you supply parameters required for VAST SyncEngine to access the storage through a given endpoint.
Create a connector for the migration destination.
Note
Ensure that the destination is configured properly to accommodate the data.
Define and run the migration.
During the migration, VAST SyncEngine workers perform data transfer and synchronization using the connectors specified in migration parameters.
Monitor migration progress and completion
VAST SyncEngine lets you view detailed real-time migration status on a Grafana dashboard, aggregate migration logs with Grafana Loki, and export migration metrics with Prometheus. In addition, the full range of VAST cluster metrics and analytics can be used to verify migration results on the VAST Data platform.
The requirements are as follows:
A minimum configuration includes three hosts: one for the control plane and two for workers.
Minimum 4GB of RAM, 20GB of free disk space per host
Network connectivity between the control plane and worker hosts, and also to migration source and destination (NFS: port 2049 for TCP/UDP; S3: port 80 for HTTP or 443 for HTTPS)
Rocky Linux v9.5 is recommended; Python 3.8 or higher, Docker or Podman (automatically detected), curl and tar utilities; Docker or Podman installed
Root or sudo access on all hosts; admin access to the VAST Management Service (VMS)
The following limitations apply:
S3 object tags, S3 versioned objects and locks are not preserved.
S3 access control lists are not preserved. It is recommended to use identity policies for access control.
POSIX ACLs are not preserved.
Migration of hardlinks will result in multiple duplicate copies created on the destination. The relationship between hardlinks is not preserved.
For further information and guidelines, see VAST SyncEngine documentation.