Deploy SyncEngine

# Deploy SyncEngine

To deploy SyncEngine:

1. Ensure that SyncEngine host requirements are met.

2. Complete the steps to deploy SyncEngine control plane and UI/CLI.

3. Complete the steps to deploy one or more SyncEngine workers.

Host Requirements

A minimum SyncEngine configuration includes three hosts: one for the control plane and two for workers.

• CPU 4 cores

• Minimum 128GB of RAM for the control plane and 256GB for workers

• 50GB of free disk space per host at /opt/syncengine (or your chosen install path)

• Network: at least 10 GB for the control plane and 100 GbE for workers (recommended)

• Network connectivity between the control plane and worker hosts, and also to migration source and destination

• Rocky Linux v9.5 is recommended; Python 3.8 or higher, Docker or Podman installed, curl and tar utilities

• Root or sudo access on all hosts; admin access to the VAST Management Service (VMS)

• Available ports per Containers and required ports.

Firewall

Open Ports

Ensure the required ports are open on the control plane host.

• On RHEL / Rocky Linux / CentOS:

  1. Ensure that firewalld is running:

     sudo firewall-cmd --state
     

  2. Open the required ports by running the following command for each of the required ports:

     sudo firewall-cmd --permanent --add-port=<port number>/tcp
     

  3. Apply the configuration:

     sudo firewall-cmd --reload
     

  4. Verify by listing open ports:

     sudo firewall-cmd --list-ports
     

• On Debian / Ubuntu:

  1. Ensure that ufw is running:

     sudo ufw status
     

  2. Open ports by running the following command for each of the required ports:

     sudo ufw allow <port number>/tcp
     

  3. Verify by listing open ports:

     sudo ufw status
     

Check iptables Rules

Check for iptables rules that may block traffic:

sudo iptables -L -n

Containers and Required Ports

The following SyncEngine containers run on the control plane host:

The following ports must be open to ensure worker connectivity:

If you are going to use SyncEngine CLI (deployed via a additional RPM), open port:

Deploy Control Plane, UI and CLI

Perform these steps on the host intended for the control plane.

Step 1: Download and Extract Installation Files

Using the links provided by VAST, download and extract the following files for the latest SyncEngine release to your control plane host:

• ms - The install script

• Vastdata_SyncEngine_v<version>.tar.gz - The container images and
  configs, including SyncEngine UI

• syncengine-cli-<...>.<platform>.tar.gz - The SyncEngine CLI
  (optional)

Step 2: Make Install Script Executable

Run to ensure that the install script can be executed:

chmod +x ms

Step 3: Install Control Plane

Run to install the control plane:

./ms install control /<path>/Vastdata_SyncEngine_v<version>.tar.gz /opt/syncengine

If needed, replace /opt/syncengine with the install directory of your choice.

The install procedure:

• Checks that curl, tar, and Podman are available.

• Extracts the SyncEngine bundle contents to /opt/syncengine.

• Validates and loads all SyncEngine container images.

• Starts all control plane services.

Step 4: Verify Control Plane

• Ping the control plane:

  ./ms ping
  

• List SyncEngine containers:

  podman ps --filter label=project=syncengine
  

  The output should include containers listed in Containers and required ports. In case any of the containers is not running, check the logs with podman logs <container_name>.

Step 5 (Optional): Install CLI

• On the control plane host, install the downloaded RPM:

  sudo rpm -Uvh syncengine-cli-<...>.<platform>.rpm
  

Step 6: Log in to SyncEngine UI

In your browser, navigate to:

http://<control plane IP>:18088

Use credentials provided by VAST.

In addition to the UI, the following resources are available:

• Grafana performance dashboards:

  http://<control plane IP>:3009
  

• Raw metrics (Prometheus):

  http://<control plane IP>:19091
  

• API documentation:

  http://<control plane IP>:5009/docs
  

Set up TLS for SyncEngine UI

Complete these steps to enable TLS for the SyncEngine UI web server using a self-signed certificate or a CA-issued certificate.

When TLS is enabled, the server listens on two ports:

• HTTP port 8888

• HTTPS port 8443

1. Create the ssl directory:

   sudo mkdir -p /etc/ssl/syncengine
   sudo chown root:syncengine /etc/ssl/syncengine
   sudo chmod 750 /etc/ssl/syncengine
   

2. Provide a certificate:

   • If you have a CA-issued certificate, copy the certificate and the private key to the ssl directory:

     sudo cp your-certificate.crt /etc/ssl/syncengine/server.crt
     sudo cp your-private-key.key /etc/ssl/syncengine/server.key
     

   • If you want to use a self-signed certificate, generate the server.crt and server.key files under /etc/ssl/syncengine/:

     sudo syncengine-cli tls generate --hostname $(hostname -f)
     

     If the UI is to be accessed via multiple hostnames or IPs, add more hostnames, for example:

     sudo syncengine-cli tls generate \
       --hostname $(hostname -f) \
       --san "localhost,192.168.1.100,syncengine.example.com"
     

3. Set certificate permissions:

   sudo chown root:syncengine /etc/ssl/syncengine/server.key
   sudo chmod 640 /etc/ssl/syncengine/server.key
   sudo chmod 644 /etc/ssl/syncengine/server.crt
   

4. Edit /etc/default/syncengine-cli so that it includes these lines:

   SYNCENGINE_CLI_WEB_PORT=8888
   SYNCENGINE_CLI_TLS_ENABLED=true
   SYNCENGINE_CLI_HTTPS_PORT=8443
   

5. Restart the UI service:

   sudo systemctl daemon-reload
   sudo systemctl restart syncengine-ui
   

6. Check the UI status:

   sudo systemctl status syncengine-ui
   

   The output should indicate that both HTTP and HTTPS are running:

   Web UI started successfully!
      HTTP:   http://localhost:8888
      HTTPS:  https://localhost:8443
      Certificate expires in 365 days (2027-05-05)
   

7. Verify that TLS is working:

   • Check service logs:

     sudo journalctl -u syncengine-ui -n 50
     

   • Test the HTTP endpoint:

     curl http://localhost:8888/api/webuihealth
     

   • Test the HTTPS endpoint:

     curl -k https://localhost:8443/api/webuihealth
     

     Note: The -k flag is required only if you're using a self-signed certificate.

   • Check TLS status:

     curl http://localhost:8888/api/tls/status
     

     The response should be similar to:

     {
       "tls_enabled": true,
       "https_port": 8443,
       "certificate": {
         "subject": "your-hostname",
         "issuer": "Self-Signed",
         "days_remaining": 365,
         "status": "valid"
       }
     }
     

8. Access the SyncEngine UI in your browser using HTTPS:

   https://<control plane IP>:8443
   

   If using a self-signed certificate, the browser will display a security warning.

Troubleshoot TLS

Service won't start

sudo journalctl -u syncengine-ui -n 100 --no-pager

Certificate file not found

Check if the certificate files exist:

ls -la /etc/ssl/syncengine/

Regenerate the files if needed.

Private key is world-readable

Update the permissions:

sudo chmod 640 /etc/ssl/syncengine/server.key
sudo chown root:syncengine /etc/ssl/syncengine/server.key

HTTPS port already in use

1. Check which service is using port 8443:

   sudo ss -tlnp | grep 8443
   

2. Either stop the conflicting service or change the HTTPS port. For example, if the CLI was installed as a RPM:

   echo "SYNCENGINE_CLI_HTTPS_PORT=8444" | sudo tee -a /etc/default/syncengine-cli
   sudo systemctl restart syncengine-ui
   

Certificate expired

Check expiration:

syncengine-cli tls info /etc/ssl/syncengine/server.crt

Regenerate the files if needed.

Deploy Worker

Perform these steps on the host intended for a SyncEngine worker.

Step 1: Copy SyncEngine Bundle and Install Script to Worker Host

Copy the two files extracted from the SyncEngine bundle (ms and Vastdata_SyncEngine_v<version>.tar.gz ) to the worker host.

Step 2: Make Install Script Executable

Run to ensure that the install script can be executed:

chmod +x ms

Step 3: Install Worker

Install worker images:

./ms install worker /<path>/Vastdata_SyncEngine_v<version>.tar.gz /opt/syncengine

If needed, replace /opt/syncengine with the install directory of your choice.

Step 4: Deploy Worker

Deploy the worker:

cd /opt/syncengine
WORKER_LABEL=<label> CONTROL_IP=<control plane IP> ./ms deploy worker

The deploy command includes worker deployment variables:

• WORKER_LABEL=<label> sets a text string that lets you associate the worker with a particular migration task. You assign this label when configuring a migration.

• CONTROL_IP=<control plane IP> sets the IP of the host where the control plane is deployed.

Tip: For more worker deployment variables, see Worker deployment variables.

Tune Workers

The worker can be tuned for a particular type of workload:

• Many small files

  Set the PROCESS_COUNT variable to a higher value so that all threads are kept busy (e.g., 80-128).

• Large files

  Set the PROCESS_COUNT variable to a lower value to avoid memory pressure and let each thread move data efficiently (e.g., 16-32).

• Mount points outside of /mnt

  Supply a comma-separated list of mount paths on the VOLUMES variable.

For example, to migrate a very large number of small files across multiple mount points:

VOLUMES='/mnt,/data' PROCESS_COUNT=96 WORKER_LABEL=bulk-migration CONTROL_IP=<IP> ./ms deploy worker

Step 5: Verify Worker State

• Check the container:

  podman ps | grep worker
  

• Check the logs:

  podman logs -f syncengine-worker
  

  You should see that the worker has registered with the control plane and started polling for jobs.

Worker Deployment Variables

Restart Control Plane

Complete these steps if you need to reboot the control plane host or restart all SyncEngine services:

1. Stop all containers:

   cd /opt/syncengine
   podman-compose -f podman-compose.yaml stop
   

2. Reboot the host (if needed):

   sudo reboot
   

3. After the host is online, bring the services back up:

   cd /opt/syncengine
   ./ms deploy control
   

4. Verify the control plane is reachable:

   ./ms ping
   

5. Verify that all containers are up and running:

   podman ps --filter label=project=syncengine
   

Uninstall SyncEngine

Use the SyncEngine install script (ms) to uninstall the control plane. workers, and the UI/CLI RPM.

Uninstall Control Plane and UI

On the control plane host, run from your install directory (/opt/syncengine/):

./ms uninstall

The install script removes all SyncEngine containers, images, volumes and networks, and verifies the cleanup.

Uninstall Worker

In the worker host:

• Stop the worker container:

  podman stop syncengine-worker
  

• Remove the container:

  podman rm syncengine-worker
  

• Remove the worker image and install directory:

  cd /opt/syncengine
  ./ms uninstall
  

Uninstall CLI RPM

On the host where the CLI RPM was installed:

sudo rpm -e syncengine-ui