--- title: "Migrate from POSIX to POSIX" slug: "migrate-from-posix-to-posix-34" updated: 2026-10-24T13:46:13Z published: 2026-10-23T15:28:04Z canonical: "kb.vastdata.com/migrate-from-posix-to-posix-34" --- > ## Documentation Index > Fetch the complete documentation index at: https://kb.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Migrate from POSIX to POSIX ## Overview The POSIX connector copies files between local POSIX paths that are already mounted and visible on SyncEngine workers. SyncEngine walks the source directory tree locally and copies with `rsync`. Use POSIX connectors when source and destination are ordinary filesystem paths on the worker (local disk, bind mounts, or NFS client mounts that the OS already mounted). The following capabilities are supported for POSIX-to-POSIX migrations: - To determine when data synchronization occurs: - [Periodic Resync](/documentation/docs/migrate-from-posix-to-posix-34#periodic-resync) - To manage which items are being synchronized: - [Exclude List](/documentation/docs/migrate-from-posix-to-posix-34#exclude-list) - [Mirror Deletions](/documentation/docs/migrate-from-posix-to-posix-34#mirror-deletions) Complete these steps to set up migration between POSIX paths on a worker host: 1. Ensure that the [prerequisites](/documentation/docs/migrate-from-posix-to-posix-34#before-you-begin) are met. 2. [Create](/documentation/docs/migrate-from-posix-to-posix-34#create-posix-connector) two POSIX connectors. 3. [Create](/documentation/docs/migrate-from-posix-to-posix-34#create-migration) a migration. 4. [Start](/documentation/docs/migrate-from-posix-to-posix-34#start-migration) the migration. > **Note:** This step is only required when the migration does not have periodic resynchronization enabled. 5. [Monitor](/documentation/docs/migrate-from-posix-to-posix-34#monitor-migration) migration progress and completion. 6. If necessary, [troubleshoot](/documentation/docs/migrate-from-posix-to-posix-34#troubleshoot). ## Features ### Periodic Resync In addition to starting a migration once it is created, SyncEngine can be configured to automatically run a resynchronization at user-supplied intervals. - In SyncEngine UI, set the **SyncMode** to **Automatic** and enter an interval (in hours) in the **Sync Interval** field. - In SyncEngine CLI, include the `--sync-mode automatic=true` and `--sync-interval <interval>` options when submitting the `migration create` command. For migrations set to **Automatic** synchronization mode, the first synchronization starts immediately after the migration is created, and the subsequent synchronizations are performed periodically with the specified interval. A subsequent synchronization cannot be triggered manually. To be able to manually start a synchronization on demand, set **SyncMode** to **Manual**. ### Mirror Deletions You can determine how SyncEngine handles the items in the source store that have been deleted since the previous migration run. By default, SyncEngine does not delete such items from the destination. If you want to have them deleted, configure the migration to mirror source deletions at the destination: - In SyncEngine UI, select the **Mirror source deletions at destination** option when creating a migration. - In SyncEngine CLI, include the `--enable-deletes=true` option when submitting the `migration create` command. > **Note:** Deletion of items at the destination cannot be undone by disabling this setting. ### Exclude List You can supply a list of files or directories to be skipped during migration. The list can contain file paths or wildcard patterns, one path or pattern per line. Supported wildcards are: `*` (zero or more characters), `?` (one character), and `[seq]` (one character from the sequence or range). The wildcards can appear anywhere in the path. For example: ``` /mnt/source/projects/logs/*.log /mnt/source/projects/data/temp/* /mnt/source/projects/reports/image[1-2000].jpg ``` Do not add trailing slashes to directory paths, e.g. specify `/path/dir` (but not `/path/dir/`). Common patterns include: | Pattern | Effect | | --- | --- | | `/path/to/file.txt` | Exclude the specified file | | `/path/to/dir` | Exclude the directory and its entire subtree | | `/path/to/dir/*` | Create an empty directory but exclude anything within that directory | | `/path/to/dir/*.tmp` | Excludes matching files in that directory | To supply an exclude list: - In SyncEngine UI, upload the exclude list file in the **Exclusion list** field when creating a migration. - In SyncEngine CLI: - Include the `--exclusion-file <file path>` option on the `migration create` command, or - Run this command after the migration is created: ``` migration exclusion-file upload --migration-id= ``` The exclude list file must use the UTF-8 encoding. The following limitations apply: - An exclude list can contain up to 10,000 lines. - Comment lines (#) are not supported. ## Before You Begin Ensure that the prerequisites are met: ### SyncEngine - The SyncEngine control plane and at least one worker are up and running. - At least one worker label of a deployed worker is known. (The worker label is set when you deploy the worker.) - The source and destination paths exist on all workers labeled for a particular POSIX migration. A worker must have read access to the source path and read/write access to the destination path. ## Create POSIX Connector A POSIX connector is used for POSIX filesystem migrations. ### SyncEngine UI 1. In the **Connectors** page, click **New Connector**. A wizard dialog appears to walk you through the connector creation steps. 2. In the **Type & name** step: - Select the **POSIX** connector type. - In the **Connector name** field, enter a unique name for the connector. The name can contain up to 63 characters, including uppercase and lowercase English letters, digits, periods (.), hyphens (-) and underscores (_). 3. In the **Tunables** step: - Under **Content Types Supported**, determine whether the connector will handle data and metadata, only data, or only metadata. By default, the connector handles both data and metadata. - Under **Batch & Scan Tunables**: - Optionally adjust the size of batches that SyncEngine uses when copying the data. By default (if no values are entered in the fields provided), SyncEngine uses VAST cluster's defaults which are suitable for most workloads. - In the **Batch Size** field, specify the maximum allowed batch size in GB. If not specified, a default of 1 GB is used. - In the **Max Files per Batch** field, specify the maximum allowed count of files per batch. If not specified, a default of 1000 files is used. - Optionally configure the **Block Size** and **Task Flush Threshold** values as needed. 4. In the **Rsync** step, optionally configure `rsync` flags for file transfers. 5. In the **Review** step, review the settings you've made. If you want to make changes, click **Back**. Otherwise, click **Create Connector**. The connector is created and displayed in the **Connectors** page. ### SyncEngine CLI The following command creates a POSIX connector named `src-posix`: ``` syncengine-cli connector create \ --type=filesystem \ --name=src-posix \ ``` > **Note:** The connector type can be specified either as `filesystem` or `posix`. Optional flags: ``` --batch-size-gb --batch-file-count --block-size --task-flush-threshold --rsync-archive --rsync-compress --rsync-checksum --rsync-hard-links --rsync-acls --rsync-xattrs --rsync-no-perms --rsync-no-owner --rsync-no-group --rsync-bwlimit --rsync-timeout ``` The output should be similar to the following: ``` Successfully created connector 'src-posix' with ID 15 ``` > **Tip:** Note the connector ID in the output. ## Create Migration Create a migration to copy files from a source path to a destination path on a SyncEngine worker host. ### SyncEngine UI 1. In the **Migrations** page, click **New Migration**. A wizard dialog appears to walk you through the migration creation steps. 2. In the **Source** step, select the source POSIX connector. 3. In the **Destination** step, select the destination POSIX connector. 4. In the **Paths** step: - Under **Migration Label**, specify a SyncEngine worker label for the migration. The label is used to route migration jobs to the appropriate SyncEngine worker. One worker can be associated with one or more migration labels. Workers deployed with a certain label handle jobs for all migrations that have that label. You can create a new label or use an existing one. - To create a new label, select **Create new label** and enter a text string in the **New Label** field. The label can contain up to 128 characters, including uppercase and lowercase English letters, digits, periods (.), hyphens (-) and underscores (_). - To use an existing label, select **Use existing label** and enter the label string in the **Existing Label** field. - Under **Paths**, specify the paths to the migration source and destination. - Under **Source Path**, enter the absolute path where the data to be migrated are located. - In the **Destination Path**, enter the absolute path to which to copy the data. 5. In the **Sync mode** step: - Under **Sync Mode**, specify if you want to run the migration once or [periodically](/documentation/docs/migrate-from-posix-to-posix-34#periodic-resync): - To run the migration once, select **Manual** in the **SyncMode** field. Migration will be started after you submit the migration job manually. - To run the migration automatically at certain intervals, select **Automatic** in the **Sync Mode** field and enter an interval (in hours) in the **Sync Interval** field. The default interval is 24 hours. - Under **Include & exclude lists**: optionally add an exclude list for the migration: - Use the **Exclusion list** field to upload a file listing specific file paths to be skipped during the migration. To learn more about this feature, see [Exclude list](/documentation/docs/migrate-from-posix-to-posix-34#exclude-list). 6. In the **What you're copying** step: - Under **What you're copying**: - Review the settings you've made. If necessary, click *Back* and change as needed. - Under **Delete propagation**: - Decide how SyncEngine [handles the files that have been deleted](/documentation/docs/migrate-from-posix-to-posix-34#mirror-deletions) since the previous migration run. By default, SyncEngine does not delete such files from the destination path. If you want to have them deleted, select **Mirror source deletions on the destination**. 7. In the **Batch sizing** step, optionally adjust the size of batches that SyncEngine uses when copying the data. By default (if no values are entered in the fields provided), SyncEngine uses VAST cluster's defaults which are suitable for most workloads. - In the **Batch size** field, specify the maximum allowed batch size in GB. If not specified, a default of 1024GB is used. - In the **Batch file count** field, specify the maximum allowed count of files per batch. If not specified, a default of 1000 files is used. 8. In the **Per-task timeouts** step, optionally configure suitable timeouts for the migration jobs. 9. In the **Review** step, review the settings you've made. If you want to make changes, click **Back**. Otherwise, click **Create Migration**. The migration is created and displayed in the **Migration** page. ### SyncEngine CLI The following command creates a migration from `/mnt/posix-source` to `/mnt/posix-dest`. The migration is to be accomplished by workers labeled with `MYFIRSTMIGRATION` using connectors with connector IDs `4` and `5`. ``` syncengine-cli migration create \ --source-connector-id=4 \ --destination-connector-id=5 \ --source-path=/mnt/posix-source \ --destination-path=/mnt/posix-dest \ --label=MYFIRSTMIGRATION ``` Optional flags: ``` --sync-mode {manual | automatic --sync-interval } --enable-deletes={true|false} --exclusion-file --batch-size --batch-file-count ``` The CLI creates a migration and displays its properties. > **Tip:** Note the migration ID in the output. ## Start Migration Manual migration start is only relevant for migrations that do not have periodic resynchronization configured. For migrations with periodic resynchronization, the migration starts automatically upon creation. ### SyncEngine UI Navigate to the ​**Migrations**​​ page and click the start button for the migration you want. ### SyncEngine CLI Run the command: ​ syncengine-cli migration start --migration-id=
​
The output should be similar to the following:
​
Starting sync job for migration: MYFIRSTMIGRATION
✓ Sync job created successfully!
Job ID: 2
Status: pending ## Monitor Migration As the migration progresses, its status changes as follows: `Not Started` (or `init`) -> `Waiting for Worker` (or `pending`) -> `Syncing` -> `Synced`. When complete, the destination path will have files and directories copied from the source path. To verify, compare file counts or spot-check files on the destination path from a worker host. ### SyncEngine UI Check the migration status in the **Migrations** page. ### SyncEngine CLI Run the command: ``` syncengine-cli migration status --migration-id= ``` The output should be similar to the following: ``` === Migration Status: MYFIRSTMIGRATION === ID: 125 Status: synced Source: /mnt/posix-source Destination: /mnt/posix-dest Sync Mode: manual Last Sync: 2026-07-15 14:06:08 Copy Tags: always Settings: Error Retry: 0s Task Retry: 0 Batch Size: 0 GB Batch File Count: 0 Sync Jobs: 1 Recent Sync Jobs: +----+-----------+------------------+-------------+-------+------+ | ID | STATUS | START TIME | DURATION | FILES | SIZE | +----+-----------+------------------+-------------+-------+------+ | 96 | completed | 2026-07-15 14:06 | PT4.623401S | 1 | 29 B | +----+-----------+------------------+-------------+-------+------+ ``` ## Troubleshoot | Symptom | Likely Cause | Solution | | --- | --- | --- | | Sync job stays *pending* | Migration label does not match worker label | Update the migration to use the same label as the deployed worker. | | Permission denied | Mount permissions | Update UID/GID and mount options on workers. | | Path not found | Path missing on worker | Mount source and destination on all workers for the label. |