Configuring Data Sync

The Data Sync option allows you to set up automatic synchronization rules between the DMS and CAM. This rule can be configured to run on a schedule or as triggered. Data Sync gives you an option to synchronize the documents and the audit details. The configuration once set up can be monitored, edited, or deleted. Read below for detailed instructions to set up a data sync task. For information on the Azure Stack Deployment or AWS Stack Deployment, see the relevant links.

1 Configuring Data Sync
- 1.1 Sync Configuration
  - 1.1.1 Add a Data Sync Job
- 1.2 Sync Job Status
  - 1.2.1 View Details
- 1.3 Content Sync Settings
- 1.4 Deletion of Data Sync Data
- 1.5 Create Config
  - 1.5.1 Full Sync Worker Process
- 1.6 System Settings
  - 1.6.1 iManage
  - 1.6.2 NetDocuments
  - 1.6.3 Office365
- 1.7 Metadata Mapping Review
- 1.8 Skip Events
- 1.9 Ignore Events by Name:
- 1.10 Required Permissions

Configuring Data Sync

There are three tabs available Sync Configuration, Sync Job Status, and Content Sync Settings.

Sync Configuration

In the panel for Create Sync Configuration, the created Data Sync configurations are listed. Users can sync from any source system or external system for a new sync and for delta sync.

Filter Options

You can configure various filters here. Set any filter as required and click the Search button. The following filter options are available.

Filter by System
Filter by Instance
Filter by Database
Filter by Process Type
Filter by Status

Add a Data Sync Job

You can also create a Data Sync job from this panel. Click the Create Config button to add a data sync job. Click the refresh icon to manually update and refresh the page information.

To sync metadata values which include users, groups, custom parameters, types, classes, and subtypes that are associated with folders and documents including security.

For NetDocuments customers in the Canada region, the Data Sync process automatically will skip syncing metadata. This is to workaround the NetDocuments issue that doesn’t allow CAM to sync more than 10,000 metadata records.

The list of information is displayed in the following columns:

Column Name	Description

Column Name	Description
System	Displays the selected system
Repository	Displays the selected data sync repository
Database	Displays the selected database
Sync Status	Displays the sync status set for the configured job. Full Sync Queued: Displayed when the job is queued for sync. Full Sync Running: Displayed when the initial sync is running. Full Sync Completed: Displayed when the initial full sync is completed. Incremental Sync Active: Displayed when Incremental sync schedule is active. Incremental Sync Needs Attention: Displayed when incremental sync status is Stopped.
Deleted	Displays Yes if the job is deleted.
Stopped	Displays Yes if the data sync is stopped.
Interval	Displays the data sync schedule
Document Content	Displays if the documents are selected for sync from the repository
Audit	Displays if the document audit (document history) is selected for sync from the repository
Created On	Displays the configuration date for the data sync job.
Details	Click View Details to view complete details of data synced to CAM. The sync status can also be viewed in the Sync Job Status tab. The Sync Jobs dialog displays complete details of the sync job with the start and end time. The count of the workspaces and documents is shown when synced with the completion percentage of the sync job. The Full Sync tab displays the completion status of the data sync job scheduled to run once. The Incremental Sync tab displays the completion status of the scheduled sync jobs.
Monitoring	Displays the log monitoring of the data sync job and steps.
Actions	Click the Action Hamburger menu to perform the following actions- Stop Indexing: Allows to stop an indexing job on data sync if necessary. A status Indexing Stopped will display. Monitoring: Opens the monitoring page Reindex Data: Allows to reindex the data for that sync. Rescan S3 Data: Allows to rescan the S3 data to reindex and reprocess anything missed. Reprocess failed indexes: Allows to reindex any failed indexing jobs. Additional actions after clicking View Details are: View Config - Click View Config to view the existing config details. Edit Config - Click to edit the config details. This option is enabled until the job is ready to execute. Once the job is executed, you can not change the config. Edit Incremental Sync Interval - Click to edit the incremental sync schedule. Edit Recent Doc sync Interval- Click to edit the recent sync schedule. Is this a test configuration? Allows to add a test configuration sync so a smaller set can be run to confirm the right settings and metadata and mappings are set before you do a complete cabinet or workspace or database sync. Edit Metadata Sync Setting (iManage) - The following screen opens - CAM recommends a new value set to “No” by default to prevent additional time in every sync for metadata values. Next, new metadata values are synced which are associated with the folder/document/workspace profile. By default, the value is set to No. If you set Sync Metadata Values to “Yes”, during every full or incremental sync, the sync will fetch metadata values which are associated with the relevant folders and documents including security. Also, an interval can be set in the number of days. It is possible to set a custom value interval. Note: Change events beyond 7 days are now stored in CAM so that incremental changes can be processed after large initial data syncs are complete. Enable/Disable Audit Sync - Enable or Disable Audit syncs from the source. Note: If you set Sync Metadata Values only to “Yes”, the user and groups mapping metadata values will be fetched. Delete Config - Click to delete the configured data sync job.

While running the full data sync, if your DMS goes unhealthy, CAM shows a warning message directly on the Data Sync screen to help you know the status of your data sync and makes it easier to troubleshoot these intermittent DMS sync-related errors.

Sync Job Status

This tab displayed the status of data sync - Full Sync, Incremental Sync, and Recent Documents sync (for the iManage system).

Filter Options

You can configure various filters here. Set any filter as required and click the Search button. The following filter options are available.

Filter by System
Filter by Instance
Filter by Database
Filter by Process Type
Filter by Status

View Details

This option opens an informational window to display stats on the job like the processing rates and other information.

Clicking KPI shows:

Full Sync vs Incremental Sync vs Recent Documents Sync

The Full Sync tab displays the completion status of the data sync job scheduled to run once.

The Incremental (Delta) Sync tab displays the completion status of the scheduled sync jobs.

The Recent Documents Sync tab displays the completion status of the scheduled Recent document sync jobs. It only checks for content changes to get recent documents periodically at configured intervals. It also allows scheduling the recent document sync jobs for existing incremental syncs that are active.

Click the refresh icon to manually update and refresh the page information.

The list of information is displayed in the following columns:

Column Name	Description

Column Name	Description
System	Displays the selected system
Repository	Displays the selected data sync repository
Database	Displays the selected database
Status	Displays the progress status of the data sync job. Active: Initial run executed and active. Error: Execution failed. New: Created and yet to initial run.
Deleted	Displays Yes if the job is deleted.
Start From Days	Only displayed in the Recent Sync tab. Displays the number of days in the past.
Content Change	Only displayed in the Recent Sync tab.Yes, if the content of documents changes.
Last Sync Time	Only displayed in the Incremental and Recent Sync tab. Displays the last sync date and time
Next Sync Time	Only displayed in the Incremental and Recent Sync tab. Displays the next sync date and time
Job Type & Interval	Displays the configured data sync type and the sync schedule
Documents (Count & Size)	Displays the count & size of the document synced from the repository to CAM
Workspaces	Displays the count of the workspaces synced from the repository to CAM
Start Time	Displays the start date and time of the data sync job Start from Days is used for the sync to queue data for processing starting from the amount of past days specified and ending today.
End Time	Displays the completion date and end time of the sync job
Details	Click View Details to view Full Sync Job Details. The dialog displays complete details of the data synced.
Action Menu (Displayed only for Incremental and Recent Sync)	Pause Job - Click to stop the incremental sync Resume Job - Click to resume a paused job Start Now - Click to manually trigger an incremental task when an existing task is not running. Job History - It provides a Job history detail based on the status.

Content Sync Settings

In this tab you can select the sync storage options.
On successful validation, click Save.

For information on the Azure Stack Deployment or AWS Stack Deployment, see the relevant links.

Field Name	Description

Field Name	Description
Choose Cloud Provider	Select the cloud provider option. This defaults to AWS. It is disabled by default.
Choose Storage	This defaults to S3. It is disabled by default.
Choose Storage	Select the storage option, if you want to sync content to the Litera Encrypted Storage bucket or to your Private Cloud.
Bucket Name	Based on the storage option selected above, the bucket name option is available for selection. Select the bucket name.
API Endpoint Base URL	Enter the API URL for the site provided by Litera.
API Secret Key	Enter the API secret key. If using the Litera Encrypted storage option, the API secret key will be provided by Litera. On successful validation the content sync configuration is saved.

Deletion of Data Sync Data

There may be reason for the customer to want to delete all the data in the analytics database. The customer may require this because of compliance reasons - they no longer wish to have the data in the cloud, or they decide they do not need this any longer.

To do this:

Open the Admin>Data Sync panel.
Select the data sync configuration.
From the action menu, they can select to delete the configuration by hitting delete.

Azure Communication Settings

This section describes Azure communication through Content Mover.

Field	Description

Field	Description
Application (Client) ID	Specify the application or client id.
Directory (tenant) ID	Specify the tenant id.
Client Secret	Specify the client secret that was received in the email.
User principal (From Email Address)	Specify the user’s email address that is used for the Azure login.
Test Email Address (Inbox Enabled)	Enter the receiver’s email address. Click Validate to validate the email address.
Save	Click Save to save the changes.
Cancel	Click Cancel to discard the changes.

To set up a new config, click the Create Config button. To edit existing configs, click the icon next to the connection and click Edit Config.

Create Config

In the Sync Configuration tab click Create Configuration.
It will pop-up a checklist to be verified before you submit the Initial Sync request. In case of any doubt, please contact CAM support.

Field	Description

Field	Description
Expected Documents For Database	Select how many millions of records expected to be synced. Contact the CAM support team if the expected records are more than 4 Million records and enter the following details: CAM Support Approval Password - Required field. Contact the Litera customer care team or your Litera Professional Services consultant to get the password. Choose Worker For Initial Sync - Required field. The dropdown will provide the following 3 options: Initial sync Worker1 (Worker Status: READY, Type: RESERVED) Initial sync Worker2 (Worker Status: BUSY, Type: OPEN) Initial sync Worker3 (Worker Status: READY, Type: OPEN) To choose available workers, verify the workers' status by clicking the Check Worker Status link.
Choose System	It will list the systems configured in the External System Configuration. You can choose any one of them.
System Specific Settings	These are enabled based on the system selected from the dropdown. iManage click here. Office 365 click here. NetDocuments click here.
Choose Repository	Based on the selected system, all the repositories configured within the system is displayed here. Select the repository to be synced.
Select Database	Select the database from the repository to be synced.
Sync Document Content	Select Yes to sync the documents. Set the sync schedule for the document by selecting the number of days from which the document(s) must be picked to sync with CAM. Available time periods (in days): 7 30 60 90 120 150 180
Metadata Snapshots	Allows to keep an older snapshot copy of the folder or document metadata in CAM. This used to be called Keep Folder/Document versions. If yes is selected, 1-3 snapshot versions can be kept. Undo allows to revert a snapshot. By default, this option is disabled. Available for C4+ or Analyze.
Sync Document Audit	Select Yes to sync the audit data.
Sync Users and Groups	Select Yes to sync users and groups.
Incremental Sync Interval	Set the incremental sync schedule. After the initial sync, the subsequent data sync will be triggered as per the unit number of days (e.g. every 7 days).
For the iManage system, the Recent Document Sync feature is to be deprecated.

Click Create Configuration, and the following dialog opens -

Allows you to check system health on UI by selecting the provided checkboxes
For acknowledging specifically provided characters “Read & Agree“
Click Submit to create configuration successfully and click Cancel to close the dialog

Full Sync Worker Process

CAM implemented a feature to support more than one full sync at a time in each region.

Three parallel stacks are deployed and one stack can be used by one tenant at a time to run full sync.

The status of each worker is tracked: READY or BUSY.

READY -> The worker/stack is available to be allocated for any new initial sync.

BUSY -> The worker/stack is unavailable and is at present in use to run the initial sync.

Required Permissions

Available for the Analyze tier. Permissions can also be found: https://pdocs.atlassian.net/wiki/spaces/CCAM/pages/32440595

Permissions	Allows User To

Permissions	Allows User To
View Data Sync	Enables user to view Administration>Data Sync
Manage Data Sync	Enables user to view Administration>Background Jobs
Allow Audit Sync	Enables the synching of audit information.
Allow Content Sync	Enables the use of Content Sync. Syncs doc content.