Configuring Data Sync

The Data Sync option allows you to set up automatic synchronization rules between the DMS and CAM. This rule can be configured to run on a schedule or as triggered. Data Sync gives you an option to synchronize the documents and the audit details. The configuration once set up can be monitored, edited, or deleted. Read below for detailed instructions to set up a data sync task. For information on the Azure Stack Deployment or AWS Stack Deployment, see the relevant links.

Configuring Data Sync

There are three tabs available Sync ConfigurationSync Job Status, and Content Sync Settings.

Sync Configuration

In the panel for Create Sync Configuration, the created Data Sync configurations are listed. Users can sync from any source system or external system for a new sync and for delta sync.

Filter Options

You can configure various filters here. Set any filter as required and click the Search button. The following filter options are available.

  • Filter by System

  • Filter by Instance

  • Filter by Database

  • Filter by Process Type

  • Filter by Status

Add a Data Sync Job

You can also create a Data Sync job from this panel. Click the Create Config button to add a data sync job. Click the refresh icon to manually update and refresh the page information.

To sync metadata values which include users, groups, custom parameters, types, classes, and subtypes that are associated with folders and documents including security.

For NetDocuments customers in the Canada region, the Data Sync process automatically will skip syncing metadata. This is to workaround the NetDocuments issue that doesn’t allow CAM to sync more than 10,000 metadata records.

The list of information is displayed in the following columns:

Column Name

Description

Column Name

Description

System

Displays the selected system

Repository

Displays the selected data sync repository

Database

Displays the selected database

Sync Status

Displays the sync status set for the configured job.

  • Full Sync Queued: Displayed when the job is queued for sync.

  • Full Sync Running: Displayed when the initial sync is running.

  • Full Sync Completed: Displayed when the initial full sync is completed.

  • Incremental Sync Active: Displayed when Incremental sync schedule is active.

  • Incremental Sync Needs Attention: Displayed when incremental sync status is Stopped.

Deleted

Displays Yes if the job is deleted.

Stopped

Displays Yes if the data sync is stopped.

Interval

Displays the data sync schedule

Document Content

Displays if the documents are selected for sync from the repository

Audit

Displays if the document audit (document history) is selected for sync from the repository

Created On

Displays the configuration date for the data sync job.

Details

Click View Details to view complete details of data synced to CAM. The sync status can also be viewed in the Sync Job Status tab.

The Sync Jobs dialog displays complete details of the sync job with the start and end time. The count of the workspaces and documents is shown when synced with the completion percentage of the sync job. The Full Sync tab displays the completion status of the data sync job scheduled to run once. The Incremental Sync tab displays the completion status of the scheduled sync jobs.

Monitoring

Displays the log monitoring of the data sync job and steps.

Actions

Click the Action Hamburger menu to perform the following actions-

  • Stop Indexing: Allows to stop an indexing job on data sync if necessary. A status Indexing Stopped will display.

  • Monitoring: Opens the monitoring page

  • Reindex Data: Allows to reindex the data for that sync.

  • Rescan S3 Data: Allows to rescan the S3 data to reindex and reprocess anything missed.

  • Reprocess failed indexes: Allows to reindex any failed indexing jobs.

Additional actions after clicking View Details are:

 

  • View Config - Click View Config to view the existing config details.

  • Edit Config - Click to edit the config details. This option is enabled until the job is ready to execute. Once the job is executed, you can not change the config.

  • Edit Incremental Sync Interval - Click to edit the incremental sync schedule.

  • Edit Recent Doc sync Interval- Click to edit the recent sync schedule.

  • Is this a test configuration? Allows to add a test configuration sync so a smaller set can be run to confirm the right settings and metadata and mappings are set before you do a complete cabinet or workspace or database sync.

  • Edit Metadata Sync Setting (iManage) - The following screen opens -

    • CAM recommends a new value set to “No” by default to prevent additional time in every sync for metadata values.

    • Next, new metadata values are synced which are associated with the folder/document/workspace profile.

    • By default, the value is set to No.

    • If you set Sync Metadata Values to “Yes”, during every full or incremental sync, the sync will fetch metadata values which are associated with the relevant folders and documents including security. Also, an interval can be set in the number of days.

It is possible to set a custom value interval.

Note: Change events beyond 7 days are now stored in CAM so that incremental changes can be processed after large initial data syncs are complete.

  • Enable/Disable Audit Sync - Enable or Disable Audit syncs from the source.

Note: If you set Sync Metadata Values only to “Yes”, the user and groups mapping metadata values will be fetched.

  • Delete Config - Click to delete the configured data sync job.

While running the full data sync, if your DMS goes unhealthy, CAM shows a warning message directly on the Data Sync screen to help you know the status of your data sync and makes it easier to troubleshoot these intermittent DMS sync-related errors.

 

Sync Job Status

This tab displayed the status of data sync - Full Sync, Incremental Sync, and Recent Documents sync (for the iManage system).

Filter Options

You can configure various filters here. Set any filter as required and click the Search button. The following filter options are available.

  • Filter by System

  • Filter by Instance

  • Filter by Database

  • Filter by Process Type

  • Filter by Status

 

View Details

This option opens an informational window to display stats on the job like the processing rates and other information.

 

image-20240517-130142.png

Clicking KPI shows:

 

 

Full Sync vs Incremental Sync vs Recent Documents Sync

The Full Sync tab displays the completion status of the data sync job scheduled to run once.

The Incremental (Delta) Sync tab displays the completion status of the scheduled sync jobs.

The Recent Documents Sync tab displays the completion status of the scheduled Recent document sync jobs. It only checks for content changes to get recent documents periodically at configured intervals. It also allows scheduling the recent document sync jobs for existing incremental syncs that are active.

  1. Click the refresh icon  to manually update and refresh the page information.

The list of information is displayed in the following columns:

Column Name

Description

Column Name

Description

System

Displays the selected system

Repository

Displays the selected data sync repository

Database

Displays the selected database

Status

Displays the progress status of the data sync job.

Active: Initial run executed and active.

Error: Execution failed.

New: Created and yet to initial run.

Deleted

Displays Yes if the job is deleted.

Start From Days

Only displayed in the Recent Sync tab. Displays the number of days in the past.

Content Change

Only displayed in the Recent Sync tab.Yes, if the content of documents changes.

Last Sync Time

Only displayed in the Incremental and Recent Sync tab. Displays the last sync date and time

Next Sync Time

Only displayed in the Incremental and Recent Sync tab. Displays the next sync date and time

Job Type & Interval

Displays the configured data sync type and the sync schedule

Documents (Count & Size)

Displays the count & size of the document synced from the repository to CAM

Workspaces

Displays the count of the workspaces synced from the repository to CAM

Start Time

Displays the start date and time of the data sync job

Start from Days is used for the sync to queue data for processing starting from the amount of past days specified and ending today.

End Time

Displays the completion date and end time of the sync job

Details

Click View Details to view Full Sync Job Details. The dialog displays complete details of the data synced.

Action Menu

(Displayed only for Incremental and Recent Sync)

  • Pause Job - Click to stop the incremental sync

  • Resume Job - Click to resume a paused job

  • Start Now - Click to manually trigger an incremental task when an existing task is not running.

  • Job History - It provides a Job history detail based on the status.

Content Sync Settings

  1. In this tab you can select the sync storage options.

  2. On successful validation, click Save.

For information on the Azure Stack Deployment or AWS Stack Deployment, see the relevant links.

Field Name

Description

Field Name

Description

Choose Cloud Provider

Select the cloud provider option.

This defaults to AWS. It is disabled by default.

Choose Storage

This defaults to S3. It is disabled by default.

Choose Storage

Select the storage option, if you want to sync content to the Litera Encrypted Storage bucket or to your Private Cloud.

Bucket Name

Based on the storage option selected above, the bucket name option is available for selection. Select the bucket name.

API Endpoint Base URL

Enter the API URL for the site provided by Litera.

API Secret Key

Enter the API secret key. If using the Litera Encrypted storage option, the API secret key will be provided by Litera. On successful validation the content sync configuration is saved.

Deletion of Data Sync Data

There may be reason for the customer to want to delete all the data in the analytics database. The customer may require this because of compliance reasons - they no longer wish to have the data in the cloud, or they decide they do not need this any longer.

To do this:

  1. Open the Admin>Data Sync panel.

  2. Select the data sync configuration.

  3. From the action menu, they can select to delete the configuration by hitting delete.

Azure Communication Settings

This section describes Azure communication through Content Mover.

Field

Description

Field

Description

Application (Client) ID

Specify the application or client id.

Directory (tenant) ID

Specify the tenant id.

Client Secret

Specify the client secret that was received in the email.

User principal (From Email Address)

Specify the user’s email address that is used for the Azure login.

Test Email Address (Inbox Enabled)

Enter the receiver’s email address.
Click Validate to validate the email address.

Save

Click Save to save the changes.

Cancel

Click Cancel to discard the changes.

  1. To set up a new config, click the Create Config button. To edit existing configs, click the   icon next to the connection and click Edit Config.

Create Config

  1. In the Sync Configuration tab click Create Configuration.

  2. It will pop-up a checklist to be verified before you submit the Initial Sync request. In case of any doubt, please contact CAM support.

Field

Description

Field

Description

Expected Documents For Database

Select how many millions of records expected to be synced.

Contact the CAM support team if the expected records are more than 4 Million records and enter the following details:

  • CAM Support Approval Password - Required field. Contact the Litera customer care team or your Litera Professional Services consultant to get the password.

  • Choose Worker For Initial Sync - Required field. The dropdown will provide the following 3 options:

  1. Initial sync Worker1 (Worker Status: READY, Type: RESERVED)

  2. Initial sync Worker2 (Worker Status: BUSY, Type: OPEN)

  3. Initial sync Worker3 (Worker Status: READY, Type: OPEN)

To choose available workers, verify the workers' status by clicking the Check Worker Status link.

Choose System

It will list the systems configured in the External System Configuration. You can choose any one of them.

System Specific Settings

These are enabled based on the system selected from the dropdown.

iManage click here.

Office 365 click here.

NetDocuments click here.

Choose Repository

Based on the selected system, all the repositories configured within the system is displayed here. Select the repository to be synced.

Select Database

Select the database from the repository to be synced.

Sync Document Content

Select Yes to sync the documents. Set the sync schedule for the document by selecting the number of days from which the document(s) must be picked to sync with CAM.

Available time periods (in days):

  • 7

  • 30

  • 60

  • 90

  • 120

  • 150

  • 180

Metadata Snapshots

Allows to keep an older snapshot copy of the folder or document metadata in CAM. This used to be called Keep Folder/Document versions. If yes is selected, 1-3 snapshot versions can be kept. Undo allows to revert a snapshot. By default, this option is disabled. Available for C4+ or Analyze.

Sync Document Audit

Select Yes to sync the audit data.

Sync Users and Groups

Select Yes to sync users and groups.

Incremental Sync Interval

Set the incremental sync schedule. After the initial sync, the subsequent data sync will be triggered as per the unit number of days (e.g. every 7 days).

For the iManage system, the Recent Document Sync feature is to be deprecated.

  1. Click Create Configuration, and the following dialog opens -

  1. Allows you to check system health on UI by selecting the provided checkboxes

  2. For acknowledging specifically provided characters “Read & Agree

  3. Click Submit to create configuration successfully and click Cancel to close the dialog

Full Sync Worker Process

CAM implemented a feature to support more than one full sync at a time in each region.

Three parallel stacks are deployed and one stack can be used by one tenant at a time to run full sync.

The status of each worker is tracked: READY or BUSY.

READY -> The worker/stack is available to be allocated for any new initial sync.

BUSY -> The worker/stack is unavailable and is at present in use to run the initial sync.

Required Permissions

Available for the Analyze tier. Permissions can also be found:

Permissions

Allows User To

Permissions

Allows User To

View Data Sync

Enables user to view Administration>Data Sync

Manage Data Sync

Enables user to view Administration>Background Jobs

Allow Audit Sync

Enables the synching of audit information.

Allow Content Sync

Enables the use of Content Sync. Syncs doc content.

 

Let's Connect📌

☎ +1 630.598.1100
☎ ‪+44 20 3880 1550‬
📧 support@litera.com
💻 https://www.litera.com/support/

📝 Support is available:
4 am - 8 pm US Eastern
(9 am - 1 am GMT/BST
7 pm - 11 am AET) on normal business days (excluding holidays)

© 2024 Litera