Data Uploader Configuration

Overview

Data Uploader is a CAM Windows utility to read data from your on-premises, document management, billing, NBI, or Active Directory system and upload the information to the CAM cloud for processing. Based on the rules, metadata properties, and configurations set up, CAM will create or update workspaces, users, groups, and group membership across all specified systems.

Important: All Data Uploader users are encouraged to always run the latest build available for download, as that is always the most optimized. If you get a notification in Data Uploader itself, or from the CAM Administration Download tab that Data Uploader is outdated, you can find the latest version in your CAM Admin-> Downloads.

See the Data Uploader FAQ page for additional help.

Note: For each task added via the Data Uploader, a job is queued in the Jobs tab which will then be further processed by CAM. No changes are ever made to your on-premises database.

Data Uploader Process Workflow

Downloading Data Uploader from CAM

  1. Navigate to the Administration tab.

  2. Click on Downloads.

  3. Click on Data Uploader and save it to your machine.

Installation Requirements

Important: Currently, CAM supports and uses TLS 1.2 by default, but allows TLS 1.0/1.1 if the Data Uploader is run on a Windows 2012 Server.

On June 28, 2023, AWS is dropping all support for TLS 1.0/1.1 and this could affect users using Windows Server 2012 or older with the use of Data Uploader.

Windows Server 2016 and above are the natively supported versions with 1.2.

Hardware Requirements

  • A Windows PC or Server

    • Windows Server 2016+ is recommended for the note below.

  • At least 1GB of free space for program installation and execution

  • Internet access to upload the CSV to CAM; 50mbps or higher upload speed is recommended

Software Requirements

  • .NET Framework 4.8 or higher*

*= 4.8 is required for the new version of Data Uploader since Sprint 23.

Date Format

  • Format all date values as 

yyyy-MM-dd HH:mm:ss

SQL Requirements

Firms are strongly recommended to use an on-premises SQL database (instead of SQLLite) when the firm is a medium to large-sized company or it has more than 5,000 active matters. This allows Delta syncs to perform more efficiently as only necessary workspaces and metadata will be updated.

If the firm is using an on-premises SQL database, these are the requirements:

  • SQL Server 2012 Standard Edition or above.

  • Windows Server 2016+ is recommended for the note below.

Important Note for Windows Server 2012

  • A dedicated or Shared SQL server

  • 8+GB of free RAM

  • 4+CPU (cores/threads)

  • 10+GB free hard disk space*

Firewall and Port Requirements

Whitelist the following ports.

  • Port 443 should be open. CAM and Data Uploader utilize the HTTPS ports for requests. The API manages the requests, and no data is locally accessed or stored. If you’ve changed the default port for HTTPS, use the changed port. Check with your IT team if your firm has changed the port, and if so they need to open that default port on the corresponding HTTPS.

  • Port 1433 should be open for SQL communication between the on-premises SQL and the API. Check with your IT team if your firm has changed the port, and if so they need to open that default port on the corresponding SQL server.

 

User Account Requirements

Litera recommends a service account be used to run Data Uploader. For more details on setting up a user account, see

This service account should have:

  • Permission to download, install, and run the Data Uploader on the Desktop or server.

  • Read/Write access to the designated Data Uploader application directory.

  • Read access to the source database on the SQL server (to query the database and upload to CAM)

  • Permission to the local system to edit/enable the Windows Task Scheduler.

  • Read access to Active Directory.

  • Must be a system admin

Install Data Uploader

This section describes the Connection Strings tab in Data Uploader and how Data Uploader is using, storing, and accessing data. In both the current and previous versions of Data Uploader, there exists a database called ClientApp2.sqlite. This database is used to store the scripts and results of the queries. This database is also used to run on-premises comparisons of the CSV files for matter provisioning and user and group management to improve performance and cut down on unnecessary transactions due to false positives.

An example of this might be that a matter property that is unrelated to the workspace was updated in the time & billing system. Because the modification date on the matter in that system was updated, it might feed through many matters into the provisioning CSV. However, since none of those matter changes are related to the workspace as provisioned by CAM, CAM still needs to compare each matter entry to make sure no data has changed that would need to be updated in the downstream system (iManage, NetDocuments, etc.). By running a comparison of the files by Data Uploader using an on-premises database, we can only submit matter jobs with true changes, thus improving overall performance.

Using SQL

If you are using an SQL Database, it is recommended you recreate the connection strings to the source database for workspace provisioning. This will be outlined in the steps below.

  1. If you are using an on-premises SQL database, create a Database – suggested name = “CAMLocal”

  2. To create a connection string to this database, CAM will need an account that has dbo rights to this database so it can create the schema and manage the data in this database.

  3. The schema for this database will be created after you launch Data Uploader and create a connection string to this database outlined in the Create a Connection String section.

Running Data Uploader

  1. From the Windows Start menu, right-click on Prosperoware Data Uploader and Run as administrator.

  2. If Data Uploader prompts you to reauthenticate, put in the tenant URL then the user email and password for your CAM account.

  3. Click on the Connection Strings tab

  4. If using the SQLite database, and if you copied your connection strings configurations to the new ConnectionString.config, you should see your connection strings here.

    1. If not, then you can manually add the connection string by selecting Add, then selecting the “Select Connected System” as Source Database.

  5. If using an on-premises SQL database, you must first create the CAM Local Database connection string before creating any other connection strings.

Create a Connection String

  1. Click the Add button.

  2. Create a name for your connection string, e.g., CAM Local.

  3. From the “Select Connected system”, choose CAM Local Database.

  4. d. Select whether this is a direct SQL Connection or ODBC Connection.

    1. i. If ODBC, you will need to add a connection string.

    2. An example is: Data Source=100.10.200.220\MSSQL2017;Database=localdb;User=sa;Password=yourpassword.;Integrated Security=false

      1. Type in the name of the SQL Server or instance first. If in azure or a remote system include the ports first.

      2. Type in the name of the Database.

      3. Select login type and pass the correct credentials. Reminder: this account needs to have dbo rights to this database.

      4. Click Connect or Save

  5. You may now add all other desired connections to the data source for the workspace provisioning. Make sure to select Source Database from the “Select Connected System” drop-down.

Other Verifications

Click on the other tabs to verify that the data from the old configurations have been copied over correctly

SQL Files

  1. Click on the SQL Files tab and click the Refresh button if you do not see your SQL files. If you still do not see them, make sure they were copied to the correct directory.

  2. Next to each script, click the hamburger menu and select Edit.

    1. Select whether you want to enable this script and select the appropriate connection string against which this script will be executed. CAM supports multiple databases against which you can execute scripts.

  3. From the hamburger menu, you can also select the Validate option to test and validate your scripts before you run them in production.

Active Directory

Click on the Active Directory tab and check that your LDP configurations have been copied over correctly. If you do not see them, go back and verify you copied the data correctly from the old configuration file to the new one. Otherwise, you can recreate this by clicking the Add button.

CAM Agent Update

Data Uploader comes with a new CAM Agent. There is nothing you need to do to upgrade this – the old one will be uninstalled and a new one will be installed running under the service account credentials you provided during the installation.

If you did not enter the correct service account for the CAM Agent, then you can update this by doing the following:

  1. Go to the Windows Services

  2. Double-click on CAM Agent

  3. Click on the Log on tab. The CAM agent will be installed by the installer. The user has to select just Service Account and should have access to the CAM data uploader folder or accessible part.

Read here for complete instructions to configure and use the Cloud Data Uploader.

Upgrade Data Uploader

Installation

  1. Review the Hardware and Software Requirements for Data Uploader.

  2. You must create a backup of the existing DU database before the upgrade. If you are not utilizing a dedicated SQL database for DU, then skip this step.

  3. Make a backup copy of the current Data Uploader installation directory:

    • Prosperoware Data Uploader folder found at: C:\Program Files (x86)\Prosperoware\Prosperoware Data Uploader.

  1. Log into your CAM website.

  • If you are already logged in, sign out and sign in back in to ensure the latest update is available.

  1. Navigate to Administration

  2. Click on Downloads and download the latest version of Data Uploader onto the server where DU is installed.

  1. Run the installation wizard.

  2. Click Next. Keep the install folder path the same.

  3. Click Next.

  1. This version of the Data Uploader now requires that you enter a Windows Services domain, username, and password of the service account.

  2. Click Install to begin the installation.

  3. Once the installation is complete, click Close to close the wizard.

Updating the Configurations

The new Data Uploader application has replaced many of the previous configuration files with new config files. Since the following instructions for copying old configurations to the new files reference line numbers, we recommend using a program like Notepad++ or other tools that have line numbering available.

All configuration files are now in a directory under the install path called config located here:

C:\Program Files (x86)\Prosperoware\CAM\Data Uploader\service\config

If you have made any configuration changes to the following files, please follow the instructions below to copy your configurations to the new files. If you have not modified any of these files, you can skip this section.

  1. ActiveDirectoryIntegration.config

  2. Data.config

  3. UserPropertiesMapping.config

  4. SourceFilesConfig

  5. ADMapping File

  6. AppSetting.config

ActiveDirectoryIntegration.config > ADMapping.config

If you did not make any changes to the ActiveDirectoryIntegration.config, you may skip to section 2.

  1. Open the backed-up ActiveDirectoryIntegration.exe.config file.

  2. Copy the lines between the <configuration> and </configuration> tags.

    1. Exclude lines

    <startup> <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.1" /> </startup>
  3. Paste the configuration in the ADMapping.config file between lines 2-51.

  4. Use the commented lines 2-48 as mapping for AD. Samples are commented.

  5. Use commented lines 49-51 as fields for AD. Samples are commented.

Active Directory Filter

The organization units in Active Directory contain multiple groups with multiple users.
Now you can filter the user from a specific group in a single organization unit. The supported property is the "memberof." The "memberof" property takes the group name as a value and operators (%% - contains and !% - not contains) that are used to help in the filter. These parameters return the list of the users who are present or not present in the specific group.

The following are examples of supported operators:

Contain (%%)

<ad-property name="memberof" value="testgroup" operator="%%" />

Not Contain (!%)

<ad-property name="memberof" value="testgroup" operator="!%" />

Wildcard

A wildcard is a character(s) that substitutes for another character or string of characters when searching a database.

Searching for values in the ad-property name property using a wildcard is now supported.

The following are examples of a supported wildcard:

In the above example,

  • ad-property name - represents the search value or string.

  • Value – represents character(s). It must be present in the search value or string.

  • Operator – represents the searchable value. It must come either at the start of the string (s%) or at the end of the string (e%)

Data.config >ADMapping.config

If you did not make any changes to the Data.config, you may skip to section 3. If you applied filters for users and groups in the Data.config file, you will need to recreate these manually in the new ADMapping.config as there have been some changes. There are instructions in the ADMapping.config on how to apply the user and group filters.

  1. Open the backed-up Data.config file and note all the filters applied.

  2. Recreate the filter logic between the <configuration></configuration> tags.

User.config >ADMapping.config

  1. Open the backed-up User.config file

  2. Copy the fields between the <fields> and </fields> tags to the same tags in the ADMapping.config (located at the bottom of the file before the </configuration> tag.

Read more on ADMapping.config.

SourceFilesConfig

This is a new configuration file that allows you to set a separate unique identifier for each of your provisioning scripts. Some content systems might support a matter index ID for a unique identifier, while others might only be able to support client and matter ID. This file configures the unique ID per script.

  1. Open this file and update the uniqueId field. E.g. <uniqueId columns="clientId,matterid" source="CreateOrUpdateWorkspace.sql"/>

ADMapping.config

  • MultivaluedDatabasevalueMapping

    • It is a new database property value "MultivaluatedDatabase" which allows you to add multiple databases to the value field in AD-mapping that allows you to generate multi-input CSV or multi- delta CSV, respectively.

Sample Multivalueddatabase property value

  • SkipUpdateIfExists

    • While importing new AD users to iManage for the first time, the property name "SkipUpdateIfExists" with a value ="True" for iscamuser and iscamgroup has been added in the AD-Mapping that allows you to skip the update programmatically if it exists already.

Sample SkipUpdateIfExists property value - iManage

Sample SkipUpdateIfExists property value - M365

  • Preferred Database field

    • Add property value "PreferredDatabase", allows you to specify the user's Preferred Database on user creation

Sample PreferredDatabase value

Password flag Mapping

The property name “passwordExpired“ allows you to set the password expiration against user creation in iMange. Allows you to set password update on the first login after user creation in iManage using the property name “ForceUserToChangePassword“.

Sample Password flag Mapping

SQL Scripts

  1. Copy the scripts from the C:\Program Files (x86)\Prosperoware\Data Uploader\sql file to the new. C:\Program Files (x86)\Prosperoware\CAM\Data Uploader\sql

  1. Move any additional custom scripts to this SQL folder.

AppSetting.config

It provides a new export option to allow you to exclude duplicate documents like emails from the iManage and NetDocuments system during export.

The following code snippet allows you to create a folder where all duplicate emails are present:

The following code snippet allows you to skip all duplicate emails during export:

Data Uploader Limits

Required Permissions

Permissions

Allows User To

Permissions

Allows User To

View Data Uploader

To download Data Uploader from the Administration tab->Downloads.

|

 

Let's Connect📌

☎ +1 630.598.1100
☎ ‪+44 20 3880 1550‬
📧 support@litera.com
💻 https://www.litera.com/support/

📝 Support is available:
4 am - 8 pm US Eastern
(9 am - 1 am GMT/BST
7 pm - 11 am AET) on normal business days (excluding holidays)

© 2024 Litera