CAM’s New Data Sync & Analytics Functionality

Building on the success of CAM Analyze, we're excited to announce a substantial update, released as part of 23.12.14, to the CAM’s Data Sync & Analytics framework. This update delivers a more user-friendly experience, enhanced features, and performance improvements.

The update reflects our commitment to continuously improving our products to meet the critical business needs of our customers. The aim is to address previous challenges and incorporate features that enhance usability and efficiency.

Key Updates

• Data Sync performance: Changes to the sync process and storage greatly improve overall sync times and accuracy.

• Metadata configuration changes will no longer require a re-sync. With the new update, users can add or change metadata configurations and then re-index the content, saving time and enhancing the platform's efficiency.      

• User Interface: The updated analytics interface will provide a more intuitive and user-friendly experience, akin to reporting interfaces from other cloud applications. This means users can generate reports without needing SQL skills, providing non-technical stakeholders the ability to create their own reports.

• Bulk Actions: Users running reports from the Analytics tab are also able to action on the result set including bulk profile or security updates, check-in or unlock documents, declare as records, download documents, or download as CSV.

• Speed: The updated version significantly improves speed, ensuring quick report generation without any timeouts. This means users can access insights faster and make timely decisions.

Note to Existing CAM Analytics Customers

If you are an existing CAM Analytics customer using the old MySQL interface, you will be migrated over to the new data sync and Analytics in a planned migration. Someone from Litera will reach out to you to coordinate this migration, setup, and training. In some cases, it might be best to re-sync the data from scratch, but that will be determined on a case-by-case basis.

Pre-Requisites for New CAM Analyze Customers

Data Sync

Because starting an initial data sync can be resource-intensive, there are a few steps you must take before kicking off an initial data sync. This ensures the appropriate settings and resources are configured on the CAM backend. Additionally, customers should wait until any DMS cloud migrations are completed before running a sync as large incremental changes can impact performance.

1. When you are within 1 week of being ready to do your initial sync, please open a ticket with Litera Support and request two things - that your CAM tenant is configured for the new data sync and that a “worker” be assigned. Please make sure to include the URL to your CAM tenant in this request.

2. Ensure all metadata configurations are correct in your tenant. Please see Manage Metadata. While you can add or change metadata configurations after the initial sync, it is best to confirm these ahead of the initial sync in order to avoid having to reindex.

Analytics Setup Steps

1. In order to configure Analytics, you must enable these permissions in the role of the administrators:

2. To view the Analytics tab and create and run reports these permissions are required:

3. To allow users to take action on any report results, users must have permissions to each available action. You can determine which actions the user is permitted to take and set the role permissions accordingly.

4. As an administrator, you must set up the entities on which you would like to report. Follow the steps below in the Configuring an Entity section.

5. Now you are ready to use analytics and build a report.

The New Analytics Interface

We’ve redesigned the user interface to give users more space around filters, joins, and options, plus hid un-necessary options.

1. Existing reports now show when you open analytics. There are no two tabs now.

2. Click New Report to open those options.

3. The information needed to initially create the report shows here in the Info tab.

4. All joins can be set in the Join tab

5. Configurations like filters, summaries, and sorts are here in the Configure tab.

6. Report actions can be done after previewing.

7. Save as allows to save the new query or edit the existing query.

Open

Open

Creating a New Report

Let’s get started to create a new report.

Enter the following information:

• Choose Entities/ Entity. Select from the available entities. By default, the available options are: document, folder, workspace, document_history, and now User and Group.

1. Manage Columns

• Check/Uncheck the columns to be used in the report. The number of columns changes depending on the currently selected Entity.

  • A select all button is available to select all columns if desired or to unselect all.

  • Only mapped entities will show in the results.

Once the information is provided, it is possible to perform actions on the data.

Examples of Reports Step by Step

A Document Report Example

Our first example we are going to use an out of the box report functionality to view all documents.

1. Enter a report name and a description that makes sense. E.g. All Documents.

Note that even though this field has an asterisk to be required, it is not required for adhoc reports. It is required if you choose to save the report.

2. Choose the Entity this query will utilize. In this case, we are choosing Documents.

3. If you choose not to select your individual columns, we’ve preselected the common columns for each entity you can report on.

Columns that are required no matter what query was selected and are not editable and have an asterisk next to them.

E.g. document queries must have the document_number or extid plus several other metadata fields. This is so users can perform actions on those results, and so CAM can pass the fields to iManage.

4. Click Preview. You’ll see the count of documents in the system shown instantly below, paginated per 100 results.

A Workspace Search by Practice Group Report Example

Now we will do a more complex query filtering workspaces based on a practice group.

1. Enter a report name. E.g. Workspace search by Practice Group.

2. Pick Entity Workspaces.

3. In this example we will pick our columns:

• extid, description, extparentid, clientcode, mattercode, createddate, editeddate, extsystemid, system, containerid, subclasscode, extworkspaceid

When searching for a column, start typing in the box. Check the boxes of the columns you choose. To return to all columns after checking the ones you searched for, delete the text in the column search. This will not uncheck the boxes you just checked.

If you need to know the datatype, look on the icon next to the column and if you hover it will say if it is Boolean, number, string, date etc.

4. Set the filters to be what you want to filter it by. This would be the “where” clauses in a SQL query. If the Filters box is hidden, click the Add Filters button on the bottom.

   1. Click Add filter to add a new one.

   2. Pick the column to filter it on. Here we are going to do Practice Code  equal to Corporate.

   3. Greyed out filters are systematically added based on the entity type selected when the entity is created. E.g. if document was set on the entity, the greyed-out filter would be object_type equal to document.

5. Add any sorting your query needs to have.

   1. Click Add Sorters. A new section appears under Filters.

   2. Here we want to sort by the external workspace id. Select ext_workspace_id in the sorter option.

   3. By default, it will be added ascending. Click the arrow to switch it to descending.

6. Click Summarize if the query needs to have a count, sum, average, or distinct line added to it.

   1. Select the option and select the field that needs to be summarized. If multiple columns need to be added, click the plus sign again. Here we will add a count of workspaces.

   2. When a Summarize column is added, the query should have a group by field. This works the same way as in SQL.

7. Click Preview once you’ve completed building your query. In our example, we had 32 workspaces that had a Practice Group with Corporate.

8. Click Save to save your query for the future.

9.  Saved queries show on the Existing Reports table.

10.  Now the user can perform bulk actions off the reports as they were able to beforehand off the 32 results:

Available actions are:

Add Filters

It is possible to select from any of the available columns to set a filter based on how the value of the column relates to another value that can be freely set.

1. To view the tooltip of what each filter means, click the Info“i” next to the Filter. Self-explanatory filters will not have a tooltip.

2. Click the filter to edit the filter.

Parameters

Parameters are now supported which allows one to add parameters on their query to add a value that auto populates.

When toggled on the lookup for the field appears where you choose the server/repo and then select the value.

1. Add a filter by clicking Add Filter from the Configure tab.

2. Don’t add the column value of the filter first, just pick the column filter and click Add.

3. Now do not pick a value here, just click add.

3. In the Configure tab, toggle on the column switch to expose the filter for the saved report. This causes the filter to be a parameter and you pick the value in the report preview later.

4. Finish your configurations and save the report using Save As. For the parameters to work, you MUST save the report.

5. Go to Existing Queries and Preview the report you created.

6. A new set of tabs show on the left including Filter. Click on this tab.

7. Expand the column you picked for the filter parameter (e.g. )Practice Area (CM) dropdown and pick the value of the practice code. In this case, we are picking Corporate. Click Add.

8. Click Preview now and results will display.

Available conditions are:

A Not Contains Example: Excluding Administrators

As a user, when I run an activity report, I would like to filter out all actions performed by administrative users. As such, I would like to filter out all rows where the activity_action_user_name contains ‘admin’ - so a not contains operator.

1. Access the activity reporting feature.

2. Run a standard activity report.

3. Apply a filter to exclude all actions where the activity_action_user_name contains 'admin'.

   • Use the "not contains" operator for filtering.

4. Execute the filtered report.

Sorting

1. It is possible to add sorters to rearrange the information.

2. For each sorter, click the arrow to switch between ascending or descending sorting. Multiple sorting criteria can be added.

3. Sorting aggregations in Elastic and other data sources is now possible.

Summaries

1. It is possible to add summaries by selecting a metric, and then a column from either of the joined entities to group by. it is possible to add multiple metrics and groupings. For example, a count by client would be done using a summarize function for a count of client_code.

Available metrics:

Joins

Joins are available in the May 25th release, so that users can join to other tables in their queries to bring additional data.

1. Open the Join tab.

2. Pick the type of join you are looking for: Has parent, or has child.

   1. For example, a child join would be the folder table to the document table, or the document table to the document_history table.

   2. For parent joins, examples are document_history has the parent of document, or document has the parent of folder.

3. Users can pick one more join for each type of join if they need two joins.

A Join Example: A user’s history for documents

In this example, we want to run a governance query to report what actions a user named Tina has done on the entire system over time.

1. Pick Entity Document History. This is the primary entity in this query since we want to see history.

2. Include the selected as-is columns but include activity_action_username.

3. In the Join tab, the “has parent” option is already selected and select the Documents entity. Document history is the furthest child table and has the parent Documents or Folders.

4. Include in the join columns, the following: activity_action_code and activity_action_code_name.

5. In the Configure tab, select the Filter activity_action_user_name contains Tina. Note if you save this query, this filter is saved as a parameter that has a toggle next to it. Turn it on to use it in your report to populate it when you Preview it.

6. Then select a Summarize of a Count of Rows of activity_action_code_name.

7. Finally, set a group by on the activity_action_name.

8. Preview the results.

9. In this case, the report may show those actions that a user shouldn’t be performing, like if Tina isn’t supposed to be Exporting Documents, or flag if too many modifications were made over time. Those count summaries are in the activity_action_code_name_count_fc column.

Preview

1. It is possible to run a preview of the report by clicking the Preview button at the bottom.

2. Click Export to CSV if you want to download results.

3. Click Save at the top of the page to save this query.

4. JSON columns will display as a hyperlink visualized in the preview window. Currently, these are the fields supported:

A User Example

1. Pick the Entity User.

2. Add a filter on the system. Set the filter equal to the system, and set to your system (imanage, netdocuments)

3. Add another filter on the user id or username for a user.

Configurations

Creating or Editing an Analytics Entity

In CAM, there is the concept of an entity.  An entity is the category of object that is being indexed.  Currently there are four supported entities:  documents, folders, workspaces, and history.  The next step is to configure the index entities against which you will be running reports. This is a one-time step and should be done after the initial data sync is completed. 

There are four entities you should create:

1. From the Administration tab, click on Dashboard.

2. Click on the Entities tab and select Add Entity.

The following window will display.

3. Select the CAM Secondary Elastic Search data source from the list.
   
   

4. Under Entity name, select the option that looks like this:

   1. [tenantId]_cam_dms
      
      

5. Under Display name, provide a unique, friendly name for each of the four supported entities, e.g.

   1. Documents

   2. Folders

   3. Workspaces

   4. History (or Audit).

If you are searching for document history enter a name like Document History. Description is optional but may help the user know which entity is for which purpose.

6. Under Query Type, select the associated type of the entity query that will be pulled.

This should match the entities you are creating:

Managing Previously Created Entities

1. Find the prior created entity in the table on the Entities tab.

2. Edit the available fields.

Note: The Entity name cannot be edited on this screen.

3. Click Save when completed.

Deletion of entities can only be completed if the entities are not used actively in a saved query or report. A warning will appear if there are such links, and prevent the deletion until those reports are edited/changed/removed first.