AD Sync / ADSync-how it works

Owned by Former user (Deleted)
Last updated: Oct 30, 2024 by David Elkarat
5 min read

Background

Active Directory Sync Adapter is a service that is required for Ethical Wall, DLP, Webex retention policies and Teams Governance (MS / Webex). It connects AD using LDAP to Azure or on prem AD and takes the relevant users and groups and inserts them to the DB.
The Ethical Wall, DLP, Webex retention policies and Teams Governance (MS / Webex) will query the DB and not the LDAP. This will save time and resources for the EW policy engine, DLP, Retention policies engine and Governance. At this point Active Directory Sync Adapter caches only users and groups that are configured in the Ethical Wall Policies, DLP rules, Webex retention policies and Teams Governance (MS / Webex). It supports Azure and local AD.

The Adapter has 5 main roles independent from each other

  1. Update group membership for EW, DLP, Teams Governance (MS / Webex), and Webex retention policies

  2. Update users for SharePoint webhooks

  3. Fill missing information in the Users table based on the email address

  4. Sync information (UPN & email address) of users in the Users table

  5. Update nested groups

Note: AD Sync Adapter can be monitored by AGAT Sphere Shield Service Agent.

Technical details:

  1. Update group membership for EW, DLP, Teams Governance (MS / Webex) and Webex retention policies:

Active Directory Sync Adapter takes groups from:

  • Policies with groups from FEDERATION_POLICIES table

  • DLP rules with groups from DLP_RULES table

  • MS Teams Governance settings

  • Webex Governance settings

  • Webex retention policies of groups

After taking all groups, Active Directory Sync Adapter connects to Active Directory provider (local or azure) and takes all the users that belongs to these groups.

All security groups will be fetched If Sentiment analysis is enabled

MANAGED_USER_GROUPS

From AD Sync 1.2.0 it will always fill in the managed user groups table.

The users will be inserted to MANAGED_USER_GROUPS table where each user has multiple records of UPN, group name and email.

MANAGED_USER_GROUPS table structure:

for each group which the user belongs to there is a different row and for each email address there is a different row of the combination UPN + group name.

If the “PopulateAllUserGroupsTables” configuration is true than FEDERATION_USER_GROUPS table is also updated. In FEDERATION_USER_GROUPS table each user has a record with the groups it belongs to and email addresses.

Note:

Table name is kept FEDERATION_USER_GROUPS even though it is serving DLP groups too (for simplicity reasons)

FEDERATION_USER_GROUPS table structure:

AdditionalEmails column is for users that have more than one email address. It’s being used for applications that have only email address of the users (webexTeams application) to get the UPN of those users.

MANAGED_USER_GROUPS table replaced FEDERATION_USER_GROUPS table. The following components version are using MANAGED_USER_GROUPS table:

  • DLP from version 1.4.4

  • Retention policies in AP 5.6.11

  • EW version 5.5.5

Active Directory Sync Adapter recognizes when a user was updated or deleted from the groups in EW policies and updates the date on FEDERATION_POLICY_CACHE table, UPDATED_AT column, for recalculation.

in Webex the field UserName will be filled by user’s email. For all the rest, by user’s name

2. Update users for SharePoint webhooks:

Active Directory Sync Adapter gets all the group policies of EW and takes the groups that file sharing is blocked, than it takes the users that belong to those groups from FEDERATION_USER_GROUPS table. For all the users that was brought from the DB, the adapter brings all windows365 groups from Azure that the users belong to. The Windows365 groups and users are inserted to CLOUD_USERS_GROUPS_SCOPE table. This table is being used to create webhooks for those users and groups.

Set “InitializeRefreshCloudUsersGroups” in Application settings file to “true” to update this table.

Updating CLOUD_USERS_GROUPS_SCOPE

From version 1.2.3 of AD Sync Adapter: In case MS Teams Governance is enabled and Allow upload files to OneDrive or Share point is “No”: Active Directory Sync Adapter will get all the users from the tenant and the Windows 365 groups they belong to. All the users and groups will be inserted to CLOUD_USERS_GROUPS_SCOPE table to create webhooks.

CLOUD_USERS_GROUPS_SCOPE structure:

AD sync fills in Entityid and EntityType

3. Fill missing Users Information based on the email address:

Active Directory Sync can fill missing users information based on email address. It takes the email address from Users table and gets the user information from Azure, than insert the missing information of the user to Users table.

Set “FillUsersMissingValues“ in Application settings file to “true” to update the users information.

4. Sync information (UPN & email address) of users in the Users table

From AD-Sync 1.2.8, there is support for syncing users’ information, like UPN and email address from the active directory to the USERS table in the database.

This feature is important when a customer do some changes in the user’s UPN or the user’s email address in the active directory. This feature works only in the Azure active directory for now. It doesn’t work in a local active directory.

To enable this feature please set the value of the “EableSyncUsersInfoTable“ setting to true.
To set how often the AD-Sync will sync the users’ information, please use the “IntervalSyncUsersInfoTable“ setting.

5. Update nested groups:

From AD-Sync 1.2.8, there is support for syncing nested groups from LDAP into the MANAGED_NESTED_GROUPS table.

The AD-Sync is syncing only 1 degree of nested groups.
The nested groups that AD-Sync is syncing are only the nested groups in all the parent groups with policies. Groups with policies there are groups that are in EW policies or in allowed groups of SharePoint sites governance or groups to be inspected, etc.
This feature is required for SharePoint governance for nested groups inspection in SharePoint sites.

MANAGED_NESTED_GROUPS table structure:

AD sync fills all the columns:

This feature run by default in the “IntervalTimeRefreshEwPoliciesUsersGroups“ timer, and you don’t need to enable some configuration for that.

 

Overwrite UPN with email

For some use cases, the UPN is different from the email; there is a need to overwrite the UPN with the email as the office application works with UPN when searching for users in M365 for eDiscovery, for example.

To avoid changes in the Office application- the ADSync includes the ability to overwrite the UPN value with the email value.
This is done in the Managed Users Groups table

To enable this, set the following parameter in the ADSync config: OverwriteUpnWithEmail=true

Handling deleted users

When a user leaves the company and is deleted from the Azure AD (Entra), issues can happen because he is still considered a member of group chats.

After some time, the filter changes him to “displayname-unknown@domain.com”.

Because the user is no longer a group member, the default policy applies and can cause unexpected behavior.

To solve this, ADSync from 1.3.6.0 will mark the deleted users as such in Users table - UpdatingFromADStatus column. This process will run periodically once a day based on IntervalSyncUsersInfoTable in config file and will be pended on value of EnableSyncUsersInfoTable in config file (should be true)

In addition, the routine process of syncing the groups also adds a group with a hard-coded name, “DeletedUsers,” to the managed users group table after each update to allow adding a rule to handle these deleted users. This will be done as part of updating the managed users group table, by default every 3600 secs (IntervalTimeRefreshEwPoliciesUsersGroups)

Versions:

ADSync - 1.3.6.0

SphereShield.CasbCommon - 1.5.3.4

Infra - 6.9.4

CasbMsOfficeApplication - 1.5.23

CasbSharePointApplication - 1.3.6

 

EnableRefreshUsers in adapter config should be true

OverwriteUpnWithEmail in ADSync should be false

 

In TeamsProtector.xml - policyWorkerInterval - set to 5

USERS table

After user aabb@agat.live is deleted (rest of the users like before the change)

After user aabb@agat.live is restored in AD

 

These are the 2 policies needed:

The policy with DeletedUsers needs to be created from database