Data Synchronization Services
Relevant source files
The following files were used as context for generating this wiki page:
Data Synchronization Services in Casdoor provide a mechanism to automate the import and continuous alignment of user identity information from external systems into the Casdoor ecosystem. These services, referred to as "Syncers," support a wide variety of sources including relational databases (MySQL, PostgreSQL, SQL Server, Oracle, SQLite), directory services (Active Directory, LDAP), and modern SaaS APIs (Google Workspace, AWS IAM, WeCom, SCIM).
Sources: object/syncer.go, web/src/SyncerEditPage.js:314-325, web/src/SyncerListPage.js:37-56
Core Architecture and Components
The synchronization system is built on a provider-based architecture where a generic Syncer object defines the configuration and scheduling, while specialized SyncerProvider implementations handle the protocol-specific logic for fetching and mapping external data.
The Syncer Data Model
The Syncer struct represents the primary configuration for a synchronization task. It includes connection details, scheduling intervals, and mapping rules defined via TableColumn definitions.
| Field | Type | Description |
|---|---|---|
Name | string | Unique identifier for the syncer. |
Type | string | The source type (e.g., Database, Keycloak, SCIM, AWS IAM). |
Organization | string | The Casdoor organization users will be synced into. |
SyncInterval | int | Frequency of synchronization in seconds. |
IsReadOnly | bool | If true, Casdoor only pulls data; it does not push updates back. |
IsEnabled | bool | Master switch to start/stop the background sync job. |
TableColumns | array | List of TableColumn objects mapping source fields to Casdoor fields. |
Sources: object/syncer.go:28-60, swagger/swagger.yml:1955-2015
Specialized Syncer Providers
Each integration type implements a specific set of methods to interact with the target system. For example, API-based syncers (like WeCom or AWS IAM) often operate in a read-only mode, fetching JSON payloads and converting them into Casdoor's OriginalUser format.
The diagram above illustrates the relationship between the core Syncer configuration and the specialized provider implementations.
Sources: object/syncer_wecom.go:30-65, object/syncer_scim.go:30-60, object/syncer_awsiam.go:30-70
Data Flow and Logic
The synchronization process involves a lifecycle of connection testing, fetching original users, and mapping fields to the Casdoor user model.
Mapping Mechanism
The TableColumn structure defines how an external field (e.g., primaryEmail in Google Workspace) maps to a Casdoor field (e.g., Name). It also supports basic transformations like hashing.
// Example mapping for Google Workspace
{
"name": "primaryEmail",
"type": "string",
"casdoorName": "Name",
"isHashed": true,
"values": []
}
Sources: web/src/SyncerEditPage.js:199-228, object/syncer.go:20-27
Synchronization Sequence
The following sequence diagram describes the execution of a synchronization task when triggered by the system or a manual "Sync" request from the UI.
API Endpoints
The synchronization service is managed via several RESTful endpoints as defined in the system's Swagger documentation.
| Endpoint | Method | Description |
|---|---|---|
/api/get-syncers | GET | Retrieves a paginated list of syncers for an owner. |
/api/add-syncer | POST | Creates a new syncer configuration. |
/api/update-syncer | POST | Modifies an existing syncer. |
/api/run-syncer | GET | Manually triggers the synchronization process. |
/api/test-syncer-db | POST | Tests connection to the configured source. |
Sources: swagger/swagger.yml:420-435, swagger/swagger.yml:1020-1045, web/src/backend/SyncerBackend.js:15-79
Implementation Details for API-based Syncers
Modern SaaS integrations utilize specific API protocols rather than direct database access.
WeCom (WeChat Work)
The WeCom provider fetches an access token using corpid and corpsecret, then iterates through departments to collect all members.
- Authentication:
https://qyapi.weixin.qq.com/cgi-bin/gettoken - Data Retrieval:
https://qyapi.weixin.qq.com/cgi-bin/user/list
Sources: object/syncer_wecom.go:73-107
SCIM 2.0
The SCIM provider implements pagination (startIndex and count) to handle large user sets via the standard SCIM /Users endpoint.
- Headers:
Content-Type: application/scim+json - Auth Support: Bearer Token or Basic Auth.
Sources: object/syncer_scim.go:120-140, object/syncer_scim.go:167-210
AWS IAM
The AWS IAM provider uses the AWS SDK to paginate through users and specifically retrieves tags to populate extended metadata like Email or Department.
- Primary Source:
iamClient.ListUsersWithContext - Metadata Source:
iamClient.ListUserTagsWithContext
Sources: object/syncer_awsiam.go:111-180
Conclusion
Casdoor's Data Synchronization Services offer a robust, extensible framework for identity lifecycle management. By abstracting the complexities of different protocols into a standardized Syncer/Provider model, the system ensures that user data remains consistent across the enterprise, whether sourced from legacy SQL databases or modern cloud IAM systems.