Cluster Configuration Model
Relevant source files
The following files were used as context for generating this wiki page:
- src/main/java/liquibase/ext/clickhouse/params/ClusterConfig.java
- src/main/java/liquibase/ext/clickhouse/params/ParamsLoader.java
- src/main/java/liquibase/ext/clickhouse/sqlgenerator/SqlGeneratorUtil.java
- src/main/java/liquibase/ext/clickhouse/sqlgenerator/CreateDatabaseChangeLogTableClickHouse.java
- README.md
- src/test/resources/testLiquibaseClickhouse.conf
The Cluster Configuration Model provides the necessary infrastructure to manage Liquibase operations within a ClickHouse cluster environment. It enables the replication of Liquibase's internal tracking tables (the DatabaseChangeLog and DatabaseChangeLogLock tables) across multiple nodes, ensuring that schema migration state is synchronized and consistent throughout the cluster.
When cluster mode is activated, Liquibase transitions from using standard MergeTree engines to ReplicatedMergeTree engines for its internal tables. This ensures that metadata regarding executed change sets is shared among all replicas, preventing duplicate executions and maintaining a unified migration history across the distributed system.
Sources: README.md:21-25, src/main/java/liquibase/ext/clickhouse/sqlgenerator/SqlGeneratorUtil.java:49-55
Core Components and Data Structures
The model is built around a centralized configuration object and a specialized loader that handles the transition from flat file definitions to active Java objects.
ClusterConfig Class
The ClusterConfig class is a Plain Old Java Object (POJO) that stores the mandatory parameters for cluster operation. It holds the cluster name, the ZooKeeper path prefix for table replication, and the specific replica name.
Sources: src/main/java/liquibase/ext/clickhouse/params/ClusterConfig.java:24-37
Configuration Parameters
| Parameter | Type | Description |
|---|---|---|
clusterName | String | The name of the ClickHouse cluster as defined in the server configuration. |
tableZooKeeperPathPrefix | String | The ZK path where replication metadata for Liquibase tables is stored. |
tableReplicaName | String | The identifier for the specific replica instance. |
Sources: src/main/java/liquibase/ext/clickhouse/params/ClusterConfig.java:25-27, README.md:21-25
Parameter Loading Logic
The system utilizes a ParamsLoader to retrieve cluster settings from a configuration file named liquibaseClickhouse.conf. This loader uses the Typesafe Config library to parse HOCON/JSON formatted files.
Validation Workflow
The loader enforces strict validation of the properties. If any of the three required properties (clusterName, tableZooKeeperPathPrefix, tableReplicaName) are missing, or if unknown properties are present, the system defaults to "single-instance mode".
SQL Generation for Clusters
The SqlGeneratorUtil class uses the ClusterConfig to modify standard SQL statements into cluster-aware statements. This involves two primary modifications: adding the ON CLUSTER clause and configuring the ReplicatedMergeTree engine.
On Cluster Clause
If cluster configuration is present, every DDL and mutation operation (like ALTER TABLE ... UPDATE or DELETE) is appended with an ON CLUSTER '{clusterName}' clause. This ensures the command is distributed to all nodes in the cluster.
Sources: src/main/java/liquibase/ext/clickhouse/sqlgenerator/SqlGeneratorUtil.java:44-47, src/main/java/liquibase/ext/clickhouse/sqlgenerator/UpdateChangeSetChecksumClickHouse.java:54-58
Engine Transformation
The system dynamically selects the table engine based on the availability of a ClusterConfig.
Logic Comparison:
- Standard Mode:
ENGINE MergeTree() ORDER BY ID - Cluster Mode:
ENGINE ReplicatedMergeTree('{path}', '{replica}') ORDER BY ID
Sources: src/main/java/liquibase/ext/clickhouse/sqlgenerator/SqlGeneratorUtil.java:49-55
Implementation in Liquibase Tables
The CreateDatabaseChangeLogTableClickHouse and CreateDatabaseChangeLogLockTableClickHouse classes implement the logic to create the tracking tables using the cluster configuration.
Replicated ChangeLog Table Example
When creating the DATABASECHANGELOG table in a cluster, the SQL is constructed as follows:
- The table name is retrieved.
- The
ON CLUSTERclause is injected after the table name. - Column definitions are added.
- The
ReplicatedMergeTreeengine is configured using thetableZooKeeperPathPrefixconcatenated with the lower-cased table name.
Summary of Configuration Usage
The cluster configuration influence is pervasive across the extension's SQL generators to ensure atomicity across the cluster.
| File | Operation | Cluster Modification |
|---|---|---|
CreateDatabaseChangeLogTableClickHouse | Table Creation | Adds ON CLUSTER and ReplicatedMergeTree |
LockDatabaseChangeLogClickHouse | Lock Acquisition | Adds ON CLUSTER to ALTER TABLE ... UPDATE |
UpdateChangeSetChecksumClickHouse | Checksum Update | Adds ON CLUSTER to mutation |
TagDatabaseGeneratorClickhouse | Tagging | Adds ON CLUSTER to ALTER TABLE ... UPDATE |
Sources: src/main/java/liquibase/ext/clickhouse/sqlgenerator/LockDatabaseChangeLogClickHouse.java:56-61, src/main/java/liquibase/ext/clickhouse/sqlgenerator/TagDatabaseGeneratorClickhouse.java:60-65, src/main/java/liquibase/ext/clickhouse/sqlgenerator/CreateDatabaseChangeLogTableClickHouse.java:51-81
The Cluster Configuration Model ensures that Liquibase treats a ClickHouse cluster as a single logical unit by leveraging ClickHouse's native replication and distributed DDL capabilities through a standardized configuration interface.