Lock Table Creation

Relevant source files

The following files were used as context for generating this wiki page:

• src/main/java/liquibase/ext/clickhouse/sqlgenerator/CreateDatabaseChangeLogLockTableClickHouse.java
• src/main/java/liquibase/ext/clickhouse/sqlgenerator/InitializeDatabaseChangeLogLockClickHouse.java
• src/main/java/liquibase/ext/clickhouse/lockservice/ClickHouseLockService.java
• src/main/java/liquibase/ext/clickhouse/sqlgenerator/SqlGeneratorUtil.java
• src/main/java/liquibase/ext/clickhouse/database/ClickHouseDatabase.java
• src/main/java/liquibase/ext/clickhouse/params/ParamsLoader.java
• src/main/java/liquibase/ext/clickhouse/params/ClusterConfig.java

The Lock Table Creation process in the Liquibase ClickHouse extension ensures that a dedicated tracking table exists within the ClickHouse database to manage concurrent migration access. This mechanism prevents multiple Liquibase instances from executing changesets simultaneously on the same database, which is critical for maintaining data integrity in distributed environments.

The system utilizes specialized SQL generators to handle the unique syntax requirements of ClickHouse, including support for cluster-aware deployments and specific storage engines like MergeTree or ReplicatedMergeTree.

Architecture and Core Components

The lock management system is built upon the standard Liquibase extension points, specifically overriding the lock service and SQL generation logic to accommodate ClickHouse's architectural nuances.

ClickHouseLockService

The ClickHouseLockService manages the lifecycle of the lock table. It checks for the table's existence and ensures it is properly initialized with a seed record before any locking operations occur.

• Priority: Sets PRIORITY_DATABASE to ensure it is selected over the standard Liquibase implementation when a ClickHouse database is detected.
• Initialization Check: Validates if the table exists and contains the required row (ID=1) to facilitate locking.

Sources: src/main/java/liquibase/ext/clickhouse/lockservice/ClickHouseLockService.java:34-45

SQL Generation Logic

The generation of the lock table is handled by CreateDatabaseChangeLogLockTableClickHouse. This class constructs a CREATE TABLE statement tailored to ClickHouse, incorporating cluster configurations if provided.

This diagram illustrates the decision path for generating SQL based on whether the extension is running in single-instance or cluster mode.
Sources: src/main/java/liquibase/ext/clickhouse/sqlgenerator/CreateDatabaseChangeLogLockTableClickHouse.java:44-64, src/main/java/liquibase/ext/clickhouse/sqlgenerator/SqlGeneratorUtil.java:43-57

Data Model: DATABASECHANGELOGLOCK

The lock table structure is defined with specific ClickHouse data types to support atomic updates and nullable fields for metadata.

Column Name	Data Type	Description
ID	Int64	Primary identifier for the lock record (typically 1).
LOCKED	UInt8	Boolean flag (0 or 1) indicating if the database is currently locked.
LOCKGRANTED	Nullable(DateTime64)	Timestamp when the lock was acquired.
LOCKEDBY	Nullable(String)	Identifier of the host or process holding the lock.

Sources: src/main/java/liquibase/ext/clickhouse/sqlgenerator/CreateDatabaseChangeLogLockTableClickHouse.java:54-58

Table Initialization

Once created, the table must be initialized with a starting record. This is handled by InitializeDatabaseChangeLogLockClickHouse.

1. Cleanup: An ALTER TABLE ... DELETE WHERE 1 statement is issued to clear any existing records. This is executed with mutations_sync = 1 to ensure the operation completes before proceeding.
2. Seed Record: An INSERT INTO ... (ID, LOCKED) VALUES (1, 0) statement creates the initial unlocked state.

Sources: src/main/java/liquibase/ext/clickhouse/sqlgenerator/InitializeDatabaseChangeLogLockClickHouse.java:48-63

Engine Selection and Cluster Support

The engine used for the lock table depends on the configuration loaded from liquibaseClickhouse.conf.

• Single Instance: Uses ENGINE MergeTree() ORDER BY ID.
• Cluster Mode: Uses ENGINE ReplicatedMergeTree with parameters for ZooKeeper path and replica name. It also appends the ON CLUSTER clause to the table creation DDL.

The relationship between configuration and SQL generation utility.
Sources: src/main/java/liquibase/ext/clickhouse/params/ClusterConfig.java:23-31, src/main/java/liquibase/ext/clickhouse/sqlgenerator/SqlGeneratorUtil.java:48-57

Conclusion

Lock Table Creation in the ClickHouse extension is a specialized workflow that adapts Liquibase's standard locking mechanism to ClickHouse's unique storage engines and distributed nature. By leveraging ReplicatedMergeTree and ON CLUSTER clauses, the extension ensures that locking is consistent across all nodes in a ClickHouse cluster.