Migrations & Seeding

Relevant source files

The following files were used as context for generating this wiki page:

• ground-control/migrator/migrator.go
• ground-control/main.go
• scripts/local-test.sh
• scripts/e2e-test.sh
• taskfiles/e2e.yml
• CLAUDE.md

The Migrations & Seeding system in Harbor Satellite ensures that the Ground Control component has a consistent and up-to-date database schema upon startup. Ground Control uses a PostgreSQL database to manage satellites, groups, configurations, and artifact metadata. The migration logic is embedded within the Ground Control service lifecycle, executing automatically before the HTTP server starts to handle requests.

Sources: ground-control/main.go:26-30, CLAUDE.md:65-68

Database Migration Architecture

Ground Control utilizes the goose migration tool to manage SQL-based schema changes. The migration process is encapsulated in the migrator package, which handles database connection string parsing, availability checks, and the execution of migration scripts.

Migration Lifecycle

When Ground Control starts, it follows a strict sequence:

1. Environment Loading: Database credentials and connection details are read from environment variables.
2. Readiness Check: The system pings the PostgreSQL instance repeatedly (with a timeout) to ensure the database is ready for queries.
3. Migration Execution: goose runs the migration files located in the schema directory to bring the database to the latest version.

Sources: ground-control/migrator/migrator.go:19-79, ground-control/main.go:26-30

Database Configuration

The migrator relies on specific environment variables to construct the PostgreSQL Data Source Name (DSN). By default, it uses a non-SSL connection for local development and internal networking.

Environment Variable	Description	Source File
DB_HOST	Hostname of the PostgreSQL server	migrator.go:23
DB_PORT	Port number (default 5432)	migrator.go:22
DB_DATABASE	Name of the database (e.g., groundcontrol)	migrator.go:19
DB_USERNAME	Database user	migrator.go:21
DB_PASSWORD	Password for the database user	migrator.go:20

The connection string is formatted as: postgres://<username>:<password>@<host>:<port>/<database>?sslmode=disable.
Sources: ground-control/migrator/migrator.go:26-34

Migration Implementation Details

The migration logic is implemented using goose/v3. It dynamically determines the location of migration files based on the environment, typically looking in /migrations (containerized) or sql/schema (local development).

Key Functions

• DoMigrations(): The entry point called by main.go. It opens the database connection and orchestrates the readiness check and migration run.
• waitForPostgresReady(db, timeout): Implements a retry loop that pings the database for up to 60 seconds before failing.
• runMigrations(db): Uses a goose provider to apply all "Up" migrations from the filesystem to the database.

Sources: ground-control/migrator/migrator.go:44-91

func runMigrations(db *sql.DB) {
	migrationsPath := "/migrations"
	if _, err := os.Stat(migrationsPath); os.IsNotExist(err) {
		migrationsPath = "sql/schema"
	}

	provider, err := goose.NewProvider(goose.DialectPostgres, db, os.DirFS(migrationsPath))
	if err != nil {
		log.Fatalf("failed to create goose provider: %v", err)
	}

	ctx := context.Background()
	if _, err := provider.Up(ctx); err != nil {
		log.Fatalf("failed to run migrations: %v", err)
	}
}

Sources: ground-control/migrator/migrator.go:65-81

Seeding and Initialization

While Ground Control handles schema migrations automatically, initial data "seeding" (creating the first groups, configurations, and satellites) is typically handled via API calls during deployment or via E2E testing scripts.

Test Environment Seeding

For local development and testing, scripts like local-test.sh and e2e-test.sh simulate the seeding process by hitting the Ground Control API endpoints.

Seeding Tasks in E2E

The project's Taskfile defines specific initialization steps for E2E environments, covering both Harbor and Ground Control setup.

Task	Purpose	Derived From
init-harbor	Creates projects, pushes images, and sets up replication policies in Harbor.	taskfiles/e2e.yml:218-271
init-gc	Creates the initial test-config and test-group via the Ground Control API.	taskfiles/e2e.yml:273-317
register-satellite	Registers a new satellite and captures the registration token.	taskfiles/e2e.yml:319-350

Sources: taskfiles/e2e.yml:218-350

Summary

Migrations in Harbor Satellite are a critical startup dependency for Ground Control, managed by the goose library to ensure schema consistency across environments. The system is designed to wait for database readiness, making it resilient in containerized environments where the database may start slower than the application. Seeding of initial data is decoupled from the migration logic, instead being handled by API-driven orchestration during the deployment and testing phases.

Sources: ground-control/migrator/migrator.go:44-91, ground-control/main.go:23-35, taskfiles/e2e.yml:273-317