Connection Management

Relevant source files

The following files were used as context for generating this wiki page:

• internal/commands/connections_list.go
• internal/commands/connections_describe.go
• internal/opendpi/parse.go
• internal/opendpi/writer.go
• internal/opendpi/writer_test.go
• internal/commands/ports_describe.go

Connection Management is a core system within the Daco CLI designed to handle infrastructure metadata within an OpenDPI specification. It provides the mechanisms to define, list, and inspect the physical or logical locations where data is stored or transmitted, such as PostgreSQL databases or Kafka clusters. These connections act as reusable resources that Ports reference to define data lineage and access points.

The system facilitates interaction between the CLI user and the OpenDPI specification, supporting both programmatic access through structured output (JSON/YAML) and human-centric interaction via interactive prompts and formatted tables.
Sources: internal/commands/connections_list.go:1-30, internal/commands/connections_describe.go:1-35

Data Structure and Models

Connections are defined as key-value pairs within the connections block of an OpenDPI specification. Each connection entry encapsulates the protocol, host address, and optional configuration variables required to establish a link to a data source or sink.

Connection Properties

Field	Type	Description
type	string	The protocol or driver type (e.g., postgresql, kafka, s3).
host	string	The network address or endpoint of the service.
description	string	Optional human-readable explanation of the connection's purpose.
variables	map[string]any	Arbitrary key-value pairs for specific configurations like database names or SSL settings.
Sources: internal/opendpi/parse.go:210-215, internal/opendpi/writer.go:88-95

Internal Representation

The system uses a Spec object to store resolved connections, where the map key serves as the unique identifier for the connection within the document.

type Connection struct {
	Type        string         `yaml:"type" json:"type"`
	Host        string         `yaml:"host" json:"host"`
	Description string         `yaml:"description,omitempty" json:"description,omitempty"`
	Variables   map[string]any `yaml:"variables,omitempty" json:"variables,omitempty"`
}

Sources: internal/opendpi/parse.go:210-215, internal/opendpi/writer.go:40-45

Core Logic and Operations

Connection Resolution and Parsing

The parsing logic transforms raw JSON or YAML input into a structured Spec. During this phase, the parser validates that every connection referenced by a Port actually exists in the connections map.

This diagram shows how raw input strings are converted into logical links between ports and infrastructure definitions.
Sources: internal/opendpi/parse.go:48-60, internal/opendpi/parse.go:102-115

Usage Tracking

A critical feature of connection management is determining which ports are utilizing a specific connection. The CLI calculates this by iterating through all defined ports and matching the connection's Type and Host signatures.

func countConnectionUsage(conn *opendpi.Connection, ports map[string]opendpi.Port) int {
	count := 0
	for _, port := range ports {
		for _, pc := range port.Connections {
			if pc.Connection != nil &&
				pc.Connection.Type == conn.Type &&
				pc.Connection.Host == conn.Host {
				count++
				break
			}
		}
	}
	return count
}

Sources: internal/commands/connections_list.go:107-120, internal/commands/connections_describe.go:142-160

User Interface and Commands

Listing Connections

The list command provides a summary of all infrastructure defined in the spec. It supports multiple output formats:

• Table: A human-readable summary including name, type, host, and usage count.
• JSON/YAML: Raw data export for automation.

The table view specifically identifies how many ports are currently "using" each connection, providing a high-level view of infrastructure dependency.
Sources: internal/commands/connections_list.go:50-80

Describing Connections

The describe command provides a deep-dive into a specific connection. If a connection name is not provided as an argument, the CLI initiates an interactive selection prompt using the huh library.

The sequence above illustrates the interactive flow for inspecting connection details.
Sources: internal/commands/connections_describe.go:45-90

Connection Matching for Persistence

When writing a spec back to disk, the Writer must convert internal object pointers back into string references (e.g., #/connections/my_db). The system performs a reverse lookup by comparing the attributes of the connection object against the entries in the main connection map.
Sources: internal/opendpi/writer.go:47-65, internal/opendpi/writer_test.go:94-115

Summary

Connection Management provides the structural backbone for defining "where" data resides in an OpenDPI specification. By decoupling infrastructure details from the logical data ports, the system allows for reusable, maintainable, and easily inspectable data product definitions. The CLI supports this through robust parsing, dependency tracking, and multi-format reporting tools.