Documentation Generation Tool

Relevant source files

The following files were used as context for generating this wiki page:

• cmd/gendocs/main.go
• internal/opendpi/parse.go
• internal/opendpi/writer.go
• internal/commands/ports_describe.go
• CONTRIBUTING.md

The Documentation Generation Tool is a utility within the Daco CLI project designed to generate LLM-friendly markdown documentation. Its primary purpose is to programmatically extract the CLI's command structure and translator capabilities to produce standardized documentation files, typically stored in the ./docs/cli directory.

The tool leverages the cobra/doc package to traverse the command tree and integrates with the various schema translators (such as Avro, Protobuf, and PySpark) to ensure that the generated documentation accurately reflects the available data transformation capabilities of the CLI.

Sources: cmd/gendocs/main.go:1-15, CONTRIBUTING.md:65-67

Architecture and Command Flow

The documentation tool operates by bootstrapping the entire CLI command structure, including all registered translators, and then exporting that structure to Markdown.

Execution Logic

1. Directory Setup: The tool accepts an optional output directory argument, defaulting to ./docs/cli. It ensures this directory exists using os.MkdirAll with 0o750 permissions.
2. Translator Registration: It initializes a translate.Register and populates it with various language-specific translator instances.
3. Command Rooting: It creates a new instance of the Root Command (NewRootCmd) using the registered translators.
4. Markdown Generation: It utilizes doc.GenMarkdownTree to recursively generate documentation for every command and subcommand.

Sources: cmd/gendocs/main.go:34-62

Component Relationship

The following diagram illustrates how the gendocs command interacts with the internal CLI logic to produce documentation:

The diagram shows the initialization flow from the entry point to the generation of markdown files on the file system.
Sources: cmd/gendocs/main.go:34-68

Supported Translators for Documentation

The documentation generation includes metadata and help text for all supported schema translators. These translators are critical for converting JSON schemas into target language structures.

Translator Name	Target Format	Package Reference
pyspark	PySpark Types	internal/translate/pyspark
avro	Avro Schema	internal/translate/avro
protobuf	Proto3	internal/translate/protobuf
pydantic	Pydantic Models	internal/translate/pydantic
databricks-sql	Databricks SQL	internal/translate/databrickssql
gotypes	Go Structs	internal/translate/gotypes

Sources: cmd/gendocs/main.go:44-55

Documentation of Ports and Schemas

While the gendocs tool handles command-line documentation, the CLI also provides internal mechanisms for documenting and describing data ports and their associated OpenDPI schemas.

Port Description Logic

The ports describe command complements the static documentation by providing real-time technical details of specific ports. This includes:

• Schema Layout: Printing types, references, and descriptions for properties.
• Connection Metadata: Listing hostnames and protocols for port locations.
• Recursive Definitions: Handling $defs within the documentation output.

Sources: internal/commands/ports_describe.go:94-135

Data Flow for Schema Documentation

The system parses OpenDPI specifications (YAML or JSON) and resolves references before they can be documented or translated.

This flow ensures that documentation generated or displayed by the CLI contains fully resolved type information rather than broken pointers.
Sources: internal/opendpi/parse.go:60-120, internal/commands/ports_describe.go:142-160

Implementation Details

The core of the documentation generator is found in cmd/gendocs/main.go. It specifically disables the "Auto-generated by spf13/cobra" tag to maintain clean, professional documentation.

func main() {
	dir := "./docs/cli"
	if len(os.Args) > 1 {
		dir = os.Args[1]
	}

	translators := make(translate.Register)
	// ... translator registration ...

	rootCmd := commands.NewRootCmd(translators)
	rootCmd.DisableAutoGenTag = true

	if err := doc.GenMarkdownTree(rootCmd, dir); err != nil {
		fmt.Fprintf(os.Stderr, "error generating docs: %v\n", err)
		os.Exit(1)
	}
}

Sources: cmd/gendocs/main.go:34-68

Summary

The Documentation Generation Tool provides a bridge between the CLI's implementation and its external documentation. By programmatically traversing command structures and translator registrations, it ensures that developers and LLMs have access to accurate, up-to-date information regarding command usage, schema translations, and OpenDPI port configurations. This automated approach, coupled with the ports describe functionality, creates a comprehensive documentation ecosystem for the Daco CLI.