Translator Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• internal/translate/databricksscala/translator.go
• internal/translate/databrickspyspark/translator.go
• internal/translate/sparkscala/translator.go
• internal/translate/sparksql/translator.go
• internal/translate/pydantic/translator.go
• internal/translate/scala/translator.go
• internal/translate/databrickssql/translator.go

The Translator Architecture in Daco CLI is a modular system designed to convert JSON Schemas into various target data structures, including programming language classes (Scala, Python), database schemas (Spark SQL, Databricks SQL), and Big Data framework types (PySpark, Spark Scala). The system leverages a standard preparation phase to normalize schema data before applying target-specific templates.

Sources: internal/translate/scala/translator.go, internal/translate/pydantic/translator.go

Core Components and Workflow

The translation process follows a structured pipeline where a raw JSON schema is prepared, resolved, and finally rendered into the target format using Go's text/template engine.

Translation Pipeline Flow

The following diagram illustrates the lifecycle of a translation request from the initial Go Translate call to the final output generation.

This flow ensures that complex logic like reference resolution and type mapping is handled consistently across all output formats.

Sources: internal/translate/databricksscala/translator.go:31-34, internal/translate/pydantic/translator.go:33-36

The Translator Interface

Every translator implementation adheres to a common structure, providing two primary methods:

• FileExtension(): Returns the specific suffix for the generated file (e.g., .scala, .py, .sql).
• Translate(): Orchestrates the preparation of data and the execution of the associated template.

Sources: internal/translate/sparkscala/translator.go:24-32, internal/translate/sparksql/translator.go:28-36

Supported Target Implementations

The architecture supports multiple targets, categorized by their output type and target platform.

Target Format	File Extension	Implementation Details
Pydantic	.py	Generates Python BaseModel definitions; manages optional/required field ordering.
Scala	.scala	Generates Scala case class definitions; supports package naming based on output directory.
PySpark	.py	Generates pyspark.sql.types.StructType definitions.
Spark SQL	.sql	Generates CREATE TABLE DDL; inlines nested structures as STRUCT.
Databricks SQL	.sql	Similar to Spark SQL but includes USING DELTA and specific comment escaping.

Sources: internal/translate/pydantic/translator.go:28-31, internal/translate/scala/translator.go:31-34, internal/translate/databrickssql/translator.go:41-44

Logic and Data Transformation

Schema Preparation and Resolution

The translate.Prepare function is the entry point for all translators. It utilizes a resolver to map JSON schema types to the specific types of the target language. For example, in the Pydantic translator, the logic explicitly checks for datetime types to determine if additional imports are required.

Sources: internal/translate/pydantic/translator.go:38-53

Field Ordering and Nullability

For programming languages like Scala and Python (Pydantic), the translators implement specific sorting logic to ensure that required fields appear before optional fields in the generated constructors or class definitions.

Struct Inlining (SQL Targets)

Unlike language targets that often keep definitions separate, SQL translators (Spark SQL and Databricks SQL) use a recursive inlineStruct function. This function replaces schema references ($ref) with inline STRUCT<...> definitions to create a single CREATE TABLE statement.

func inlineStruct(fields []translate.Field, defs map[string]*translate.TypeDef, visited map[string]bool) {
	for i := range fields {
		name := fields[i].Type
		if def, ok := defs[name]; ok {
			if visited[name] {
				continue
			}
			visited[name] = true
			inlineStruct(def.Fields, defs, visited)
			fields[i].Type = renderStruct(def.Fields)
			delete(visited, name)
		}
	}
}

Sources: internal/translate/sparksql/translator.go:55-70, internal/translate/databrickssql/translator.go:58-73

Template Execution

The final stage of translation uses embedded file systems (embed.FS) to load templates. This ensures that the CLI binary is self-contained. The templates use custom function maps for operations like:

• last: Detecting the final field in a collection to avoid trailing commas.
• sqlEscape: Escaping single quotes in SQL comments.
• ToPascalCase: Formatting names for Scala objects or Python classes.

Sources: internal/translate/databrickssql/translator.go:21-28, internal/translate/databricksscala/translator.go:17-21

Conclusion

The Translator Architecture provides a robust framework for multi-language code generation. By separating the schema normalization (Prepare) from the target rendering (Templates), the system maintains high consistency while allowing for platform-specific optimizations like SQL struct inlining or Scala field sorting.