Adding a New Language

Relevant source files

The following files were used as context for generating this wiki page:

• rust-code-analysis-book/src/developers/new-language.md
• src/langs.rs
• src/macros.rs
• enums/src/languages.rs
• enums/src/macros.rs
• enums/src/rust.rs
• rust-code-analysis-book/src/developers/update-grammars.md

Supporting a new language in rust-code-analysis involves a two-stage process: generating the necessary grammar structures and integrating that grammar into the core library. The project relies on Tree-sitter for parsing source code into Abstract Syntax Trees (ASTs), which are then used to compute various Supported Metrics.

The implementation requires modifications across the enums crate (responsible for grammar generation) and the main rust-code-analysis crate (responsible for analysis logic). Developers must ensure that a compatible Tree-sitter version exists for the target language before beginning integration.

Sources: rust-code-analysis-book/src/developers/new-language.md:1-12, README.md:14-17

Grammar Generation

The enums crate is a dedicated tool within the repository used to generate Rust enum files representing the grammar of a language. These enums allow the analysis engine to identify specific node types within an AST to evaluate metrics.

Configuration Steps

1. Dependency Addition: Add the language-specific Tree-sitter crate to enums/Cargo.toml. The version must match the Tree-sitter version used by the main project.
2. Language Registration: Append the language to enums/src/languages.rs. This requires defining a pair: the Enum name (e.g., Rust) and the function to retrieve the grammar (e.g., tree_sitter_rust).
3. Macro Implementation: Update the mk_get_language macro in enums/src/macros.rs to map the new Lang enum variant to the actual Tree-sitter language call.
4. Execution: Run the recreate-grammars.sh script. This script executes the enums crate, which uses templates (via the askama library) to generate a new grammar file in src/languages/.

Sources: rust-code-analysis-book/src/developers/new-language.md:14-29, enums/src/rust.rs:18-34, enums/src/macros.rs:17-32

Grammar Workflow

The following diagram illustrates the flow from a Tree-sitter crate to a generated Rust grammar file:

The diagram shows how metadata and external parser crates are transformed into internal Rust source files through the enums project.
Sources: rust-code-analysis-book/src/developers/new-language.md:14-32, enums/src/rust.rs:21-28

Library Integration

Once the grammar file is generated, the language must be formally registered within the rust-code-analysis library to enable its use in CLI, Web, and API contexts.

Registration Process

1. Main Dependency: Add the Tree-sitter language crate to the main Cargo.toml.
2. Module Exposure: Add the generated grammar module to src/languages/mod.rs using pub mod language_[name];.
3. Language Definition: Define the language metadata in src/langs.rs using the mk_langs! macro.

The mk_langs! Configuration

The mk_langs! macro requires a specific tuple format to initialize the language:

Parameter	Description	Example
Enum Name	The CamelCase identifier for the language	Rust
Description	A brief string describing the language	"The Rust language"
Display Name	The string used for output/display	"rust"
Struct Name	An empty struct for trait implementation	RustCode
Parser Name	The type alias for the parser	RustParser
TS Function	The crate function returning the Language	tree_sitter_rust
Extensions	A list of supported file extensions	[rs]
Emacs Modes	A list of strings for Emacs mode detection	["rust"]

Sources: rust-code-analysis-book/src/developers/new-language.md:34-63, src/langs.rs:13-25, src/macros.rs:271-279

Core Macros and Architecture

Integration is heavily driven by declarative macros in src/macros.rs. These macros generate the boilerplate for language detection, parsing, and metric computation.

Macro Responsibilities

• mk_lang!: Creates the LANG enum and implements get_ts_language() to return the Tree-sitter parser object.
• mk_action!: Generates functions like action, get_function_spaces, and get_ops which dispatch requests to the correct language parser.
• mk_extensions!: Generates the get_from_ext lookup function.
• mk_emacs_mode!: Generates the get_from_emacs_mode lookup function.

Sources: src/macros.rs:88-100, src/macros.rs:125-145, src/macros.rs:214-230

Parser Dispatch Sequence

When a user requests analysis, the system identifies the language and dispatches to the appropriate parser:

The sequence shows the transition from language detection in tools.rs to execution via the action macro.
Sources: src/tools.rs:188-215, src/macros.rs:149-158

Conclusion

Adding a new language is a structured process that leverages Tree-sitter's parsing capabilities. By following the generation steps in the enums crate and the registration steps in src/langs.rs, developers can extend the library's multi-language support while maintaining a unified interface for metric extraction and code analysis.

Sources: rust-code-analysis-book/src/developers/new-language.md:1-12, README.md:1-10