Language Support

Relevant source files

The following files were used as context for generating this wiki page:

• rust-code-analysis-book/src/languages.md
• src/languages/mod.rs
• rust-code-analysis-book/src/developers/new-language.md
• src/macros.rs
• src/tools.rs
• README.md
• src/lib.rs
• enums/src/languages.rs

Language support in rust-code-analysis is built upon the Tree-sitter incremental parsing library, which allows the project to extract syntax and semantic information from source code across multiple programming languages. This infrastructure enables the computation of various maintainability and complexity metrics.

The library supports common languages such as C++, Rust, and Python, as well as specialized internal grammars designed for the Firefox project, such as mozcpp and mozjs.
Sources: README.md:12-16, rust-code-analysis-book/src/README.md:3-8, src/lib.rs:1-8

Supported Languages and Grammars

The project categorizes supported languages into standard third-party grammars and internal grammars. Internal grammars are defined within the project to handle variants of languages specifically used in Firefox.
Sources: rust-code-analysis-book/src/developers/update-grammars.md:14-19

Language Catalog

Language	Description/Variant	Extensions
C	Standard C	.c, .h
C++	Standard C++	.cpp, .hpp, .cc, .hh
Mozcpp	Firefox-specific C++ variant	N/A (Internal)
Rust	Rust Language	.rs
Python	Python Language	.py
Java	Java Language	.java
JavaScript	Standard JavaScript	.js
Mozjs	Firefox-specific JavaScript	N/A (Internal)
TypeScript	TypeScript and TSX	.ts, .tsx

Sources: rust-code-analysis-book/src/languages.md:1-15, src/langs.rs, rust-code-analysis-book/src/developers/new-language.md:65-75

Architecture of Language Support

The architecture relies on a multi-step process involving the enums crate to generate Rust-compatible grammar representations from Tree-sitter.

The diagram shows how external Tree-sitter grammars are processed into internal language definitions used by the library.
Sources: rust-code-analysis-book/src/developers/new-language.md:18-45

Core Components and Macros

The system uses several specialized macros to register and manage languages:

• mk_langs!: The primary entry point for defining a language, its extensions, and its parser.
• mk_lang!: Generates the LANG enum which contains all supported languages.
• mk_extensions!: Maps file extensions to specific LANG variants.
• mk_emacs_mode!: Maps Emacs modes (extracted from file headers) to LANG variants.
• get_language!: Retrieves the specific tree-sitter language object.

Sources: src/macros.rs:1-10, src/macros.rs:80-100, src/macros.rs:200-220, src/macros.rs:245-255

Detection Logic

rust-code-analysis employs a tiered detection strategy to determine the language of a source file, combining file extensions with metadata found within the source code itself.

The diagram illustrates the priority logic used in guess_language to resolve conflicts between file extensions and internal metadata.
Sources: src/tools.rs:207-238

Language Guessing Functions

• get_from_ext(ext: &str): Performs a direct lookup based on the file extension. Sources: src/macros.rs:200-213
• get_emacs_mode(buf: &[u8]): Scans the first and last five lines of a file for Emacs or Vim modelines (e.g., -*- mode: c++; -*-). Sources: src/tools.rs:163-188
• guess_language(buf: &[u8], path: P): The top-level function that coordinates extension and metadata detection to return a LANG variant and its display name. Sources: src/tools.rs:207-238

Implementation for New Languages

Adding a new language involves two primary phases: grammar generation and library integration.

1. Grammar Generation (Enums Crate)

1. Add the tree-sitter-[language] crate to enums/Cargo.toml.
2. Update enums/src/languages.rs with the new language name.
3. Add a match arm to the mk_get_language macro in enums/src/macros.rs.
4. Run ./recreate-grammars.sh to produce the Rust enum file in src/languages/.

Sources: rust-code-analysis-book/src/developers/new-language.md:23-42

2. Library Integration

Developers must register the generated grammar in the main library by providing a definition to mk_langs!.

// Example registration in src/langs.rs
(
    Rust,                      // Enum name
    "The `Rust` language",     // Description
    "rust",                    // Display name
    RustCode,                  // Info struct
    RustParser,                // Parser type
    tree_sitter_rust,          // Tree-sitter function
    [rs],                      // Extensions
    ["rust"]                   // Emacs modes
)

Sources: rust-code-analysis-book/src/developers/new-language.md:65-88

Summary

The language support system provides a unified interface for parsing disparate source codes into a standard Abstract Syntax Tree (AST) format. By leveraging Tree-sitter and a sophisticated detection engine, rust-code-analysis ensures that metrics are computed accurately across different programming paradigms, ranging from system languages like Rust and C++ to web technologies like JavaScript and TypeScript.
Sources: README.md:1-10, src/lib.rs:1-10