Adding Custom Languages

Relevant source files

The following files were used as context for generating this wiki page:

• code_review_graph/custom_languages.py
• tests/test_custom_languages.py
• code_review_graph/parser.py
• CONTRIBUTING.md
• CLAUDE.md

code-review-graph provides a configuration-driven mechanism to extend its parsing capabilities beyond built-in languages. This allows users to support additional languages by mapping file extensions and Tree-sitter node types to specific grammars available in the tree-sitter-language-pack.

The system supports two primary methods for adding languages: local configuration via a languages.toml file for project-specific needs, and upstream contributions to the core package for permanent built-in support.

Sources: CLAUDE.md, CONTRIBUTING.md:52-54, tests/test_custom_languages.py:1-6

Configuration-Driven Custom Languages

Custom languages can be defined without modifying the project's source code by placing a languages.toml file within the .code-review-graph/ directory of a repository.

Configuration Structure

The languages.toml file uses a specific schema where each language is defined as a table under the [languages] header.

Field	Type	Description
extensions	List of Strings	File extensions to map to this language (must start with a dot).
grammar	String	The name of the Tree-sitter grammar to use.
function_node_types	List of Strings	Node types in the AST that represent functions.
class_node_types	List of Strings	Node types in the AST that represent classes/records.
import_node_types	List of Strings	Node types in the AST that represent imports.
call_node_types	List of Strings	Node types in the AST that represent function calls.
comment	String	Optional description of the custom language configuration.

Sources: code_review_graph/custom_languages.py, tests/test_custom_languages.py:16-25

Validation Rules

The loader enforces several safety and correctness constraints:

• Extension Format: Extensions must be lowercase and must start with a . (e.g., .erl).
• Shadowing Protection: Custom languages cannot use extensions or language names already claimed by built-in support.
• Grammar Availability: The specified grammar must exist within the tree_sitter_language_pack.
• Resource Limits: A maximum of 10 custom languages (defined by MAX_CUSTOM_LANGUAGES) can be loaded per project.
• Type Requirements: Node types must be provided as lists of non-empty strings.

Sources: code_review_graph/custom_languages.py:14, tests/test_custom_languages.py:101-145

Architectural Data Flow

The following diagram illustrates how custom languages are resolved and integrated into the CodeParser during the parsing process.

The CodeParser class is responsible for triggering the loading of custom languages if a repo_root is provided during initialization.

Sources: code_review_graph/custom_languages.py:136-150, code_review_graph/parser.py, tests/test_custom_languages.py:166-170

Implementation Example: Erlang

Erlang serves as the primary test case for custom language implementation, as it is included in the grammar pack but not supported by default in the core package.

[languages.erlang]
extensions = [".erl"]
grammar = "erlang"
function_node_types = ["function_clause"]
class_node_types = ["record_decl"]
import_node_types = ["import_attribute"]
call_node_types = ["call"]

Sources: tests/test_custom_languages.py:16-24

Adding Built-in Language Support

To add permanent language support to the code-review-graph upstream repository, developers must follow a structured integration process.

Integration Steps

1. Extension Mapping: Add the new file extensions to the EXTENSION_TO_LANGUAGE dictionary in code_review_graph/parser.py.
2. Node Type Registration: Define the Tree-sitter node types by updating _CLASS_TYPES, _FUNCTION_TYPES, _IMPORT_TYPES, and _CALL_TYPES.
3. Fixture Creation: Add a sample source file for the new language in the tests/fixtures/ directory.
4. Testing: Implement parsing tests within tests/test_multilang.py to ensure the AST is correctly mapped to graph nodes and edges.

Sources: CONTRIBUTING.md:52-60, code_review_graph/parser.py

Summary

Adding custom languages allows code-review-graph to maintain token-efficient analysis for diverse codebases. Whether through the local languages.toml configuration for rapid deployment or through upstream contributions to parser.py for standard support, the system leverages Tree-sitter node types to build structural graphs of functions, classes, and calls.