Python API Reference
Relevant source files
The following files were used as context for generating this wiki page:
The Python API for the Duplicate Code Detection Tool provides a structured interface for programmatically analyzing source code similarities. It is designed to be integrated into CI/CD pipelines, pre-commit hooks, and larger software quality suites. The API is divided into a high-level object-oriented interface (via the core module) and a set of functional utilities for direct execution (via the duplicate_code_detection module).
Sources: README.md:12-20, core.py:1-10
High-Level Interface: DuplicateCodeDetector
The DuplicateCodeDetector class serves as the primary entry point for library users. It encapsulates configuration settings such as similarity thresholds and file filters, providing methods to analyze directories, specific file lists, or entire projects.
Class: DuplicateCodeDetector
This class manages the orchestration of the detection process, converting raw similarity data from the underlying engine into structured Python objects.
| Parameter | Type | Default | Description |
|---|---|---|---|
fail_threshold | int | 100 | Maximum allowed similarity percentage before marking as failure. |
ignore_threshold | int | 15 | Minimum similarity percentage required to be included in reports. |
file_extensions | list[str] | ["py", "js", "ts", "java", "cpp", "c", "h"] | Extensions of files to include in analysis. |
only_code | bool | False | If True, removes comments and docstrings from Python files. |
show_loc | bool | True | Whether to include Lines of Code (LoC) count in the results. |
Sources: core.py:108-142, README.md:204-212
Analysis Methods
The detector provides three specialized methods for different scopes of analysis:
analyze_project(project_path, exclude_patterns): Analyzes a directory structure while applying exclusion filters (e.g.,tests/,node_modules/).detect_in_directories(directories, ignore_directories, ignore_files, project_root_dir): Performs analysis across multiple specific directory paths.detect_in_files(files, ignore_files, project_root_dir): Compares a specific list of files against each other.
Sources: core.py:144-245, README.md:220-225
The diagram shows how the high-level class methods interface with the underlying execution engine to return structured results.
Data Models and Results
The API returns results using a hierarchy of dataclasses that represent the levels of similarity detected.
DetectionResult
The top-level container for an analysis run. It contains the final status and a list of reports.
is_success(): ReturnsTrueif no file exceeded thefail_threshold.has_duplications(): ReturnsTrueif any similarities were found above theignore_threshold.get_critical_duplications(threshold): Returns a list ofFileSimilarityobjects exceeding a specific percentage.
Sources: core.py:78-105, README.md:227-238
SimilarityReport
Represents the findings for a single source file.
file_path: Path to the analyzed file.loc_count: Total lines of code.similarities: List ofFileSimilarityobjects targeting other files.
Sources: core.py:58-75, README.md:240-245
FileSimilarity
Represents the similarity between two specific files.
source_file/target_file: Paths to the files being compared.similarity_percentage: The calculated similarity score (0-100).source_loc/target_loc: Respective line counts.
Sources: core.py:37-55, README.md:247-254
Functional API and Utilities
For users who prefer a functional approach or need access to internal utilities, the package exports several functions directly from the core and detection modules.
Quick Analysis Functions
These functions instantiate a temporary DuplicateCodeDetector for one-off tasks.
| Function | Purpose |
|---|---|
analyze_project_duplicates() | Scans a whole project directory with default settings. |
detect_duplicates_in_directories() | Scans specific directories. |
detect_duplicates_in_files() | Scans a provided list of file paths. |
Sources: core.py:248-273, README.md:256-261
Core Logic Functions
Functions used for processing source code before similarity calculation.
run(...): The internal engine that performs the actual TF-IDF similarity calculation usinggensim.remove_comments_and_docstrings(source_code): Uses theastandastorlibraries to strip non-code elements from Python source.get_loc_count(file_path): Calculates the line count of a file, handling potential encoding errors viasurrogateescape.get_all_source_code_from_directory(directory, file_extensions): Recursively crawls directories to find files matching specific extensions.
Sources: duplicate_code_detection.py:53-63, duplicate_code_detection.py:71-101, duplicate_code_detection.py:104-112, duplicate_code_detection.py:220-224
Execution Flow
The similarity detection process follows a specific sequence from file discovery to vector similarity calculation.
This sequence diagram illustrates the data flow from the high-level API through the file processing stages to the final result generation.
Conclusion
The Python API provides a flexible two-tier architecture. The high-level DuplicateCodeDetector and its associated dataclasses provide a modern, type-hinted interface suitable for integration into automated tools. Simultaneously, the underlying functional API offers granular control over the detection engine, pre-processing, and file system operations, allowing for custom similarity detection workflows.
Sources: core.py:1-273, init.py:14-36