Component Interaction

Relevant source files

The following files were used as context for generating this wiki page:

• init.py
• run_action.py
• entrypoint.sh
• duplicate_code_detection.py
• core.py
• action.yml

The Duplicate Code Detection Tool is structured to operate as both a standalone Python library and a GitHub Action. Component interaction is centered around the core detection engine, which utilizes Natural Language Processing (NLP) techniques to compare document similarities. The system orchestrates data flow from various entry points—such as shell scripts for CI/CD, a CLI wrapper, or direct library calls—into a unified analysis pipeline.

The high-level interaction involves three primary layers: the Integration Layer (GitHub Actions and Shell entrypoints), the Orchestration Layer (high-level Python classes), and the Analysis Engine (the functional core that performs tokenization and similarity calculation).

Sources: README.md, core.py:1-12, duplicate_code_detection.py:1-10

Core Detection Architecture

The analysis engine resides in duplicate_code_detection.py and is primarily accessed through the run() function. It coordinates with external libraries like gensim and nltk to process source code files.

Data Processing Flow

1. File Discovery: The system identifies files based on directory paths and extensions.
2. Preprocessing: Optional removal of comments/docstrings for Python files using the ast module.
3. Tokenization: Source code is broken down into lowercase word tokens.
4. Vectorization: Tokens are converted into Bag-of-Words (BoW) and then into TF-IDF (Term Frequency-Inverse Document Frequency) vectors.
5. Similarity Calculation: Matrix similarity is calculated to produce a percentage-based duplication report.

The diagram shows the sequential transformation of raw source files into a similarity index using NLP models.
Sources: duplicate_code_detection.py:183-255, core.py:150-180

Orchestration and Library Interface

The core.py module introduces a high-level DuplicateCodeDetector class that encapsulates the functional run() method. This layer provides a structured Object-Oriented interface for integration.

Key Classes and Relationships

Class	Responsibility	Source
DuplicateCodeDetector	Primary API for directory, file, and project analysis.	core.py:90
DetectionResult	Container for the outcome, success status, and structured reports.	core.py:61
SimilarityReport	Aggregates all similarities found for a specific file.	core.py:44
FileSimilarity	Individual data point representing similarity between two files.	core.py:26

This class diagram illustrates the hierarchical structure of the library's data model and its primary detector interface.
Sources: core.py:26-110

GitHub Action Integration

The tool interacts with GitHub's ecosystem through a multi-stage process involving a Docker entrypoint, an action runner, and the GitHub API.

Execution Sequence

1. entrypoint.sh: Triggered by GitHub, it clones the repository, fetches the specific Pull Request branch, and identifies the latest commit hash.
2. run_action.py: Invoked by the entrypoint. It reads environment variables (prefixed with INPUT_) defined in action.yml.
3. Analysis: Calls the detection engine.
4. Reporting: Formats the results into a Markdown table and uses the GitHub API to post or update a comment on the Pull Request.

The sequence diagram details the interaction between the GitHub infrastructure and the internal Python scripts to post automated reviews.
Sources: entrypoint.sh:5-23, run_action.py:100-155, action.yml:1-55

Configuration Mapping

Environment variables mapped from action.yml to the execution engine:

YAML Input	Internal Parameter	Description
fail_above	fail_threshold	Exit code 2 if similarity exceeds this.
ignore_below	ignore_threshold	Filter out results below this percentage.
only_code	only_code	Flag to strip comments/docstrings.
one_comment	update_existing_comment	Updates previous bot comment if true.

Sources: action.yml:4-48, run_action.py:105-115

Conclusion

Component interaction in this tool is designed for flexibility. The Analysis Engine provides the raw capability, the Orchestration Layer makes it consumable as a Python library, and the Integration Layer automates its deployment within CI/CD pipelines. This layered approach ensures that the complex NLP logic remains decoupled from the specific environment in which it is running.

Sources: core.py, duplicate_code_detection.py, run_action.py