Development Environment Setup
Relevant source files
The following files were used as context for generating this wiki page:
Setting up the development environment for the Duplicate Code Detection Tool requires a Python environment compatible with version 3.8 or higher, including support for Python 3.13. The project utilizes modern packaging standards through pyproject.toml and setup.py to manage dependencies and development tools.
This setup ensures that developers can contribute to the core library, which leverages NLP and AST-based analysis to identify code similarities across various programming languages.
Sources: pyproject.toml:1-40, README.md:1-15, setup.py:1-40
Prerequisites and Core Dependencies
The tool relies on several key libraries for text processing and code analysis. These must be present in the development environment for the tool to function as a library or via the CLI.
| Dependency | Minimum Version | Description |
|---|---|---|
python | 3.8 | Base runtime environment |
gensim | 4.0.0 | Used for document similarity and TF-IDF modeling |
nltk | 3.8 | Used for tokenizing source code into words |
astor | 0.8.1 | Used for AST manipulation and code cleaning |
numpy | 1.21.0 | Supporting numerical computations for similarity |
Sources: pyproject.toml:41-42, requirements.txt:1-3, duplicate_code_detection.py:10-25
Installation for Development
To set up a local development environment, the repository should be cloned and installed in editable mode along with development-specific dependencies.
# Clone the repository
git clone https://github.com/flext-sh/duplicate-code-detection-tool.git
cd duplicate-code-detection-tool
# Install development dependencies in editable mode
pip install -e ".[dev]"
The [dev] extra includes tools for testing, linting, and type checking to maintain code quality.
Sources: README.md:270-275, pyproject.toml:44-52
Development Tools Summary
| Tool Category | Packages | Purpose |
|---|---|---|
| Testing | pytest, pytest-cov | Execution of unit tests and coverage reporting |
| Formatting | black, isort | Automatic code formatting and import sorting |
| Linting | flake8, mypy | Style checking and static type analysis |
| Hooks | pre-commit | Managing git hooks for automated checks |
Sources: pyproject.toml:45-52
Project Structure and Build System
The project uses setuptools as the build backend, configured via pyproject.toml. The structure supports both library usage and a console script entry point.
Build and Package Configuration
The diagram shows how the build system is structured to produce a package with a functional CLI entry point.
Sources: pyproject.toml:1-5, setup.py:41-45
Static Analysis and Quality Gates
The environment is pre-configured with strict rules for mypy and pytest to ensure code reliability across the supported Python versions (3.8 to 3.13).
Mypy Configuration
The tool enforces strict typing, including requirements for:
disallow_untyped_defs = truecheck_untyped_defs = truewarn_unreachable = true- Overrides for third-party modules like
gensimandnltkthat lack type stubs.
Sources: pyproject.toml:79-105
Testing Workflow
Developers should run the test suite locally before submitting changes.
The sequence shows the standard testing flow using the provided configuration.
Sources: pyproject.toml:107-128, README.md:275-285
Environment Verification
To verify the setup, developers can run the integrated help command of the CLI tool to ensure all dependencies are correctly mapped and the entry point is functional.
duplicate-code-detection --help
Successful execution confirms that the argparse configuration and the DuplicateCodeDetector core logic are accessible within the current environment.
Sources: duplicate_code_detection.py:105-150, setup.py:42-44
Summary
The development environment for this tool is built on standard Python packaging tools, prioritizing quality through comprehensive dev-dependencies (black, mypy, pytest). By installing with the [dev] flag, contributors gain access to the full suite of tools required to maintain the library's similarity detection capabilities and broad Python version compatibility.