Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• CONTRIBUTING.md
• setup.py
• gensim/test/test_word2vec.py
• gensim/test/test_ldamodel.py
• gensim/test/test_corpora.py
• gensim/test/test_coherencemodel.py
• gensim/test/test_utils.py
• HACKTOBERFEST.md

The Gensim project employs a multi-layered testing strategy designed to ensure the reliability, performance, and mathematical accuracy of its topic modeling and vector space modeling algorithms. The strategy encompasses unit testing, integration testing, and continuous validation across multiple Python versions and operating systems. Because many algorithms in Gensim are stochastic (e.g., LDA, Word2Vec), the testing framework includes specific mechanisms to handle non-deterministic outputs through fixed seeds and convergence retries.

Sources: CONTRIBUTING.md:46-51, setup.py:341-351, gensim/test/test_ldamodel.py:49-65

Test Environment and Orchestration

Gensim uses pytest as its primary test runner and unittest as the base class for most test suites. The environment is configured to support different dependency sets based on the host operating system.

Dependency Management

The project defines specific test environments in its setup configuration:

• core_testenv: Includes pytest, pytest-cov, and testfixtures.
• linux_testenv: Adds visdom and POT (for Earth Mover's Distance/WMD tests).
• win_testenv: Skips certain problematic packages like nmslib while maintaining core functionality.

Sources: setup.py:277-313, CONTRIBUTING.md:31-36

Continuous Integration (CI)

Automated testing is triggered via GitHub Actions for every pull request and commit to the develop branch. This ensures compatibility across Python 3.8 through 3.12.

Testing Methodologies

1. Stochastic Algorithm Validation

Models like LDA and NMF are prone to getting stuck in local minima. The test suite handles this by allowing a fixed number of restarts (retries) during a test execution to verify convergence against expected values.

# Example of convergence retry logic in LDA tests
for i in range(25): # restart at most 25 times
    model = self.class_(id2word=dictionary, num_topics=2, passes=100)
    model.update(self.corpus)
    transformed = model[doc]
    passed = np.allclose(sorted(vec), sorted(expected), atol=1e-1)
    if passed:
        break

Sources: gensim/test/test_ldamodel.py:49-65, gensim/test/test_nmf.py:64-75

2. Regression Testing and Persistence

A critical part of the strategy is ensuring backward compatibility. Gensim stores "old" model files (e.g., from versions 3.8.0 or 0.13.2) and attempts to load them to verify that the current code can still process legacy data.

Test Type	Description	Source File
Persistence	Saving and loading models to/from disk (compressed and uncompressed).	gensim/test/test_word2vec.py:236-245
Backward Compatibility	Loading models created in older Python versions (2.7, 3.5).	gensim/test/test_ldamodel.py:255-264
Binary Compatibility	Testing Matrix Market (MM) and Word2Vec C-format binary loads.	gensim/test/test_word2vec.py:348-356

3. Numerical Sanity Checks

Tests frequently verify that similarity scores are within valid ranges (0 to 1 for cosine similarity) and that word analogies yield mathematically expected results.

Sources: gensim/test/test_word2vec.py:476-490, gensim/test/test_coherencemodel.py:47-60

Corpus and I/O Testing

Gensim supports a wide array of corpus formats (Blei, SVMlight, LowCorpus, etc.). The testing strategy involves:

• Serialization/Deserialization: Testing if saving a corpus and reloading it results in an identical object.
• Index Testing: Verifying that IndexedCorpus subclasses support random access and slicing.
• Compression: Testing I/O with .gz and .bz2 transparent wrappers.

Sources: gensim/test/test_corpora.py:90-110, gensim/test/test_corpora.py:126-140

Developer Requirements

Gensim mandates specific steps for contributors before code can be merged:

1. Environment Isolation: Development must occur within a virtualenv.
2. Editable Installation: Install with pip install -e .[test] to include testing dependencies.
3. Linting: Must pass flake8 with a maximum line length of 120.
4. Unit Tests: Must pass pytest -v gensim/test.

Sources: CONTRIBUTING.md:21-46

Summary

The Gensim testing strategy is centered on algorithmic stability and long-term data persistence. By combining standard unit tests with stochastic retry logic and multi-version regression tests, the project maintains a high degree of reliability for scientific and production use cases.

Sources: CONTRIBUTING.md, gensim/test/test_ldamodel.py, gensim/test/test_word2vec.py