Additional Topic Models
Relevant source files
The following files were used as context for generating this wiki page:
Gensim provides a diverse suite of unsupervised document analysis and topic modeling algorithms beyond the standard Latent Dirichlet Allocation (LDA). These "Additional Topic Models" include specialized implementations such as Non-Negative Matrix Factorization (NMF), the Author-Topic Model (ATModel), and Dynamic Topic Models (LDAMulticore/LdaSeqModel). These models allow developers to extract semantic structures from large corpora with varying constraints, such as author attribution or temporal evolution.
The architecture of these models follows Gensim's core principles: memory independence (streaming data processing) and intuitive transformation interfaces. Most models inherit from interfaces.TransformationABC, ensuring they can be plugged into standard transformation pipelines and used for similarity retrieval or document indexing.
Non-Negative Matrix Factorization (NMF)
NMF is a collaborative filtering and topic modeling algorithm that factorizes a non-negative matrix (the corpus) into two non-negative matrices representing topics and document-topic distributions. Gensim's implementation supports online learning, allowing it to process corpora larger than RAM.
Logic and Architecture
The nmf.Nmf class implements coordinate descent with Projected Gradient Descent (PGD) for its updates. It is designed to find a low-rank approximation of the term-document matrix. Unlike LDA, which is probabilistic, NMF is based on linear algebra constraints.
The diagram shows the decomposition of the input corpus into two factors that represent the underlying thematic structure of the documents.
Sources: gensim/models/nmf.py
Key Parameters
| Parameter | Type | Description |
|---|---|---|
num_topics | int | Number of requested topics. |
chunksize | int | Number of documents to be used in each training chunk. |
passes | int | Number of passes through the corpus during training. |
kappa | float | Gradient step size. |
random_state | int/RandomState | Seed for reproducibility. |
Sources: gensim/test/test_nmf.py:24-31, gensim/models/nmf.py
Author-Topic Model (ATModel)
The Author-Topic Model extends LDA by including authorship information. It represents each author as a probability distribution over topics and each document as a mixture of topics contributed by its authors.
Architecture and Data Flow
The model requires a mapping between authors and documents (author2doc) or documents and authors (doc2author). During training, it attributes the observed words in a document to the associated authors.
This sequence illustrates how authorship data is integrated into the topic inference process to produce specific author profiles.
Sources: gensim/test/test_atmodel.py:46-60
Key Functions
get_author_topics(author_name): Returns the topic distribution for a specific author.get_new_author_topics(corpus): Infers topic distributions for authors not seen during training based on a provided corpus.update(corpus, author2doc): Updates the model with new documents and author assignments.
Sources: gensim/test/test_atmodel.py:118-125, gensim/test/test_atmodel.py:276-285
Dynamic Topic Models (LdaSeqModel)
The LdaSeqModel (Discrete Temporal Model) is used to analyze how topics evolve over time. It requires the corpus to be divided into chronological "time slices."
Temporal Evolution Logic
The model assumes that the topics in one time slice are a transformation of the topics in the previous slice. It tracks the drift of word probabilities within a topic across the timeline.
The flowchart demonstrates the sequential dependency of topic states across defined time slices in the LdaSeqModel.
Sources: gensim/test/test_ldaseqmodel.py:192-205
Data Structure for DTM
Training requires a time_slice list where each integer represents the number of documents in that sequential period. For example, [10, 10, 11] indicates three time periods with 10, 10, and 11 documents respectively.
Sources: gensim/test/test_ldaseqmodel.py:214-218
Evaluation and Coherence
To determine the quality of these additional models, Gensim provides the CoherenceModel. It implements a four-stage pipeline to calculate the "interpretability" of topics.
Coherence Pipeline
- Segmentation: Dividing topics into pairs of words or word sets.
- Probability Estimation: Calculating word occurrences (e.g., via Sliding Window or BoW).
- Confirmation Measure: Scoring how strongly words in a segment support each other.
- Aggregation: Computing the mean of scores to provide a single coherence value.
Supported Measures
| Measure | Basis | Description |
|---|---|---|
u_mass | BoW | Based on document co-occurrence; intrinsic and fast. |
c_v | Sliding Window | Uses indirect confirmation measure (cosine similarity); highly correlated with human judgment. |
c_uci | Sliding Window | Based on Pointwise Mutual Information (PMI). |
c_npmi | Sliding Window | Normalized PMI score. |
Sources: gensim/models/coherencemodel.py:46-77
Summary
The additional topic models in Gensim, including NMF, ATModel, and LdaSeqModel, provide specialized semantic analysis capabilities. While standard LDA offers a general-purpose probabilistic approach, NMF provides an algebraic alternative for high-performance decomposition, ATModel incorporates social context through authorship, and LdaSeqModel captures historical trends. The CoherenceModel serves as a unified framework to validate and compare these diverse algorithms objectively.
Sources: README.md:37-45, gensim/models/nmf.py, gensim/models/atmodel.py, gensim/models/coherencemodel.py