KeyedVectors (Word Embeddings)
Relevant source files
The following files were used as context for generating this wiki page:
KeyedVectors is a fundamental data structure in Gensim designed to store and manage sets of vectors keyed by tokens (usually words). While full models like Word2Vec or FastText contain the internal neural network weights and state necessary for training, KeyedVectors focuses exclusively on the final trained vectors. This separation allows for significantly reduced RAM usage, faster loading through memory mapping (mmap), and simplified concurrent queries.
Sources: gensim/models/keyedvectors.py:11-30, gensim/models/word2vec.py:65-75
Core Architecture and Data Structures
The KeyedVectors class maps keys (strings or integers) to 1D numpy arrays. It manages the alignment between the lookup keys and the underlying raw vector matrix.
Primary Components
vectors: A 2D numpy array of shape(count, vector_size)containing the raw embedding values.index_to_key: A list that maps an integer index to the actual key (token).key_to_index: A dictionary that maps a key back to its integer index in thevectorsarray.norms: A 1D array of L2-norms for each vector, used to speed up similarity calculations.expandos: A dictionary storing additional per-vector attributes (e.g., word counts).
Sources: gensim/models/keyedvectors.py:170-195
Data Relationship Diagram
The following diagram illustrates how keys are mapped to their corresponding vector representations.
Key Features and Comparison
KeyedVectors is optimized for inference and similarity tasks. Unlike full models, it cannot be used to continue training, but it offers several operational advantages.
| Capability | KeyedVectors | Full Model | Note |
|---|---|---|---|
| Continue Training | ❌ | ✅ | Requires full model state and hidden weights. |
| Smaller Objects | ✅ | ❌ | Sheds internal training structures (syn1, syn1neg). |
| Shared RAM | ✅ | ✅ | Multiple processes can share data via mmap. |
| Native Formats | ✅ | ❌ | Supports loading Google/Facebook C formats. |
| Fast Load | ✅ | ✅ | Instant loading from disk using mmap. |
Sources: gensim/models/keyedvectors.py:31-48, gensim/models/word2vec.py:68-80
Similarity Operations
KeyedVectors provides high-performance methods for calculating relationships between vectors, primarily utilizing cosine similarity.
Semantic Queries
most_similar: Computes cosine similarity between a weighted mean of input keys and all vectors in the model.most_similar_cosmul: Uses the multiplicative combination objective (3CosMul) for analogy solving.similarity: Returns the cosine similarity between two specific keys.doesnt_match: Identifies the key that is semantically most distant from others in a list.
Sources: gensim/models/keyedvectors.py:530-580, gensim/models/keyedvectors.py:734-750, gensim/models/keyedvectors.py:864-878
Vector Computation Flow
This sequence diagram shows the process of retrieving similar words for a given query.
Persistence and Formats
KeyedVectors supports multiple serialization formats for interoperability with different tools.
Supported Formats
- Gensim Native: Saved via
.save()and loaded via.load(). - Word2Vec C Format: Supports both text and binary formats from the original Google tool.
- FastText: Can load vectors exported by Facebook's FastText.
File Loading Logic
The loading process often involves deducing the vocabulary size and vector dimensionality from a header before pre-allocating the numpy matrices.
# Loading from Word2Vec C format
header = utils.to_unicode(fin.readline(), encoding=encoding)
vocab_size, vector_size = [int(x) for x in header.split()]
kv = cls(vector_size, vocab_size, dtype=datatype)
# ... read binary or text weights ...
Sources: gensim/models/keyedvectors.py:1085-1110, gensim/models/keyedvectors.py:1135-1150
Advanced Usage and Customization
Per-Vector Attributes (Expandos)
The expandos system replaces the legacy Vocab objects. Attributes like frequency counts or custom metadata are stored in parallel numpy arrays, indexed identically to the vectors matrix.
# Setting and getting attributes
word_vectors.set_vecattr('apple', 'count', 500)
count = word_vectors.get_vecattr('apple', 'count')
Sources: gensim/models/keyedvectors.py:245-285
Memory Efficiency (mmap)
By setting mmap='r' during load(), the vectors array is not fully loaded into RAM but mapped to the disk file. This allows multiple Python processes to share the same physical memory for the embeddings.
Sources: gensim/models/word2vec.py:81-90, gensim/models/keyedvectors.py:44-48
Summary
KeyedVectors serves as the lightweight, production-ready interface for word embeddings in Gensim. By decoupling vector management from model training, it provides a highly efficient structure for similarity retrieval, analogy solving, and semantic analysis across large corpora.
Sources: gensim/models/keyedvectors.py:11-30, gensim/models/word2vec.py:65-75