Poincare Embeddings
Relevant source files
The following files were used as context for generating this wiki page:
Poincare Embeddings are a specialized class of vector representations designed to capture latent hierarchical information more effectively than traditional Euclidean embeddings. In the Gensim project, this module implements the method described by Maximilian Nickel and Douwe Kiela, which utilizes the hyperbolic geometry of a Poincaré ball to represent directed acyclic graphs (DAGs) and tree-like structures. By using hyperbolic distance, the model can represent complex hierarchies in relatively few dimensions because the "volume" of the space grows exponentially from the center, mimicking the branching factor of trees.
Sources: gensim/models/poincare.py:10-23
Architecture and Core Components
The implementation is centered around the PoincareModel class, which manages the training process, and the PoincareKeyedVectors class, which handles the resulting embeddings and similarity queries. The system uses a coordinate-descent approach optimized for hyperbolic space, specifically employing Riemannian stochastic gradient descent.
Class Hierarchy and Relationships
The following diagram illustrates the relationship between the core classes involved in Poincare Embeddings.
The PoincareModel acts as the primary interface for training, while PoincareKeyedVectors stores the learned weights and provides utility methods for navigating the hierarchy.
Sources: gensim/models/poincare.py:61-90, gensim/models/poincare.py:642-658
Training Logic and Data Flow
Training involves iterative updates to node vectors based on positive relations (links in the graph) and negative samples. The model uses a "burn-in" period with a lower learning rate to stabilize the initial vectors before full training begins.
Gradient Descent and Updates
The training process is batched. For each batch, the model calculates hyperbolic distances and gradients using the PoincareBatch class. Vectors are updated and then clipped to ensure they remain within the unit ball (norm < 1).
The clipping mechanism is critical for numerical stability in hyperbolic space, as distances approach infinity as the norm approaches 1.
Sources: gensim/models/poincare.py:461-483, gensim/models/poincare.py:539-565, gensim/models/poincare.py:279-307
Configuration Options
The PoincareModel constructor accepts several parameters to tune the embedding process:
| Parameter | Type | Default | Description |
|---|---|---|---|
size | int | 50 | Number of dimensions of the trained model. |
alpha | float | 0.1 | Learning rate for training. |
negative | int | 10 | Number of negative samples to use per positive relation. |
epsilon | float | 1e-5 | Constant used for clipping embeddings below a norm of one. |
regularization_coeff | float | 1.0 | Coefficient for l2-regularization. |
burn_in | int | 10 | Number of epochs for initial burn-in phase. |
Sources: gensim/models/poincare.py:91-118
Hierarchy Navigation and Similarity
Resulting vectors allow for operations that determine the relative position of nodes in a hierarchy. This is managed by PoincareKeyedVectors.
Key Navigation Methods
- Closest Child/Parent: Uses hyperbolic distance and vector norms to find the most immediate hierarchical relative. A lower norm generally indicates a position higher in the tree.
- Ancestors/Descendants: Recursively crawls the closest parent/child relationships to reconstruct paths in the hierarchy.
- Difference in Hierarchy: Calculated as
norm(vector_2) - norm(vector_1). A positive value indicates the first node is higher in the hierarchy than the second.
Sources: gensim/models/poincare.py:702-748, gensim/models/poincare.py:898-933
Hyperbolic Distance Calculation
The distance between two vectors $u$ and $v$ in the Poincare ball is defined as:
$d(u, v) = \text{arccosh}\left(1 + 2 \frac{|u - v|^2}{(1 - |u|^2)(1 - |v|^2)}\right)$
# Simplified representation of the distance calculation logic
euclidean_dists = np.linalg.norm(vector_1 - vectors_all, axis=1)
norm = np.linalg.norm(vector_1)
all_norms = np.linalg.norm(vectors_all, axis=1)
dist = np.arccosh(1 + 2 * ((euclidean_dists ** 2) / ((1 - norm ** 2) * (1 - all_norms ** 2))))
Sources: gensim/models/poincare.py:682-698
Evaluation Metrics
Gensim provides three primary classes for evaluating the quality of learned Poincare Embeddings:
- ReconstructionEvaluation: Measures how well the model can reconstruct the original graph. It computes Mean Rank and Mean Average Precision (MAP) of positive relations.
- LinkPredictionEvaluation: Evaluates the model's ability to predict unseen relations by splitting data into training and test sets.
- LexicalEntailmentEvaluation: Uses the HyperLex dataset to evaluate how well the embeddings capture entailment (e.g., "is-a" relationships) using Spearman correlation.
Sources: gensim/models/poincare.py:996-1010, gensim/models/poincare.py:1082-1095, gensim/models/poincare.py:1174-1185
Data Ingestion
The PoincareRelations class is used to stream relations from file formats like TSV. It treats relations as ordered pairs $(u, v)$, where $u$ is typically a subtype of $v$.
# Example of streaming relations
for relation in PoincareRelations(file_path, delimiter='\t'):
# yield (node_u, node_v)
Sources: gensim/models/poincare.py:936-963
Summary
Poincare Embeddings in Gensim provide a robust framework for learning hierarchical structures from relational data. By leveraging hyperbolic geometry via the PoincareModel and specialized PoincareKeyedVectors, the system allows for efficient training, navigation, and evaluation of complex taxonomies and directed acyclic graphs. Its primary advantage over Euclidean models is the ability to represent deep trees in low-dimensional spaces while maintaining high reconstruction accuracy.
Sources: gensim/models/poincare.py:1-40