Vector & Semantic Search
Relevant source files
The following files were used as context for generating this wiki page:
- src/mastra/lib/query-embedding.ts
- supabase/migrations/20260509205216_pgvector_semantic_search.sql
- supabase/migrations/20260510000000_vdb01_hybrid_fts_search.sql
- src/mastra/lib/tests/query-embedding.test.ts
- src/mastra/lib/intelligence-restaurant-search.ts
- src/mastra/lib/tests/search-003-ranking.integration.test.ts
The Vector & Semantic Search system provides advanced discovery capabilities for the mdeai platform, enabling natural language queries for apartments, events, and restaurants in Medellín. By leveraging pgvector and Gemini-based embeddings, the system moves beyond simple keyword matching to understand the semantic intent of user requests, such as finding a "quiet rooftop dinner" or "remote-work friendly rentals."
Sources: README.md:3-5, supabase/migrations/20260509205216_pgvector_semantic_search.sql:1-7
Architecture and Data Flow
The architecture integrates a TypeScript-based embedding client with a Supabase PostgreSQL backend equipped with the vector extension. The workflow involves converting user text into a 768-dimensional vector, which is then used to perform similarity searches against pre-indexed content in the database.
Search Data Flow
The following diagram illustrates how a user query is processed through the semantic search pipeline.
The system utilizes gemini-embedding-001 to generate consistent vector representations for both queries and stored content.
Sources: src/mastra/lib/query-embedding.ts:6-10, supabase/migrations/20260509205216_pgvector_semantic_search.sql:126-130
Embedding Management
Vector representations are stored in dedicated embedding tables that reference primary entity tables (apartments, events, restaurants).
Core Tables
| Table | Entity Reference | Dimensions | Model |
|---|---|---|---|
listing_embeddings | apartments(id) | 768 | gemini-embedding-001 |
event_embeddings | events(id) | 768 | gemini-embedding-001 |
restaurant_embeddings | restaurants(id) | 768 | gemini-embedding-001 |
Sources: supabase/migrations/20260509205216_pgvector_semantic_search.sql:15-41
HNSW Indexing
To ensure high-performance similarity searches, the system employs Hierarchical Navigable Small World (HNSW) indexes using cosine operators.
create index if not exists restaurant_embeddings_hnsw
on public.restaurant_embeddings
using hnsw (embedding vector_cosine_ops)
with (m = 16, ef_construction = 64);
Sources: supabase/migrations/20260509205216_pgvector_semantic_search.sql:57-60
Semantic Search Implementation
The platform uses PostgreSQL Remote Procedure Calls (RPCs) to execute vector similarity searches. These functions calculate similarity as (1 - cosine_distance).
Search RPCs and Parameters
| Function Name | Description | Key Result Fields |
|---|---|---|
semantic_search_listings | Searches active apartment listings | price_monthly, neighborhood, similarity |
semantic_search_events | Searches upcoming active events | event_start_time, tags, similarity |
semantic_search_restaurants | Searches active restaurants | cuisine_types, price_level, similarity |
All RPCs accept a query_embedding (vector), a similarity_threshold (default 0.6), and a match_count (default 20).
Sources: supabase/migrations/20260509205216_pgvector_semantic_search.sql:133-241
Hybrid Search Integration
In addition to pure semantic search, the system supports a hybrid model that combines Full-Text Search (FTS) with vector similarity. This is particularly relevant for "Intelligent Search" modules that use specific "venue signals" or "human_qa" feedback to boost certain results.
Hybrid Components
- FTS Helper:
public.fts_spanish()provides deterministic text indexing for Spanish content. - Venue Signals: Special signals like
wantsRooftop,wantsQuiet, orwantsDateNightare extracted from queries and used to refine ranking. - Rank Scoring: Results are ordered by a
rankScorethat combines semantic similarity with signal-based boosts.
Sources: supabase/migrations/20260510000000_vdb01_hybrid_fts_search.sql:1-12, src/mastra/lib/tests/search-003-ranking.integration.test.ts:46-70
Intelligence Search Schema
The following ER diagram shows the relationship between entity data and their vector/hybrid search components.
Client-Side Embedding Generation
The embedQueryText function manages the interaction with the Google Generative AI API to transform user input into vectors. It includes error handling for network failures and missing API keys.
export async function embedQueryText(text: string): Promise<number[] | null> {
const apiKey = process.env.GOOGLE_GENERATIVE_AI_API_KEY;
if (!apiKey || !text.trim()) return null;
// Fetch logic calling https://generativelanguage.googleapis.com/...
}
Sources: src/mastra/lib/query-embedding.ts:13-35, src/mastra/lib/tests/query-embedding.test.ts:10-25
Summary
The Vector & Semantic Search system enables the mdeai platform to deliver relevant search results by understanding user intent through multidimensional embeddings. By combining pgvector HNSW indexes with specialized RPCs and hybrid ranking signals, the architecture ensures that discovery is both technically fast and contextually accurate for the Medellín market.