Caching Service
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Caching Service in the qa-dashboard backend provides a semantic cache to store and retrieve question-and-answer pairs. Its primary purpose is to reduce redundant calls to the AI service for generating answers and to detect duplicate questions before they are submitted. It achieves this by converting questions into vector embeddings and using a vector store (FAISS) to find semantically similar entries, rather than relying on exact text matches.
This service is implemented as a singleton class, SemanticCache, and is tightly integrated with the AI Service to optimize performance and improve user experience. It is disabled if the necessary GOOGLE_API_KEY is not configured.
Sources: backend/services/cache_service.py, backend/services/ai_service.py
Architecture and Core Components
The SemanticCache is designed as a singleton, ensuring a single instance manages the vector store throughout the application's lifecycle.
Sources: backend/services/cache_service.py:12-16
Key Components
| Component | Implementation | Purpose |
|---|---|---|
| Vector Embeddings | GoogleGenerativeAIEmbeddings | Converts text (questions) into numerical vectors for similarity comparison. Uses the gemini-embedding-001 model. |
| Vector Store | FAISS (Facebook AI Similarity Search) | Stores the text vectors and allows for efficient similarity searches. The index is persisted on disk. |
| Singleton Pattern | __new__ method override | Ensures only one instance of SemanticCache is created. |
| Persistent Index | faiss_index directory | The FAISS index is saved to and loaded from this local directory, preserving the cache across restarts. |
Sources: backend/services/cache_service.py:9-10, backend/services/cache_service.py:25-29, backend/services/cache_service.py:12-16, backend/services/cache_service.py:32-38
Class Diagram
The diagram below shows the structure of the SemanticCache class.
Initialization Flow
The cache initializes when the application starts. It requires a GOOGLE_API_KEY to function; otherwise, it remains disabled. The initialization process involves loading a pre-existing FAISS index from the disk or creating a new one if it doesn't exist.
Caching and Retrieval Logic
The service exposes methods to add new Q&A pairs to the cache and to retrieve answers for semantically similar questions.
Caching an Answer
The cache_answer method is called to store a new question-answer pair. It adds the question's text and its corresponding answer as metadata to the FAISS vector store and then saves the updated index to the disk. This happens, for example, when a new answer is posted to a question.
Retrieving a Cached Answer
The get_cached_answer method is used to check for existing answers before generating a new one. It performs a similarity search and returns a cached answer if the similarity score is below a threshold of 0.1, indicating a very close match.
Integration with AI Service
The SemanticCache is a dependency of the AIService, which uses it to optimize two key features: generating suggested answers and checking for duplicate questions.
Sources: backend/services/ai_service.py:5
Answer Suggestion Flow
When a request is made to generate a suggested answer, the AIService first queries the SemanticCache.
- If a similar question with an answer is found in the cache (a "cache hit"), that answer is returned immediately, avoiding a costly LLM API call.
- If no suitable answer is found in the cache (a "cache miss"), the
AIServiceproceeds to generate a new answer using the Gemini model. - The newly generated answer is then stored in the cache using
cache_answerfor future requests.
This flow is illustrated in the diagram below.
Sources: backend/services/ai_service.py:38-57
Duplicate Question Check
The AIService.check_duplicate method leverages the cache's vector store to find semantically similar questions. This functionality is exposed via the POST /questions/check-duplicate endpoint. The frontend QuestionForm component calls this endpoint with a debounce mechanism as the user types, providing real-time feedback if a similar question already exists.
The response includes the similar question's text and its answer if one has been provided.
Sources: backend/services/ai_service.py:85-126, web/src/components/QuestionForm.js:28-56, backend/api/questions.py:202-204
Frontend Duplicate Check Flow
API Endpoints
The caching service is indirectly used by several API endpoints.
| Endpoint | HTTP Method | Description | Cache Interaction |
|---|---|---|---|
/questions/check-duplicate | POST | Checks if a given text is similar to an existing question. | Read: Performs a similarity search on the vector store. |
/questions/{id}/suggest-answer | GET | Generates an AI-powered answer suggestion for a question. | Read/Write: First checks for a cached answer. If none, generates one and then caches the new Q&A pair. |
/answers/{question_id} | POST | Creates a new answer for a question. | Write: Caches the new answer against the original question. |
Sources: backend/api/questions.py:181-204, backend/api/answers.py:10-38, web/src/config/api.js:14-16
Conclusion
The Semantic Caching Service is a critical component for optimizing the qa-dashboard's performance and user experience. By leveraging vector embeddings and a persistent FAISS index, it effectively reduces latency and operational costs associated with repeated LLM calls. Furthermore, its integration into the question submission form provides a proactive mechanism to prevent content duplication, guiding users to existing answers and maintaining a cleaner Q&A forum.
Sources: backend/services/cache_service.py, backend/services/ai_service.py