AI Note Generation

Relevant source files

The following files were used as context for generating this wiki page:

• ai_backend/agents/note_agents.py
• ai_workers/notes_worker.py
• ai_backend/services/notes_service.py
• ai_backend/api/public_api.py
• ai_backend/agents/study_assistant.py
• PROJECT_STRUCTURE.md

The AI Note Generation system in ClassMate is a sophisticated multi-agent pipeline designed to transform raw lecture or meeting transcripts into structured, actionable study materials. The system utilizes a specialized orchestrator to manage multiple focused AI agents that handle specific tasks such as summarization, extraction of key points, and identification of action items.

This module resides within the ai_backend and is executed asynchronously via ai_workers to ensure the platform remains responsive during heavy computation. It integrates closely with the transcription pipeline and the vector memory store to provide contextually aware results.
Sources: PROJECT_STRUCTURE.md:73-85, ai_backend/services/notes_service.py:12-14

System Architecture

The architecture follows a distributed pattern where the FastAPI backend receives requests and offloads the heavy lifting to Celery workers. The core logic is encapsulated in a "Multi-Agent Orchestrator" that sequences specialized agents.

Component Overview

Component	Responsibility
NotesService	Orchestrates the request flow within the FastAPI backend, interacting with the database to fetch transcripts and trigger processing.
NotesWorker	A Celery-based background worker that executes the orchestrator in an asynchronous environment.
NotesOrchestrator	Manages the execution order and data flow between individual AI agents.
Specialized Agents	Individual classes (Summary, KeyPoints, ActionItems, etc.) that perform discrete NLP tasks.

Sources: ai_backend/services/notes_service.py:16-18, ai_workers/notes_worker.py:27-31, ai_backend/agents/note_agents.py:10-15

Data Flow Diagram

The following diagram illustrates how a transcript is processed through the system:

Agent Pipeline Logic

The system employs several specialized agents that inherit from a BaseAgent class. These agents process the transcript using deterministic heuristics and keyword matching (in the current implementation) or LLM-based logic.

Specialized Agent Responsibilities

1. SummaryAgent: Splits the transcript into sentences and extracts the first few sentences to create a concise overview.
2. KeyPointsAgent: Filters sentences for specific keywords such as "important," "critical," or "remember" to identify primary concepts.
3. ActionItemsAgent: Scans for action-oriented verbs (e.g., "should," "must," "complete") to identify tasks.
4. EvaluatorAgent: Performs a quality check on the generated outputs, flagging issues like empty summaries or excessive list lengths.
5. RefinerAgent: Corrects issues identified by the Evaluator, such as trimming lists or injecting entity context into the summary.

Sources: ai_backend/agents/note_agents.py:10-117

Refinement Sequence

Integration with Study Assistant

The StudyAssistantAgent extends the note generation capabilities by providing semantic search and high-level educational content generation. It utilizes LangChain and OpenAI to provide features beyond simple extraction.

AI Study Features

• Flashcard Generation: Extracts key concepts and formats them into question/answer pairs.
• Quiz Creation: Generates multiple-choice questions with explanations based on transcript content.
• Semantic Search: Uses VectorMemoryStore to index transcripts and summaries, allowing users to ask questions about lecture content.
• Concept Explanation: Explains complex terms in various styles (e.g., "ELI5" or "Visual").

Sources: ai_backend/agents/study_assistant.py:71-180, ai_backend/api/ai_agent_api.py:46-95

Database Integration and Persistence

Once notes are generated, they are persisted back to the sessions table in the database.

# From ai_workers/notes_worker.py
async def save_notes_to_db(self, notes_data: Dict[str, Any]):
    with SessionLocal() as db:
        db.execute("""
            UPDATE sessions 
            SET summary = ?,
                key_points = ?,
                action_items = ?,
                status = 'completed',
                updated_at = ?
            WHERE id = ?
        """, (
            notes_data.get("summary"),
            json.dumps(notes_data.get("key_points", [])),
            json.dumps(notes_data.get("action_items", [])),
            datetime.utcnow(),
            notes_data["session_id"]
        ))
        db.commit()

Sources: ai_workers/notes_worker.py:64-83

Note Data Schema

Field	Type	Description
summary	Text	High-level overview of the session.
key_points	JSON (List)	List of important concepts extracted from the text.
action_items	JSON (List)	List of tasks or next steps identified.
status	String	Updated to 'completed' after successful generation.

Sources: ai_backend/services/notes_service.py:45-55, ai_workers/notes_worker.py:71-77

Conclusion

The AI Note Generation system provides an automated way to digest long transcripts into structured formats. By utilizing a multi-agent approach, the system ensures that the output is not only extracted but also evaluated and refined before being presented to the user. Integration with vector memory further allows these generated notes to serve as a searchable knowledge base for the Study Assistant.