High-Level Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• PROJECT_STRUCTURE.md
• README.md
• infra/README.md
• infra/terraform/main.tf
• docker-compose.prod.yml
• web_app/src/main.js

ClassMate is an AI-powered lecture recording and note-taking system built on a hybrid scalable architecture. It is designed to handle high user loads (100k+) by offloading heavy computational tasks, such as audio transcription and AI note generation, to a containerized backend while maintaining real-time synchronization through a control plane.

The system is organized as a monorepo containing a Flutter mobile application, a Progressive Web App (PWA), a FastAPI-based AI backend, and distributed Celery workers for background processing.

Sources: README.md:1-12, PROJECT_STRUCTURE.md:3-8

System Components

The architecture is divided into three primary layers: the Client Layer, the Control/Application Layer, and the Data/Scaling Layer.

Architecture Overview Diagram

This diagram illustrates the flow of data from client interfaces through the gateway to the processing and storage components.

Component Descriptions

Component	Technology	Role
Mobile App	Flutter	Cross-platform audio capture and session management.
Web App	JavaScript (PWA)	Browser-based recording and real-time transcription display.
API Backend	FastAPI	RESTful API, WebSocket management, and task orchestration.
AI Workers	Celery	Background processing for Whisper transcription and AI notes.
Message Broker	Redis	Task queuing for workers and caching.
Database	PostgreSQL	Persistent storage for users, sessions, and transcripts.
Object Storage	S3 / MinIO	Storage for raw audio files.

Sources: PROJECT_STRUCTURE.md:142-166, infra/README.md:20-30

Recording and Transcription Pipeline

The core functionality of ClassMate follows a distributed processing pipeline. Audio is captured at the edge (mobile or web) and streamed or uploaded to the backend. The backend then delegates heavy lifting to the AI Workers.

Audio Processing Flow

1. Capture: Audio is captured via flutter_sound on mobile or MediaRecorder API on web.
2. Streaming/Upload: The Web App uses WebSockets to send audio chunks in real-time, while the Mobile App uses an upload_queue_service.
3. Queueing: The FastAPI backend receives audio and pushes a transcription task to Redis.
4. AI Processing: A Celery worker pulls the task, processes it using the Whisper model (e.g., medium or large-v3), and identifies speakers.
5. Persistence: Processed transcripts and AI-generated notes are stored in PostgreSQL.

Sources: PROJECT_STRUCTURE.md:175-185, web_app/src/main.js:100-130, docker-compose.prod.yml:95-120

Real-time Web App Interaction

The Web App utilizes WebSockets for live transcription feedback.

Infrastructure and Deployment

ClassMate uses Infrastructure as Code (IaC) to manage environments ranging from local development to production.

Cloud Infrastructure (AWS)

The production environment on AWS is managed via Terraform and includes:

• VPC: Isolated network with public and private subnets.
• RDS: Managed PostgreSQL 15.4 for data persistence.
• ElastiCache: Managed Redis for the Celery broker.
• S3: Bucket for encrypted audio storage at rest.
• EC2: API server running Dockerized services.

Sources: infra/terraform/main.tf:20-150, infra/README.md:46-60

Containerization

The system is fully containerized using Docker. The docker-compose.prod.yml defines the multi-service stack, including observability tools like Prometheus, Grafana, Loki, and Tempo for tracing and logging.

services:
  api:
    build: ai_backend/Dockerfile
    environment:
      DATABASE_URL: postgresql://classmate:password@postgres:5432/classmate
  worker:
    command: celery -A ai_backend.celery_client worker
  nginx:
    ports:
      - "80:80"
      - "443:443"

Sources: docker-compose.prod.yml:70-140, infra/README.md:15-35

Security and Identity

ClassMate implements a hybrid authentication model:

• Firebase Auth: Handles user identity and initial authentication.
• Identity Bridge: Synchronizes Firebase users to the internal PostgreSQL database for role-based access control.
• Data Security: Audio at rest is stored in S3 with AES256 server-side encryption. API traffic is served via HTTPS via Nginx or Traefik.

Sources: README.md:60-75, infra/terraform/main.tf:165-175

Conclusion

The high-level architecture of ClassMate leverages a modern, distributed stack to ensure that high-latency AI tasks do not block user interactions. By separating the Control Plane (API) from the Data/Scaling Layer (Workers/DB), the system achieves horizontal scalability, allowing additional AI Workers to be added as demand increases.