Surgical AI Notes (Margin Notes)

Relevant source files

The following files were used as context for generating this wiki page:

• hooks/useNotes.ts
• components/NoteCard.tsx
• components/TranscriptColumn.tsx
• services/geminiService.ts
• types.ts
• services/storageService.ts
• App.tsx
• components/SmartBoard.tsx

The "Surgical AI Notes" feature, also referred to as "Margin Notes," is a core component of the Nota application. It allows users to transform segments of a YouTube video's transcript into concise, context-aware summaries. By highlighting text directly in the transcript, a user can trigger an AI-powered process that cleans the raw text and saves it as a persistent note associated with the video's timestamp.

This system is designed to turn a linear video-watching experience into an interactive workspace, enabling users to quickly capture and organize key insights without manual note-taking. The feature integrates user interface components for text selection, a state management hook for business logic, a service layer for AI processing via the Google Gemini API, and local storage for data persistence.

Sources: README.md:16-21, App.tsx:32-46

Architecture Overview

The Margin Notes feature is composed of several interconnected modules that handle user interaction, state management, AI processing, data storage, and UI rendering. The process begins when a user selects text in the TranscriptColumn and culminates in a new NoteCard being displayed in the SmartBoard.

The following sequence diagram illustrates the end-to-end flow from user action to note creation.

This diagram shows the complete data flow, from the initial user interaction in the TranscriptColumn to the final rendering of the AI-generated note in the SmartBoard.

Sources: components/TranscriptColumn.tsx:90-118, App.tsx:39-46, hooks/useNotes.ts:27-52, components/SmartBoard.tsx:260-278

Key Components

The feature is implemented through a collaboration of several key React components and hooks.

Component / Hook	Description	Source File
TranscriptColumn.tsx	Displays the video transcript and captures user text selections via mouseup and keyup event listeners.	components/TranscriptColumn.tsx
useNotes.ts	A custom hook that manages the state of notes, orchestrates the AI cleaning process, and handles CRUD operations via storageService.	hooks/useNotes.ts
geminiService.ts	Communicates with the Google Gemini API to perform the AI-powered text cleaning and summarization.	services/geminiService.ts
storageService.ts	Provides an abstraction for persisting and retrieving notes to and from the browser's localStorage.	services/storageService.ts
NoteCard.tsx	A UI component responsible for rendering a single note, including its cleaned text, timestamp, and action buttons (delete, jump-to-time).	components/NoteCard.tsx
SmartBoard.tsx	The main container that displays the list of generated NoteCard components under the "Notes" tab.	components/SmartBoard.tsx
App.tsx	The root component that integrates all parts, passing state and callbacks between the TranscriptColumn, useNotes hook, and SmartBoard.	App.tsx

Data Flow and Logic

1. Text Selection and Data Capture

The process starts in the TranscriptColumn component. An event listener attached to the document for mouseup and keyup events detects when the user makes a text selection.

• If a selection is made within the transcript container and is not collapsed, the handler extracts the selected text, its bounding rectangle (rect), and the startTime of the transcript segment.
• The startTime is retrieved from a data-timestamp attribute on the parent element of the selected text.
• This information is packaged into a SelectionData object and passed up to the App component via the onSelection callback.

Sources: components/TranscriptColumn.tsx:90-118, types.ts:15-19

2. Note Creation and AI Processing

The useNotes hook contains the core logic for creating a note. The addNote function is the entry point for this process.

The flow within addNote is as follows:

1. API Key Check: It first verifies if a Gemini API key is available using getApiKey(). If not, it returns immediately to prevent errors.
2. Status Update: It sets processingStatus to PROCESSING. This state is used by the UI (SmartBoard) to show a loading indicator.
3. AI Cleaning: It calls cleanTranscriptText from geminiService, passing the raw selected text.
4. Note Object Creation: Upon receiving the cleanedText from the service, it constructs a new Note object containing a unique ID, the video ID, original and cleaned text, timestamp, and creation date.
5. Persistence: The new note is saved using storageService.saveNote().
6. State Update: The local state is updated with the new list of notes returned from the storage service, triggering a UI re-render.
7. Final Status: The processingStatus is set back to IDLE after a short delay.

This logic is visualized in the flowchart below.

3. Gemini API Service (geminiService)

The cleanTranscriptText function in geminiService.ts is responsible for interacting with the Google Gemini API.

• It takes the raw transcript text as input.
• It uses a predefined SYSTEM_INSTRUCTION to prompt the AI model (gemini-2.0-flash) to act as an expert editor. The goal is to transform the raw text (which may contain filler words or stutters) into a concise, grammatically correct summary.
• The function includes error handling to prevent the application from crashing if the API call fails, returning the original raw text instead.

// services/geminiService.ts:4-10
const SYSTEM_INSTRUCTION = `
You are an expert editor for video transcripts. 
Your task is to take a raw snippet of text from a YouTube transcript (which may contain filler words, stuttering, or lack punctuation) and transform it into a concise, grammatically correct summary sentence or bullet point.
Retain the original meaning and context. 
Do not add any introductory or concluding remarks. Just output the cleaned text.
`;

Sources: services/geminiService.ts:4-50

Data Model and Persistence

Note Data Structure

All notes are structured according to the Note interface defined in types.ts.

This diagram outlines the fields of the Note entity.

Sources: types.ts:7-14

Storage (storageService)

The storageService.ts module handles all interactions with the browser's localStorage. It simulates what would be chrome.storage.local in a real extension.

• getNotes(videoId): Retrieves all notes for a specific videoId.
• saveNote(note): Adds a new note to the list for the corresponding videoId and saves the entire data structure back to localStorage.
• deleteNote(videoId, noteId): Removes a note by its ID and updates localStorage.

All notes are stored under a single key, tldw_notes, as a JSON object where keys are videoIds and values are arrays of Note objects.

Sources: services/storageService.ts:6-54

UI Rendering

SmartBoard and NoteCard

Once a note is created and the state is updated in App.tsx, the new list of notes is passed down to the SmartBoard component.

• The SmartBoard component contains a tabbed interface. When the "Notes" tab is active, it maps over the notes array.
• For each note object, it renders a NoteCard component, passing the note data and callbacks for deletion (onDelete) and seeking (onJumpToTime).
• The notes are displayed in reverse chronological order.
• While the useNotes hook has a processingStatus of PROCESSING, the SmartBoard displays a loading state with the text "Gemini is summarizing...".

The NoteCard component is responsible for displaying the final output:

• The AI-generated cleanedText.
• A formatted timestamp button that, when clicked, seeks the video to note.timestamp.
• Action buttons for copying the note text and deleting the note, which appear on hover.

Sources: components/SmartBoard.tsx:103-109, 260-278, components/NoteCard.tsx