Transcript Fetching and Synchronization

The transcript fetching and synchronization system is a core feature of the Nota application. Its primary purpose is to retrieve a YouTube video's transcript, display it to the user, and keep the displayed text synchronized with the video's playback time. This creates an interactive experience where users can read along, click on text to seek the video, and select text to generate AI-powered notes.

The system is composed of a Python backend service that fetches data from YouTube and a React frontend that manages state, renders the UI, and handles user interactions. The main components involved are the api/index.py backend, the YoutubeService frontend service, the useTranscript custom hook for state management, and the TranscriptColumn and VideoPlayer components for the user interface.

Sources: README.md:13-15, App.tsx:1-10

Architecture Overview

The architecture is split between a frontend React application and a backend Python API. The frontend initiates a request for a transcript, the backend retrieves it from YouTube, and the data flows back to the UI components for rendering and synchronization.

This diagram shows the high-level interaction between the frontend and backend components for fetching and displaying the transcript.

Backend Transcript Service

The backend is a FastAPI application responsible for securely and reliably fetching transcript data from YouTube. It exposes a single endpoint for this purpose.

Sources: api/index.py

API Endpoint: GET /api/transcript

This endpoint accepts a YouTube video ID and returns the transcript data as a JSON array.

Sources: api/index.py:71

Data Flow and Logic

The backend follows a clear sequence to process requests:

1. Security Middleware: All requests first pass through security headers, rate limiting, and CORS middleware.
2. Video ID Validation: FastAPI validates that the videoId parameter is exactly 11 characters long.
3. API Fetch: The get_transcript function uses the youtube-transcript-api library to fetch the transcript data. It includes support for using a Webshare proxy if configured via environment variables.
4. Response: On success, it returns the raw transcript data.
5. Error Handling: It specifically handles common errors from the YouTube API like TranscriptsDisabled, NoTranscriptFound, and VideoUnavailable, returning a 404 status. It also has a catch-all for other exceptions, returning a 500 status.

The following flowchart illustrates the backend logic for a single request to /api/transcript.

Frontend Data Fetching

The frontend is responsible for triggering the transcript fetch, managing the application state (loading, error, data), and providing the data to the UI components.

YoutubeService

This service class acts as the API client for the frontend.

• fetchTranscript(videoId): This static method is the primary function for fetching data. It first validates the video ID against a regex, then makes a fetch call to the backend /api/transcript endpoint. It handles JSON parsing and maps the raw backend response to the TranscriptSegment type used throughout the frontend. It also includes robust error handling for non-OK responses or incorrect content types.
• extractVideoId(url): A utility function to parse a YouTube video ID from various URL formats.

Sources: services/youtubeService.ts:13-50, tests/youtubeService.test.ts:11-31

useTranscript Hook

This custom React hook encapsulates all logic and state related to the transcript.

• State Management: It manages three key pieces of state:
  • transcript: TranscriptSegment[]: The array of transcript data.
  • isLoading: boolean: A flag for when a fetch is in progress.
  • error: string | null: An error message if the fetch fails.
• loadTranscript(videoId): This function orchestrates the fetching process. It sets isLoading to true, calls YoutubeService.fetchTranscript, and updates the transcript state on success or the error state on failure.
• Demo Fallback: If the fetch fails for the specific DEMO_VIDEO_ID, it has a fallback mechanism to load hardcoded demo data, ensuring a smooth demo experience even with network or CORS issues.
• resetTranscript(): A utility to clear all state, used when loading a new video.

Sources: hooks/useTranscript.ts:18-55

Data Fetching Sequence

The following diagram shows the sequence of events from a user loading a video to the transcript being displayed.

Transcript Display and Synchronization

Once the transcript data is fetched, it is passed to the TranscriptColumn component, which handles rendering and synchronization with the VideoPlayer.

TranscriptColumn Component

This component is responsible for displaying the list of transcript segments.

• Rendering: It maps over the segments array and renders a TranscriptSegmentItem for each one. While loading, it displays a TranscriptSkeleton component for a better user experience.
• Active Segment Highlighting: It determines the active segment by comparing the currentTime prop (passed down from App.tsx) with each segment's startTime and duration. The active segment is given a distinct style and automatically scrolled into view.
• Seeking: Clicking on any TranscriptSegmentItem triggers the onSeek callback, which propagates up to App.tsx to seek the video player to that segment's startTime.
• Text Selection: The component contains logic to listen for mouseup and keyup events to detect when a user has selected text within the transcript. When a valid selection is made, it calls the onSelection callback with the selected text, its start time, and its screen position, enabling the "margin notes" feature.
• Search: A search input allows users to filter the transcript segments based on a query.

Sources: components/TranscriptColumn.tsx, components/TranscriptSkeleton.tsx:3-16

Data Structures

The core data structure for a piece of the transcript is the TranscriptSegment.

// From: types.ts
export interface TranscriptSegment {
  id: string;
  startTime: number; // in seconds
  text: string;
  duration?: number; // Duration in seconds
}

Sources: types.ts:1-6

Component Interaction and Synchronization Flow

The App.tsx component acts as the central hub, managing state and connecting the VideoPlayer and TranscriptColumn.

1. The VideoPlayer component, via the react-youtube library, provides a periodic onTimeUpdate event.
2. App.tsx's handleTimeUpdate callback captures this time and updates the currentTime state.
3. This currentTime state is passed as a prop to TranscriptColumn.
4. TranscriptColumn uses this prop to identify and highlight the currently spoken segment.
5. Conversely, when a user clicks a segment in TranscriptColumn, the onSeek callback is fired.
6. App.tsx's handleSeek function receives the timestamp and uses a ref to the video player instance to call seekTo(), synchronizing the video to the transcript.

This diagram illustrates the state and event flow for synchronization.

Conclusion

The transcript fetching and synchronization system is a well-defined, multi-layered feature that spans both the backend and frontend. It begins with a secure and robust Python API that retrieves data from YouTube. This data is then consumed by a structured frontend system using a dedicated service and a custom hook to manage state. Finally, the TranscriptColumn and VideoPlayer components work in concert, orchestrated by the main App component, to provide a seamlessly synchronized and interactive user experience. This tight integration is fundamental to the application's goal of turning linear video content into an actionable workspace.