YouTube & Media Integration
Relevant source files
The following files were used as context for generating this wiki page:
YouTube and media integration in ThinkEx provides a first-class experience for incorporating video content into a visual thinking environment. The system allows users to embed individual videos or playlists, search for relevant content directly through an AI assistant, and persist playback progress across sessions. Media items are treated as spatial cards on the workspace canvas, enabling side-by-side comparison with other notes and documents.
The integration leverages the YouTube Data API v3 for metadata retrieval and search, while utilizing the YouTube IFrame Player API for interactive playback and state management. This ensures that video content is not just an external link but an integrated part of the user's reasoning and knowledge capture process.
Sources: README.md:31-35, src/lib/youtube.ts
Core Architecture & Data Flow
The YouTube integration is divided into three primary layers: the external API communication layer, the AI-driven discovery layer, and the interactive UI layer.
Media Discovery and Retrieval
Users can add YouTube content through three primary flows: manual URL entry via a dialog, AI-powered tool calls during chat, and automated workspace generation.
The system uses searchVideos to query the YouTube Data API, fetching snippets, content details (duration), and statistics (view counts).
Sources: src/lib/youtube.ts:63-138, src/components/modals/CreateYouTubeDialog.tsx:43-70, src/app/api/workspaces/autogen/route.ts:468-485
Playback State Management
ThinkEx implements persistent playback tracking. The YouTubePanelContent component monitors the player state and saves the current timestamp and playback rate to local storage at regular intervals or when the video is paused/ended.
| Key | Description | Frequency |
|---|---|---|
seconds | The current playback timestamp in seconds | Every 10s (while playing) |
playbackRate | The user's selected speed (e.g., 1.5x) | On Change / Interval |
progressKey | Unique ID based on video/playlist ID | N/A |
Sources: src/components/workspace-canvas/YouTubePanelContent.tsx:21-75
Component Implementation
YouTube Player Integration
The YouTubePanelContent uses the useYouTubePlayer hook to initialize an iframe. It supports both standalone videos and playlists by extracting IDs from provided URLs.
AI Assistant Tools
The assistant can interact with YouTube content via two primary tools:
- searchYoutube: Uses a query to return a list of video results, which are then rendered in the
YouTubeSearchToolUIwith thumbnails and view counts. - addYoutubeVideo: Adds a specific video ID and title to the current workspace as a new item.
Sources: src/lib/ai/tools/youtube-tools.ts:13-61, src/components/assistant-ui/YouTubeSearchToolUI.tsx:165-224
Data Models and Formatting
YouTube Data Schema
The integration relies on a structured YouTubeData object within the workspace item state.
| Field | Type | Description |
|---|---|---|
url | string | The full YouTube video or playlist URL |
thumbnail | string (optional) | Cached thumbnail URL for UI previews |
title | string | The title extracted via API or provided by user |
Sources: src/lib/workspace-state/types.ts, src/lib/youtube.ts:49-61
Workspace Context Formatting
When the AI assistant analyzes the workspace, the formatWorkspaceContext utility ensures the video metadata is available for reasoning. While the full transcript isn't always embedded, the metadata includes the URL, allowing the agent to reference the video during tasks like "add a video on this topic."
Sources: src/lib/utils/format-workspace-context.ts:162-171, src/lib/utils/format-workspace-context.ts:390-398
Automated Generation (Autogen)
During workspace creation via the /api/workspaces/autogen route, the system performs "Distillation." This process generates a youtubeSearchTerm based on the user's initial prompt or uploaded documents. The route then automatically searches for and adds the most relevant video to the new workspace's layout.
Sources: src/app/api/workspaces/autogen/route.ts:168-208, src/app/api/workspaces/autogen/route.ts:468-485
Summary
The YouTube & Media Integration in ThinkEx facilitates a seamless transition from content discovery to active study. By combining API-driven search, AI-assisted tool use, and robust state persistence, the system ensures that media remains a contextualized and functional component of the user's digital workspace.