Videos Data Schema

Relevant source files

The following files were used as context for generating this wiki page:

• videos.js
• redesign/videos.js
• videos.html
• redesign/videos.html
• old/site-src/VideosSection.jsx
• PLAYLIST_EXTRACTION_GUIDE.md
• old/site-src/TODO.txt

The Videos Data Schema defines the structure for video information used across the Norm Macdonald Archive website. The data is sourced from a central JSON file, consolidated_youtube_data.json, which is consumed by various front-end components to display a searchable and paginated video gallery. This system is implemented in both a modern vanilla JavaScript version and an older React-based version, both of which rely on the same underlying data source.

Sources: videos.js, old/site-src/VideosSection.jsx, old/site-src/TODO.txt:10

Data Loading and Normalization Flow

The video data pipeline begins when the user visits a video page. A JavaScript module fetches the raw data from consolidated_youtube_data.json, normalizes it into a consistent client-side format, stores it in a state object, and then renders the video cards to the DOM.

This diagram illustrates the sequence of events from page load to video display.

Raw Data Schema (consolidated_youtube_data.json)

The canonical source of video data is a single JSON file named consolidated_youtube_data.json. This file contains an array of objects, where each object represents a single video. The structure of these raw objects is derived from a YouTube playlist extraction process.

Sources: videos.js:53, old/site-src/VideosSection.jsx:9, PLAYLIST_EXTRACTION_GUIDE.md:4

Raw Schema Diagram

This Entity-Relationship diagram shows the fields present in the raw video data objects.

Raw Field Descriptions

Field Name	Type	Description
Video url	string	The full YouTube watch URL for the video.
Title	string	The title of the video.
Description	string	The video description. Often contains just "Video url:".
Channel name	string	The name of the YouTube channel that uploaded the video. Can be "N/A".
Duration	string	The duration of the video. Can be "N/A".
Views	string	The number of views. Can be "N/A".
Thumbnail url	string	The URL for the video's thumbnail. Can be "N/A".

Sources: videos.js:58-66, old/site-src/VideosSection.jsx:18-20, PLAYLIST_EXTRACTION_GUIDE.md:23-29

Client-Side Data Schema

Upon being loaded, the raw video data is immediately processed and normalized into a more developer-friendly format. This normalized data is stored in the state.videos array within the client-side JavaScript.

Sources: videos.js:57-67, redesign/videos.js:57-67

Schema Normalization

The loadVideos function maps the raw data fields to a new structure. This process includes extracting the YouTube video ID from the URL and providing default values for missing data.

Raw Field (consolidated_youtube_data.json)	Normalized Field (state.videos object)	Transformation Logic
(n/a)	id	The array index of the video during mapping.
Video url	url	The YouTube video ID is extracted using a regex.
Title	title	Used directly, or defaults to 'Untitled Video'.
Description	description	Used directly, or defaults to an empty string.
Channel name	channel	Used directly, or defaults to 'N/A'.
Duration	duration	Used directly, or defaults to 'N/A'.
Views	views	Used directly, or defaults to 'N/A'.
Thumbnail url	thumbnail	Used directly, or defaults to an empty string.

Sources: videos.js:57-67, redesign/videos.js:57-67

The following code snippet from videos.js shows this normalization process:

// redesign/videos.js

// Process and normalize video data
state.videos = rawVideos.map((video, index) => ({
    id: index,
    url: extractVideoId(video['Video url'] || ''),
    title: video['Title'] || 'Untitled Video',
    description: video['Description'] || '',
    channel: video['Channel name'] || 'N/A',
    duration: video['Duration'] || 'N/A',
    views: video['Views'] || 'N/A',
    thumbnail: video['Thumbnail url'] || ''
})).filter(video => video.url); // Only keep videos with valid URLs

Sources: redesign/videos.js:57-67

Client-Side State Object

The videos.js script maintains a global state object to manage the application's data, including the video list, filtered results, and pagination.

This diagram shows the structure of the client-side state object.

Data Consumption

Vanilla JavaScript Implementation

In the main and redesigned site, videos.js uses the normalized data to dynamically create and render video cards. The createVideoCard function takes a normalized video object and constructs the HTML for a video card, which is then appended to the #videos-container.

Key consumption points:

• Thumbnail: If video.thumbnail is not available, a default thumbnail URL is constructed using the video.url (the video ID): https://img.youtube.com/vi/${video.url}/mqdefault.jpg.
• Metadata: video.title, video.channel, and video.views are displayed on the card.
• Click Action: Clicking a card opens the video on YouTube using the video.url.

Sources: videos.js:150-200, redesign/videos.js:150-200

React Implementation (VideosSection.jsx)

The older React-based component, VideosSection.jsx, fetches the same consolidated_youtube_data.json file. It performs filtering directly on the raw data structure without the explicit normalization step seen in videos.js.

• It filters videos where Title, Description, or "Channel name" match the search term.
• It uses a getYouTubeEmbedUrl utility to extract the video ID for embedding an iframe player directly on the page, a different approach from the vanilla JS version which links out to YouTube.

// old/site-src/VideosSection.jsx

const filteredVideos = videos.filter(video =>
    video.Title?.toLowerCase().includes(searchTerm.toLowerCase()) ||
    video.Description?.toLowerCase().includes(searchTerm.toLowerCase()) ||
    video["Channel name"]?.toLowerCase().includes(searchTerm.toLowerCase())
);

const getYouTubeEmbedUrl = (url) => {
    const videoIdMatch = url.match(/(?:youtu\.be\/|youtube\.com\/(?:watch\?.*v=|embed\/|v\/|shorts\/))([^&?]+)/);
    return videoIdMatch ? `https://www.youtube.com/embed/${videoIdMatch[1]}` : null;
};

Sources: old/site-src/VideosSection.jsx:17-24

Summary

The video data schema is designed for simplicity and robustness. A single, consolidated JSON file serves as the "single source of truth" for all video content. On the client-side, this data is normalized into a clean, consistent format that decouples the rendering logic from the raw data structure. This architecture has proven flexible enough to support both a modern vanilla JavaScript implementation and an older React-based component, ensuring data consistency across different generations of the application.

Sources: videos.js, old/site-src/VideosSection.jsx