Newsletter Program Overview
Relevant source files
The following files were used as context for generating this wiki page:- newsletter-program/README-SUBSCRIBER-MANAGEMENT.md
- newsletter-program/package.json
- newsletters/README.md
- themes/matrix-flow/source/js/newsletter.js
- newsletter-program/src/core/newsletter-composer.js
- newsletter-program/src/cli/newsletter-cli.js
- newsletter-program/src/core/analytics-manager.js
- themes/matrix-flow/README-newsletter.md
- newsletter-program/src/cli/analytics-cli.js
- newsletters/newsletter-template.md
The Newsletter Program is a comprehensive, repository-based system for composing, managing, and distributing newsletters. It is designed with a focus on security, automation, and developer experience, leveraging a file-based architecture that minimizes external dependencies. The program integrates with the main portfolio website for subscriber acquisition and uses GitHub Actions for automated processing and sending.
Sources: newsletters/README.md, newsletter-program/README-SUBSCRIBER-MANAGEMENT.md
Key features include encrypted subscriber storage (AES-256), a rich command-line interface (CLI) for all management tasks, detailed analytics and reporting, and automated newsletter generation from RSS feeds. The subscription process is uniquely handled by creating GitHub Issues, which are then processed by a GitHub Action, ensuring a transparent and auditable trail.
Sources: newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:8-12, themes/matrix-flow/README-newsletter.md:25-36
System Architecture
The newsletter system is composed of several interconnected components that work together to handle subscriptions, content management, and email distribution. The frontend form initiates the subscription, which is then handled by a backend program via GitHub Actions, with external services used for email delivery.
This diagram shows the high-level architecture of the newsletter system, from user subscription to backend processing and email delivery.
Sources: newsletters/README.md:43-53, themes/matrix-flow/README-newsletter.md:21-36, newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:14-26
Subscription Flow
The subscription process is designed for security and transparency, using GitHub Issues as an intermediary.
- Form Submission: A user fills out the newsletter subscription form on the website.
- Client-Side Validation: The
newsletter.jsscript performs real-time and on-blur validation for fields like email, frequency, and consent. - GitHub Issue Creation: Upon successful validation, the script generates a URL to create a new GitHub issue, pre-filled with the subscriber's details. The user is redirected to this URL to complete the submission. This is the default and most secure method.
- GitHub Action Trigger: A new issue with the
newsletter-subscriptionlabel triggers a GitHub Action workflow. - Data Processing: The action extracts and validates the subscription data from the issue body, encrypts the email address using AES-256, and stores the subscriber information in the
subscribers.encrypted.jsonfile. - Welcome Email & Cleanup: A welcome email is sent to the new subscriber via the Resend API. The GitHub issue is then commented on, closed, and scheduled for automatic deletion to protect user privacy.
Sources: themes/matrix-flow/source/js/newsletter.js:389-411, themes/matrix-flow/README-newsletter.md:21-36, newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:104-107
This sequence diagram illustrates the user subscription flow, from form submission to final processing by the GitHub Action.
Sources: themes/matrix-flow/source/js/newsletter.js, themes/matrix-flow/README-newsletter.md:21-36, newsletter-program/tests/integration/newsletter-subscription-flow.test.js
Newsletter Management
Newsletters are managed as Markdown files with YAML frontmatter. The entire lifecycle, from creation to archival, is handled through a dedicated CLI.
Newsletter File Format
Each newsletter is a .md file containing YAML frontmatter for metadata and Markdown for the content.
| Field | Description | Status |
|---|---|---|
subject | The email subject line. | Required |
date | The display date for the newsletter. | Required |
status | The current state of the newsletter: draft, scheduled, or sent. | Required |
scheduled_at | An ISO 8601 datetime for when the newsletter should be sent. | Optional |
website_url | Link to the main website. | Optional |
github_url | Link to the GitHub profile/repository. | Optional |
preferences_url | Link for users to manage their subscription preferences. | Optional |
unsubscribe_url | Link for users to unsubscribe. | Optional |
variables | A key-value map for custom variables to be used in the template. | Optional |
Sources: newsletters/README.md:15-32, newsletters/newsletter-template.md:2-12
Newsletter Lifecycle
The state of a newsletter progresses from draft to sent and is managed by CLI commands.
This flowchart shows the lifecycle of a newsletter from creation to archival.
Sources: newsletters/README.md:35-64, newsletter-program/src/cli/newsletter-cli.js
Core Components
The backend logic is modularized into several core JavaScript classes.
NewsletterComposer
This class is responsible for handling the newsletter files themselves.
parseNewsletterFile(filePath): Reads a markdown file, parses the YAML frontmatter and the markdown content.loadTemplate(templateName, variables): Loads an HTML template (e.g.,newsletter.html) and injects variables.composeNewsletter(newsletterFile): Combines a parsed newsletter file with an HTML template to create the final email body.generatePreview(newsletterFile): Creates a local HTML file for previewing a newsletter.archiveNewsletter(newsletterFile): Copies a sent newsletter file to thearchive/directory with a timestamp.createNewsletterTemplate(fileName, options): Creates a new newsletter.mdfile from a template.
Sources: newsletter-program/src/core/newsletter-composer.js
AnalyticsManager
This class handles tracking and reporting on subscriber and newsletter metrics.
recordNewsletterSent(newsletterData): Logs an event when a newsletter is sent, including recipient counts.recordSubscriberEvent(eventType, subscriberData): Logs events likesubscribe,unsubscribe, etc.generateGrowthReport(days): Creates a report on subscriber growth over a given period.generateNewsletterReport(days): Reports on the performance of recently sent newsletters.generateEngagementReport(): Analyzes subscriber engagement based on source, preferences, and churn rate.
Sources: newsletter-program/src/core/analytics-manager.js, newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:146-180
SubscriberManager
This component is central to managing the subscriber list securely. It handles all CRUD operations for subscribers, ensuring that sensitive data like email addresses are always encrypted at rest in the subscribers.encrypted.json file. It uses AES-256 for encryption.
Sources: newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:8-12, 28-50
Data Models
The system relies on a file-based storage system with well-defined data models.
Data Relationship
This ER diagram shows the relationship between a Subscriber and the Analytics Events they can generate.
Sources: newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:31-50, 53-63
Subscriber Model
| Field | Type | Description |
|---|---|---|
id | string | Unique identifier (UUID). |
encryptedEmail | object | An object containing the encrypted email, IV, and key version. |
status | string | active, unsubscribed, or bounced. |
subscribedAt | datetime | ISO 8601 timestamp of subscription. |
source | string | Source of subscription (e.g., website, import). |
preferences | object | Contains frequency and topics array. |
unsubscribeToken | string | A cryptographically secure token for unsubscribing. |
Sources: newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:31-50
Command-Line Interface (CLI)
The newsletter-program provides a powerful set of CLI tools to manage the entire system. The main entry points are exposed as scripts in package.json.
| Command | Description |
|---|---|
npm run newsletter create <name> | Creates a new newsletter file from a template. |
npm run newsletter preview <file> | Generates an HTML preview of a newsletter. |
npm run newsletter list | Lists all available newsletters, with filtering options. |
npm run newsletter schedule <file> <datetime> | Sets the scheduled_at and status fields in a newsletter's frontmatter. |
npm run newsletter send <file> | Sends a newsletter to a test email or all active subscribers. |
npm run subscribers <add|list|update|remove> | Manages the encrypted subscriber list. |
npm run analytics <growth|newsletters|engagement> | Generates and displays various analytics reports. |
npm run rss <add|list|check> | Manages the RSS feed monitoring configuration. |
Sources: newsletter-program/package.json:13-18, newsletters/README.md:35-64, newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:66-143
Security Features
Security is a core principle of the newsletter system.
- Email Encryption: All subscriber email addresses are encrypted at rest using AES-256-CBC. The encryption key is managed via environment variables.
- Secure Unsubscribe: Unsubscribe links use cryptographically secure tokens to prevent unauthorized actions.
- GitHub Issue Cleanup: To protect user privacy, GitHub issues created during subscription are automatically closed and deleted after 24 hours.
- API Key Management: Sensitive API keys (e.g.,
RESEND_API_KEY) are stored as GitHub Secrets and are not exposed in the codebase.
Sources: newsletter-program/README-SUBSCRIBER-MANAGEMENT.md:100-111, themes/matrix-flow/README-newsletter.md:25-36, 42-47