Collecting GitHub Releases
Relevant source files
The following files were used as context for generating this wiki page:The "Collecting GitHub Releases" feature is a core component of the Node.js News Feeder, designed to automatically aggregate software release announcements from various GitHub repositories. Its primary purpose is to poll repositories associated with a specific GitHub user, identify new releases published since the last check, format them into a standardized RSS item structure, and inject them into the project's main feed.xml file.
This process is one of several data collection methods used by the project, which also include gathering information from issues, discussions, and manual posts. The overall system is typically orchestrated by a GitHub Action that runs on a schedule or can be triggered manually, ensuring the news feed remains up-to-date with the latest developments.
Sources: scripts/collect-releases.js, README.md:23-30, package.json:8-12
Configuration
The behavior of the release collection script is governed by parameters defined in config.json. The script retrieves these values at runtime to control its execution, such as API pagination limits and the timestamp for filtering new content.
Sources: scripts/collect-releases.js:4, config.json:2-6, utils/index.js:125-127
| Parameter | Type | Description |
|---|---|---|
lastCheckTimestamp | Number | A Unix timestamp (in milliseconds) indicating the last time the feed was updated. Releases published after this time are considered new. |
reposPaginationLimit | Number | The maximum number of repositories to fetch from the user's account. |
releasePaginationLimit | Number | The maximum number of releases to fetch from each repository. |
breakDelimiter | String | A specific string (</image>) in feed.xml that acts as a marker for injecting new feed items. |
Sources: config.json:2-9, scripts/collect-releases.js:4
Execution Flow
The process is executed by running the scripts/collect-releases.js script, which can be invoked via the npm run collect:releases command. The script requires a GITHUB_TOKEN environment variable for authenticating with the GitHub API.
Sources: package.json:8, scripts/collect-releases.js:7-12
The following diagram illustrates the step-by-step process of collecting and processing GitHub releases.
Sources: scripts/collect-releases.js
1. Fetch Repositories
The script begins by fetching a list of all public repositories belonging to the user thomas-iniguez-visioli. It uses the gh-got library to paginate through the results, respecting the reposPaginationLimit set in config.json.
Sources: scripts/collect-releases.js:7-12
2. Fetch Releases
For each repository retrieved, the script makes another API call to fetch its releases. The releasePaginationLimit from the configuration is used to limit the number of releases fetched per repository.
Sources: scripts/collect-releases.js:15-22
3. Filter and Format
The collected releases are then processed:
- Filtering: Releases are filtered to include only those with a
published_atdate more recent than thelastCheckTimestampfromconfig.json. - Mapping: The repository's full name (
repo.full_name) is added to each release object for context. - Composing: The final list of relevant releases is flattened and mapped into RSS
<item>XML strings. Each item is constructed using thecomposeFeedItemutility.
The title and description are dynamically generated. For example, the title is formatted as Released {repo_name} {tag_name}. The description is converted from Markdown to HTML and includes a link to the release on GitHub.
// scripts/collect-releases.js:33-39
const relevantReleases = releases.flat().map(rel => composeFeedItem({
title: `Released ${rel.repo} ${rel.tag_name}`,
description: `<`)}]]>`,
pubDate: buildRFC822Date(rel.published_at),
link: rel.html_url,
guid: rel.html_url
})).join('')
Sources: scripts/collect-releases.js:25-39
4. Update Feed File
Finally, the script reads the existing feed.xml, splits its content using the breakDelimiter (</image>), and inserts the newly generated release items.
Note: The final call to overwriteFeedContent(updatedFeedContent) is commented out in the source file, meaning that as written, the script prepares the updated content but does not write it back to feed.xml. This suggests it may be part of a larger workflow where the output is handled differently or the write operation is enabled under specific conditions.
Sources: scripts/collect-releases.js:41-44
Data Flow
The data flow involves interactions between the script, the local file system for configuration and the feed file, and the external GitHub API.
This sequence diagram shows the interactions between the components during the release collection process.
Sources: scripts/collect-releases.js, utils/index.js
Core Utilities
The collect-releases.js script relies on several helper functions from utils/index.js to perform its tasks.
| Function | Description |
|---|---|
getConfig() | Reads and parses config.json into a JavaScript object. |
md2html() | Converts a Markdown string to an HTML string using remark. |
buildRFC822Date() | Formats a date string into the RFC 822 format required for RSS feeds. |
composeFeedItem() | Constructs a single RSS <item> block from title, description, and other data. |
getFeedContent() | Reads the entire content of feed.xml as a string. |
overwriteFeedContent() | Writes a string content back to the feed.xml file. |