Validating the RSS Feed

The RSS feed validation is a critical quality assurance step in this project. Its primary purpose is to ensure that the generated feed.xml file adheres to the official RSS 2.0 specification. This is achieved by submitting the feed's content to the external W3C Feed Validation Service and parsing the result. A valid feed is essential for compatibility with various news readers, aggregators, and other services that consume the feed, such as the project's Slack channel.

The validation process is encapsulated in a script that can be run manually to check the integrity of the feed.xml file. This helps curators and developers confirm that changes made during the content aggregation and build process have not introduced any syntax errors or structural issues.

Sources: README.md:4, 48-51, scripts/validate.js

Execution

The validation process is executed via an npm script.

Sources: package.json:10, README.md:48-51

NPM Script

To run the validation, use the following command from the project's root directory:

npm run rss:validate

This command executes the scripts/validate.js Node.js script.

Sources: package.json:10, README.md:51

Sources: package.json:10-12, README.md:42-51

Validation Architecture

The validation mechanism is not self-contained; it relies on an external service and is supported by internal data sanitization and formatting utilities.

Core Validation Process

The scripts/validate.js script orchestrates the validation. It reads the local feed.xml, sends its content to the W3C validator, and interprets the HTML response to determine if the feed is valid.

This diagram illustrates the data flow for the validation script.

The sequence diagram below shows the interaction between the local script and the remote W3C service.

Pre-Validation and Sanitization

Before the final validation occurs, several scripts and utility functions prepare the feed's content. This "pre-validation" ensures the data is clean, correctly formatted, and free of duplicates, increasing the likelihood of passing the W3C check.

Formatting and Filtering

The npm run rss:format script, which executes scripts/format.js, plays a key role. It parses the existing feed.xml, applies filtering logic, sorts the items by date, and then overwrites the file with the cleaned and formatted content.

The core of this process is the filterFeedItems utility function, which performs the following checks:

1. Excludes Incomplete Entries: Items without a title, link, or pubDate are removed.
2. Removes Duplicates: It ensures uniqueness based on guid, link, or title.

Sources: scripts/format.js:24-28, utils/index.js:210-223

Content Cleaning

The utils/index.js file contains several helper functions to sanitize content before it's inserted into the XML structure. This prevents malformed XML and invalid characters.

Sources: utils/index.js:29-40, 50-60, 99-114, 210-223, utils/tests/utils.test.js

The escapeXmlCharacters function is particularly important for ensuring that content within XML tags does not break the document structure. It replaces special characters with their corresponding XML entities.

// utils/index.js:29-40
export function escapeXmlCharacters(text) {
  if (typeof text !== 'string') {
    return text
  }

  return text.replace(/[&<>"']/g, (match) => xmlEscapeMap[match])
}

Sources: utils/index.js:29-40, utils/tests/utils.test.js:21-43

Validation Logic

The core validation logic resides in scripts/validate.js. It uses the got library to send the content of feed.xml as form data to https://validator.w3.org/feed/check.cgi.

The script then uses jsdom to parse the HTML response from the validator. It determines the feed's validity by inspecting the <h2> heading of the returned page. If the heading's text is "Sorry", the feed is considered invalid. Otherwise, it is considered valid. Any recommendations or errors provided by the validator are logged to the console.

// scripts/validate.js:8-25
try {
  const data = await got.post('https://validator.w3.org/feed/check.cgi', {
    form: {
      rawdata: xml,
      manual: 1
    }
  }).text()

  // Avoid importing CSS in the document
  const dom = new JSDOM(data.replace(/@import.*/gm, ''))

  const title = dom.window.document.querySelector('h2').textContent
  const recommendations = dom.window.document.querySelector('ul').textContent

  console.log(recommendations)

  if (title === 'Sorry') {
    console.log('🚨 Feed is invalid!')
  //  process.exit(1)
  } else {
    console.log('✅ Feed is valid!')
  }
} catch (error) {
  console.log('Service is down')
  console.error(error.response.statusCode)
}

Sources: scripts/validate.js:8-25

Summary

The RSS feed validation is a multi-stage process. It begins with content sanitization and filtering to prepare a clean dataset. This is followed by a formatting step that sorts and structures the feed items. The final validation step uses the external W3C service to perform a comprehensive check on the feed.xml file, ensuring it meets the required standards for broad compatibility. This entire workflow guarantees the reliability and quality of the news feed provided by the project.

Sources: README.md:48-51, scripts/validate.js, scripts/format.js