Threat Scanning & Safe Browsing
Relevant source files
The following files were used as context for generating this wiki page:
The Threat Scanning and Safe Browsing system protects users from malicious content by validating long URLs and uploaded files. It integrates with external security services to detect malware, social engineering, and unwanted software before performing a redirect or completing a file upload.
The system primary functions within the redirection flow and the URL creation process. It utilizes Google Web Risk (via SafeBrowsingService) for URL reputation and Cloudmersive (via CloudmersiveScanService) for file-based virus scanning.
Architecture and Components
The system consists of specialized services that implement scanning interfaces. These services interact with external APIs and local repositories to cache results and enforce security policies.
Core Services
| Component | Description |
|---|---|
RedirectService | Orchestrates the redirection flow and triggers safe browsing checks if cached results are expired. |
SafeBrowsingService | Interfaces with Google Web Risk API to check if a URL is a known threat. |
CloudmersiveScanService | Interfaces with Cloudmersive API to scan files for viruses and check URL safety. |
SafeBrowsingRepository | Persists and retrieves scan results to minimize redundant API calls. |
Sources: src/server/modules/redirect/services/RedirectService.ts, src/server/modules/threat/services/SafeBrowsingService.ts
Threat Detection Logic
The following diagram illustrates how the system evaluates a URL during a redirect request.
The system treats detected threats as non-existent links to avoid user panic while ensuring security.
Sources: src/server/modules/redirect/services/RedirectService.ts:63-125
Safe Browsing Integration
SafeBrowsingService utilizes the Google Web Risk API to categorize URLs. It specifically checks for SOCIAL_ENGINEERING, MALWARE, UNWANTED_SOFTWARE, and SOCIAL_ENGINEERING_EXTENDED_COVERAGE.
Request Configuration
Requests to the Google Web Risk API are configured with specific headers and timeouts to ensure system stability.
- Endpoint:
https://webrisk.googleapis.com/v1/uris:search - Timeout: 5000ms (
WEB_RISK_TIMEOUT_MS) - Authentication: API Key provided in the
X-Goog-Api-Keyheader. - Retry Logic: The service retries once on transient errors (HTTP 5xx or connection resets).
Sources: src/server/modules/threat/services/SafeBrowsingService.ts:16-25, src/server/modules/threat/services/SafeBrowsingService.ts:77-101
Scan Execution Sequence
The sequence diagram below shows the interaction between the redirect service, the threat service, and the external API.
File and Advanced Scanning
CloudmersiveScanService provides virus scanning for files and an alternative URL threat detection mechanism.
File Scanning
When a file is uploaded, the service performs an advanced scan to detect viruses and check if the file is password-protected.
scanFile(file): Sends file buffer to Cloudmersive.- Advanced Scanning: Returns
FoundViruses(list) andContainsPasswordProtectedFile(boolean).
URL Scanning
While CloudmersiveScanService includes isThreat functionality, the project documentation indicates this has been largely superseded by Google Safe Browsing due to Cloudmersive's high sensitivity. The implementation currently returns false for URL scans to prevent false positives while remaining available as a reference.
Sources: src/server/modules/threat/services/CloudmersiveScanService.ts:25-56, src/server/modules/threat/services/CloudmersiveScanService.ts:73-90
Configuration and Error Handling
The system behavior is controlled by environment variables and specific error handling strategies to prevent leaking sensitive information.
Environment Variables
| Variable | Type | Description |
|---|---|---|
SAFE_BROWSING_KEY | String | API key for Google Web Risk. If missing, scans are skipped. |
SAFE_BROWSING_LOG_ONLY | Boolean | If true, threats are logged but redirections are not blocked. |
CLOUDMERSIVE_KEY | String | API key for Cloudmersive file scanning. |
Sources: src/server/modules/threat/services/SafeBrowsingService.ts:52-64, README.md:148-151
Security and Redaction
To prevent sensitive data from appearing in logs, the RedirectService implements a redactApiKey method. This method strips the SAFE_BROWSING_KEY from error messages produced by network-level failures (e.g., FetchError) before they are sent to the logger.
Sources: src/server/modules/redirect/services/RedirectService.ts:182-187
Fail-Open Policy
The RedirectService is designed to fail open during external API outages. If the isThreat call throws an error (e.g., a "Bad Gateway" from Google), the system logs the error and allows the redirect to proceed to ensure availability, but it does not update the safe browsing expiry in the database. This ensures the URL is re-scanned upon the next visit.
Sources: src/server/modules/redirect/services/RedirectService.ts:84-102, src/server/modules/threat/services/tests/SafeBrowsingService.test.ts:245-258
Summary
Threat Scanning and Safe Browsing in GoGovSG provide a critical security layer by intercepting malicious URLs and files. By combining Google Web Risk for URL reputation and Cloudmersive for file integrity, the system ensures that shortened links remain a "trusted" source for citizens. The implementation prioritizes security through automatic deactivation of malicious links while maintaining high availability through fail-open mechanisms and result caching.