Monitoring, Logging, & Analytics Logging
Relevant source files
The following files were used as context for generating this wiki page:
GoGovSG implements a multi-layered monitoring and analytics architecture to ensure system reliability, track performance, and provide user-facing link statistics. The system integrates server-side telemetry via Datadog and Winston with client-side tracking through Google Analytics and Datadog Real User Monitoring (RUM).
This infrastructure tracks critical application events such as link creation, user authentication, and malicious activity detection. It also powers the "Link Analytics" dashboard for users, providing insights into shortlink popularity, device types, and traffic patterns.
Server-Side Monitoring & Metrics
The backend uses dogstatsd (via the hot-shots library) to send custom metrics to Datadog. This allows the team to monitor application-specific events in real-time.
DogStatsD Metrics
The system defines several metric constants to track successes and failures across various modules.
| Metric Name | Description |
|---|---|
go.shortlink.clicks | Tracks the number of times shortlinks are accessed. |
go.shortlink.create | Tracks link creation, tagged by source and file type. |
go.otp.generate.success / failure | Monitors the health of the One-Time Password system. |
go.malicious_activity.link / file | Logs detections of unsafe links or files. |
go.error.unhandled_rejection | Tracks unhandled promise rejections for stability monitoring. |
go.bulk.hash.success / failure | Monitors bulk link creation jobs. |
Sources: src/server/util/dogstatsd.ts:4-44, src/server/index.ts:103-103
Distributed Tracing
The application initializes dd-trace at the absolute top of the server entry point to enable automatic instrumentation and distributed tracing. This provides visibility into request lifecycles and database query performance.
Sources: src/server/index.ts:1, package.json:54
Application Logging
GoGovSG uses Winston for general application logging and Morgan for HTTP request logging.
HTTP Request Logging
Morgan captures detailed information about every incoming request and writes it to the Winston transport. The log format includes:
- Client IP (derived from
X-Forwarded-Foriftrust proxyis enabled). - HTTP Method, URL, and Status Code.
- Redirect URL (for 302 responses).
- User ID (if the session is authenticated).
- Response time in milliseconds.
Sources: src/server/index.ts:106-130
Winston Configuration
The logger operates at the info level in production and debug level in development. It outputs JSON-formatted logs with timestamps to the console.
Sources: src/server/config.ts:49-74
The diagram shows the flow of HTTP request metadata from the Express middleware to the log aggregation service. Sources: src/server/index.ts:130-132, README.md:144-148
Client-Side Analytics
The frontend tracks user interactions to improve the user experience and provide link-specific analytics.
Google Analytics (GA)
The client application uses react-ga to track page views and specific events.
- Page Views: Tracked for the "API Integration" page and the "User Dashboard".
- Events: Tracked when users interact with the Link Analytics drawer, specifically when switching between "Device", "Clicks", and "Traffic" tabs.
Sources: src/client/user/components/Drawer/ControlPanel/LinkAnalytics/index.tsx:162-184, src/client/apiintegration/index.tsx:50-54
Datadog RUM
The application integrates Datadog Browser RUM for monitoring frontend performance and errors. The Content Security Policy (CSP) is explicitly configured to allow connections to Datadog's intake origins.
Sources: src/server/index.ts:137-141, package.json:44
User-Facing Link Analytics
Users can view real-time statistics for their shortlinks through the Link Analytics module. This data is visualized using chart.js.
Data Dimensions
The Link Analytics dashboard categorizes click data into three primary views:
- Devices: Visualizes clicks by device type (e.g., Mobile vs. Desktop).
- Clicks: Shows daily click trends over time.
- Traffic: Displays a heatmap of clicks by day of the week.
Sources: src/client/user/components/Drawer/ControlPanel/LinkAnalytics/index.tsx:210-245
The sequence shows how the frontend retrieves click statistics to render charts. Sources: src/client/user/components/Drawer/ControlPanel/LinkAnalytics/index.tsx:265-275, README.md:80-92
Infrastructure Configuration
Monitoring behavior is controlled by environment variables set during deployment.
| Variable | Required | Function |
|---|---|---|
DD_API_KEY | Yes (Prod) | Enables Datadog integration for traces and logs. |
GA_TRACKING_ID | No | Enables Google Analytics tracking on the client. |
CSP_REPORT_URI | No | Defines where the browser sends Content Security Policy violation reports. |
REDIS_STAT_URI | Yes | Connection string for the Redis instance storing click statistics. |
Sources: src/server/config.ts:13-33, README.md:144-148
Conclusion
GoGovSG employs a robust telemetry stack combining Datadog for system-level observability and Google Analytics for user behavior tracking. By utilizing dogstatsd for custom metrics and Winston/Morgan for structured logging, the architecture provides developers with the necessary tools to maintain high availability and security for Singapore's official link shortener.